A method for progressive domain fine-tuning and knowledge fusion of large language models for tunnel boring machines.

Abstract

This invention discloses a progressive domain fine-tuning and knowledge fusion method for a large language model in tunnel boring machine (TBM) engineering. The method includes: constructing a hierarchical TBM training curriculum encompassing broad-domain academic theories and proprietary enterprise construction methods; training multiple parameter-efficient adapters with physical isolation based on a frozen base in parallel; performing singular value decomposition on the adapters to extract geometric feature subspaces representing knowledge distribution and calculating conflict correlations; and constructing a resource-aware unified adaptation mechanism based on this. This mechanism can generate conflict-free static fusion models or dynamically activate specific rank slices of the adapters through a routing network based on real-time hardware resource budgets (GPU memory / FLOPs) and geometry-resource signatures. This invention achieves adaptive dynamic scheduling of edge hardware resources for model inference while preserving multi-source knowledge.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically a method for progressive domain fine-tuning and knowledge fusion of large language models for tunnel boring machine (TBM) engineering. Background Technology

[0002] Tunnel boring machine (TBM) engineering, as one of the core methods for underground space development, is a knowledge-intensive industry integrating multiple disciplines such as mechanics, civil engineering, and geology. With the expansion of project scale and the increasing complexity of geological conditions, leveraging artificial intelligence to assist engineers in quickly acquiring broad academic theoretical support (such as geomechanical principles) and precise enterprise-specific construction method guidance (such as parameter settings for specific TBM models) has become crucial for improving construction safety and efficiency. Large-scale language models (LLMs), with their powerful semantic understanding and generation capabilities, provide a new technical approach for processing such multi-source heterogeneous knowledge.

[0003] Currently, domain-specific model fine-tuning primarily employs techniques such as full-parameter fine-tuning or parameter-efficient fine-tuning (e.g., low-rank adapter LoRA, Adapter). When handling multi-source knowledge, a sequential fine-tuning strategy is typically used, i.e., training first on general data and then fine-tuning on domain-specific data; or a simple data mixing strategy for unified training. During model deployment, the mainstream approach is to statically merge the fine-tuned parameters with the base model parameters, or to load fixed adapter weights during inference. Furthermore, to address multi-task interference, dynamic routing techniques based on hybrid expert modules (MoEs) have emerged, typically selecting and activating different expert modules based on the semantic features of the input text.

[0004] However, in complex vertical applications such as tunnel boring machines, existing technologies still face the technical challenge of balancing the integrity of knowledge preservation with the adaptability of hardware resources. Specifically, these challenges manifest in the following ways:

[0005] Geometric conflicts and catastrophic forgetting in heterogeneous knowledge feature spaces: Academic theories in the field of tunnel boring machines tend to be logical deductions on high-dimensional manifolds, while enterprise empirical parameters tend to be numerical fittings within low-dimensional subspaces. Existing sequential fine-tuning or simple linear merging strategies ignore the differences in geometric orthogonality between these two types of knowledge in the feature space, leading to the later-obtained empirical parameters covering the earlier-obtained theoretical logic, or generating feature interference during merging, triggering catastrophic forgetting.

[0006] Lack of resource-aware dynamic reasoning adaptation mechanisms: Computing equipment at tunnel boring machine (TBM) construction sites, such as industrial control computers and edge servers, has extremely limited resources and fluctuates significantly in load. Existing technologies typically generate static, fixed-rank models or adapters that cannot dynamically adjust the model's activation scale based on real-time memory availability or computational latency estimates. This results in models failing to run when resources are scarce, or being unable to utilize additional computing power to improve accuracy when resources are plentiful, making it difficult to meet the dual demands of high availability and high performance on construction sites. Summary of the Invention

[0007] The purpose of this invention is to address the problems of catastrophic forgetting and lack of resource adaptability in existing fine-tuning methods, and to provide a progressive domain fine-tuning and knowledge fusion method for large language models in tunnel boring machine (TBM) engineering.

[0008] The technical solution provides a method for progressive domain fine-tuning and knowledge fusion of a large language model for tunnel boring machine (TBM) engineering, including the following steps:

[0009] A layered shield tunneling training course was constructed, which includes both broad-area theory and proprietary construction methods. The course consists of a set of basic knowledge derived from academic literature and a set of professional knowledge derived from on-site construction records.

[0010] Based on the frozen pre-trained model, the first parameter high-efficiency adapter and the second parameter high-efficiency adapter are trained in parallel using the hierarchical shield tunneling training course, and the theoretical features and engineering parameter features of the shield tunneling are encoded in a physically isolated manner.

[0011] Singular value decomposition is performed on the weight matrices of the first-parameter efficient adapter and the second-parameter efficient adapter to extract the geometric feature subspaces representing the knowledge distribution, and the orthogonality and conflict correlation between the geometric feature subspaces are calculated; optionally, in some embodiments, orthogonality and conflict correlation are mainly represented by cosine similarity.

[0012] Based on the geometric feature subspace and its conflict correlation, a unified adaptation mechanism for resource awareness is constructed. This mechanism is used in shield tunneling engineering reasoning tasks to coordinate the activation state or fusion form of the first parameter efficient adapter and the second parameter efficient adapter according to computational resource constraints.

[0013] Optionally, a geometric feature subspace representing the knowledge distribution is extracted, and the orthogonality and conflict correlation between the geometric feature subspaces are calculated, including:

[0014] For the first-parameter efficient adapter and the second-parameter efficient adapter after singular value decomposition, the singular value energy distribution vector and the principal subspace direction projection are extracted respectively, and the low-rank matrix of the adapter is divided into several continuous rank slices.

[0015] For each rank slice, calculate the corresponding number of floating-point operations and memory usage based on its matrix dimension, and construct a rank slice resource overhead mapping.

[0016] By combining the singular value energy distribution vector, the cosine similarity between the principal subspace directional projections, and the rank slice resource overhead mapping, a geometric-resource signature table is generated to guide dynamic reasoning decisions.

[0017] Optionally, based on the geometric feature subspace and its conflict correlation degree, a unified adaptation mechanism for resource awareness is constructed, including:

[0018] It receives natural language input requests for shield tunneling engineering inference tasks and obtains the real-time resource budget vector of the current inference environment. The real-time resource budget vector defines the maximum allowable memory increment and computation latency level.

[0019] Extract the semantic fingerprint of the natural language input request, and input it together with the feature terms in the geometry-resource signature table and the real-time resource budget vector into a pre-trained routing network;

[0020] The routing network outputs a dynamic activation mask for each rank slice in the first-parameter efficient adapter and the second-parameter efficient adapter. The dynamic activation mask is used to indicate the set of target rank slices that need to be loaded into video memory and participate in forward propagation computation under the current resource budget constraints.

[0021] Optionally, the routing network is optimized by training on a mixed validation set to minimize a joint loss function, the construction of which includes:

[0022] The sample input from the hybrid validation set is passed through a model configured with a dynamic activation mask to calculate the prediction error loss for a predetermined domain task.

[0023] Based on the rank slice selected by the dynamic activation mask, combined with the rank slice resource overhead mapping in the geometry-resource signature table, the total video memory usage and floating-point operation volume in the current active state are calculated, and a resource penalty term is constructed based on the degree to which it exceeds the real-time resource budget vector.

[0024] The prediction error loss is weighted and summed with the resource penalty term to form a joint loss function for updating the routing network parameters, so that the routing network can meet the computational resource constraints while maintaining prediction accuracy.

[0025] Optionally, before inputting the natural language input request into the routing network, the method further includes coarse-grained task routing and cache reuse steps:

[0026] Parse natural language input requests to identify their corresponding coarse-grained task types and operational stage labels;

[0027] If the coarse-grained task type is determined to be a single knowledge source request, then all rank slices of the corresponding first-parameter efficient adapter or second-parameter efficient adapter are directly activated, skipping the calculation of the routing network.

[0028] If the coarse-grained task type is determined to be a mixed knowledge request, then the pre-built route cache table is queried using the coarse-grained task type and the working condition stage label as the index key.

[0029] If a historical record is found in the routing cache table, the dynamic activation mask stored in the historical record is read and reused directly; if no record is found, the routing network is calculated and the newly generated dynamic activation mask is written to the routing cache table.

[0030] Optionally, a layered shield tunneling training curriculum can be constructed, incorporating both broad-area theory and proprietary construction methods, including:

[0031] Using a general large language model as the teacher model, text fragments were extracted from academic literature and construction site records, and question-answer pairs were automatically generated.

[0032] For each generated question-and-answer pair, the task type label and shield tunneling stage label are automatically labeled according to its source text content. The shield tunneling stage label covers at least the survey, tunneling and maintenance stages.

[0033] The question-answer pairs carrying task type labels and shield tunneling condition stage labels will be organized into basic knowledge sets and professional knowledge sets, respectively, for use in subsequent parallel training and routing network construction.

[0034] Optionally, a unified resource-aware adaptation mechanism can be constructed, including:

[0035] The conflict correlation between geometric feature subspaces is concretized into a cosine similarity matrix of feature directions, and a preset similarity threshold is introduced.

[0036] The elements in the cosine similarity matrix are compared with the similarity threshold to generate a binary conflict mask matrix. The conflict mask matrix is ​​used to divide the geometric feature subspace into orthogonal independent knowledge subspaces and overlapping knowledge conflict subspaces.

[0037] Based on the conflict mask matrix, the first-parameter high-efficiency adapter and the second-parameter high-efficiency adapter are merged into a fixed final unified adapter, achieving resource awareness without increasing inference memory usage.

[0038] Optionally, based on the collision mask matrix, the first-parameter efficient adapter and the second-parameter efficient adapter are merged, including:

[0039] For the independent knowledge subspace indicated by the conflict mask matrix, the original projections of the first-parameter efficient adapter and the second-parameter efficient adapter on the corresponding feature directions are directly retained as the orthogonal retained components of the final unified adapter;

[0040] For the knowledge conflict subspace indicated by the conflict mask matrix, we introduce fusion coefficients to be optimized and construct candidate fusion feature projections;

[0041] Using a validation set containing mixed academic and enterprise data, the predicted loss gradient is calculated by projecting candidate fusion features, and the fusion coefficients are dynamically adjusted by gradient descent until the predicted loss converges, thus obtaining the optimized fusion components of the final unified adapter.

[0042] The orthogonal preserved components and the optimized fusion components are superimposed to form the weight parameters of the final unified adapter.

[0043] Optionally, based on the frozen pre-trained model, the first-parameter high-efficiency adapter and the second-parameter high-efficiency adapter are trained in parallel using a hierarchical shield tunneling training curriculum, including:

[0044] Keep the original weight matrix of the base pre-trained model frozen and not involved in gradient updates;

[0045] Construct a first low-rank matrix pair and a second low-rank matrix pair respectively. Each low-rank matrix pair is formed by multiplying a dimension-reduced matrix and an dimension-increasing matrix, and is initialized to an independent parameter space that does not interfere with each other.

[0046] The first low-rank matrix pair is updated by backpropagation using the basic knowledge set to obtain the first parameter efficient adapter. The second low-rank matrix pair is updated by backpropagation using the professional knowledge set to obtain the second parameter efficient adapter, ensuring the physical isolation between academic theoretical features and engineering parameter features on the parameter update path.

[0047] Optionally, singular value decomposition is performed on the weight matrices of the first-parameter efficient adapter and the second-parameter efficient adapter to extract a geometric feature subspace representing the knowledge distribution, including:

[0048] Perform full singular value decomposition on the weight matrix to obtain the complete sequence of singular values ​​and the corresponding left and right singular vectors;

[0049] The sum of squares of the singular value sequence is calculated as the total energy, and the minimum truncation rank that can cover the main knowledge information is adaptively determined based on the preset cumulative energy threshold.

[0050] The left and right singular vectors are truncated according to the minimum truncation rank, and the main feature directions of the preceding sequence are retained as the geometric feature subspace. This filters out training noise and reduces the computational dimension of the subsequent construction of the unified adaptation mechanism.

[0051] Beneficial effects: This invention, while preserving multi-source knowledge, achieves adaptive dynamic scheduling of edge hardware resources for model inference. This invention addresses the forgetting problem through a dual mechanism of physical isolation and geometric conflict resolution.

[0052] During the training phase, this invention employs a parallel training architecture, constructing independent low-rank matrix pairs for academic theoretical knowledge and enterprise construction method knowledge, respectively. The two sets of adapter parameters occupy isolated storage spaces in memory and are updated independently during backpropagation, cutting off the mutual interference paths between different domain knowledge during parameter updates. This design ensures that the model does not overwrite or modify the encoded academic theoretical parameters when learning enterprise-specific construction methods.

[0053] During the fusion phase, this invention extracts the geometric feature subspaces of each adapter through singular value decomposition and quantifies the degree of conflict between different knowledge sources in the feature directions based on cosine similarity. For orthogonal independent knowledge subspaces, they are directly superimposed to preserve complete information; for conflicting subspaces with overlap, the optimal fusion coefficients are found through gradient optimization. This geometric prior-based processing avoids feature interference caused by simple linear merging, allowing the logical deduction capability of academic theories to coexist with the numerical fitting capability of enterprise parameters, thus achieving non-destructive fusion of multi-source heterogeneous knowledge.

[0054] This invention achieves adaptive resource scheduling through a geometry-resource signature table and a dynamic rank routing mechanism.

[0055] During the offline phase, this invention divides the adapter's low-rank matrix into several consecutive rank slices and pre-calculates the number of floating-point operations and memory usage for each slice to construct a rank slice resource overhead mapping. This mapping, together with information such as singular value energy distribution and subspace conflict degree, forms a lightweight geometry-resource signature table, which resides in the inference server's memory.

[0056] During the inference phase, the routing network dynamically outputs activation masks for each rank slice based on the semantic features of the input request and the real-time hardware resource budget. When memory is sufficient, the system activates more high-energy rank slices to achieve optimal accuracy; when resources are limited, it only loads the main feature subspace most relevant to the current task, ensuring availability while meeting hardware constraints. Combining coarse-grained task offloading and routing cache reuse mechanisms, the system can handle approximately 70% to 80% of routine queries at tunnel boring machine (TBM) construction sites without calling the routing network, reducing average response latency. This elastic scalability enables large language models to run stably on edge devices with limited computing power and fluctuating loads. Attached Figure Description

[0057] Figure 1 This is a flowchart of this application.

[0058] Figure 2This is a flowchart for calculating the orthogonality and conflict correlation between geometric feature subspaces in this application.

[0059] Figure 3 A flowchart for constructing a resource-aware unified adaptation mechanism for this application.

[0060] Figure 4 A flowchart for constructing the joint loss function for this application.

[0061] Figure 5 This is a flowchart of the dataset construction process for this application.

[0062] Figure 6 This is a functional module architecture diagram of an embodiment of this application. Detailed Implementation

[0063] Combination Figures 1 to 6 This describes the implementation details of this application.

[0064] Example 1 describes how to solve the conflict and resource scheduling problems in multi-source knowledge fusion by using phased learning and feature subspace analysis for the complex knowledge system of tunnel boring machine (TBM) engineering.

[0065] Step 101: Construct a layered shield tunneling training course that includes wide-area theory and proprietary construction methods. The course consists of a set of basic knowledge derived from academic literature and a set of professional knowledge derived from construction site records.

[0066] The course includes: a first dataset obtained from the first knowledge source, broad-area shield tunneling-related academic literature, as the basic knowledge set; and a second dataset obtained from the second knowledge source, proprietary enterprise documents such as shield tunneling company construction manuals, geotechnical index reports, and shield machine design documents, as the professional knowledge set.

[0067] Wide-area theory refers to general, non-exclusive public knowledge applicable within the field of tunnel boring machine (TBM) engineering. This type of knowledge is typically distributed across a wide range of academic publications, covering fundamental principles of soil mechanics, the basic structure of TBMs, and common geological risk classification standards. For example, constitutive equations regarding the rheological properties of soft soil and theoretical derivations of the TBM cutterhead cutting mechanism fall into this category.

[0068] Specific construction methods refer to the exclusive or highly scenario-dependent experiential knowledge accumulated by a particular construction unit or project team through long-term practice. This type of knowledge usually exists in unstructured form in internal documents, such as tunneling parameter settings under specific geological conditions, troubleshooting manuals for a certain type of tunnel boring machine, and daily construction logs and risk management plans from past projects. Layered tunnel boring machine training courses are not simply a data compilation, but rather an orderly organization based on the generality and specificity of knowledge. The construction process typically involves data cleaning, deduplication, and formatting.

[0069] For academic literature, PDF or HTML files are retrieved from databases using existing methods or modules, text is extracted using OCR technology, and segmented based on paragraph semantics. For construction site records, scanned documents and tabular data need to be digitized, with a focus on extracting working condition descriptions and corresponding operational parameters.

[0070] In some alternative implementations, to enhance the model's ability to perceive knowledge boundaries, source labels are attached to each data sample when constructing the basic knowledge set and the professional knowledge set. For example, all data originating from academic journals are labeled as academic domains, and data originating from enterprise internal servers are labeled as enterprise domains. This type of explicit domain labeling helps the model distinguish between features from different sources in subsequent steps. Furthermore, knowledge can be categorized according to its modality, classifying plain text descriptions, tabular parameters, and text-image pairs containing engineering drawing instructions separately to construct a multimodal training curriculum.

[0071] Step 102: Based on the frozen basic pre-trained model, the first parameter high-efficiency adapter and the second parameter high-efficiency adapter are trained in parallel using the hierarchical shield tunneling training course, and the theoretical features and engineering parameter features of the shield tunneling are encoded in a physically isolated manner.

[0072] This step employs PEFT (Performance-Efficient Fine-Tuning) technology, which injects domain-specific knowledge without altering the capabilities of the underlying large model.

[0073] The base pre-trained model refers to a large-scale language model that has been pre-trained on massive general-purpose corpora, such as the Qwen and Llama series models, which possess powerful general-purpose language understanding and generation capabilities. During training, all weight parameters of this base model remain frozen, meaning they do not participate in gradient updates, serving as a shared foundation for knowledge across all domains. The parameter-efficient adapter preferably adopts a low-rank adapter, or LoRA architecture. This architecture bypasses each layer or specific layer (such as the Q and V matrices of the Attention layer) of the base model, injecting two low-rank matrices A and B.

[0074] The first-parameter efficient adapter is specifically trained using a basic knowledge set, and its optimization objective is to minimize the prediction error of the model when answering academic theoretical questions.

[0075] The second-parameter high-efficiency adapter is specifically trained using a knowledge set, aiming to master specific construction parameters and methods. Physical isolation means that the first and second adapters have two independent sets of weight matrices A1, B1 and A2, B2. When training the first adapter, only A1 and B1 are updated; A2 and B2 either do not exist or are frozen; and vice versa.

[0076] This type of design cuts off the mutual interference path between knowledge from different domains when updating parameters, ensuring that the model does not overwrite or forget old knowledge (such as academic theories) when learning new knowledge (such as enterprise construction methods), effectively avoiding the problem of catastrophic forgetting.

[0077] As an example, assume the hidden layer dimension *d* of the base model is 4096, and the adapter rank *r* is set to 8. For each layer, the number of parameters introduced by the first adapter is approximately 2 × 4096 × 8, far less than the number of parameters in that layer of the base model (4096 × 4096). This lightweight design makes it possible to load and train multiple adapters simultaneously on a single GPU. In some implementations, to further improve training efficiency, quantization techniques such as QLoRA can be used to quantize the base model to 4 bits, while the adapters are trained with 16-bit or 32-bit precision. Furthermore, for academic knowledge requiring strong logical reasoning, adapters can be inserted into the multilayer perceptron (MLP) layer; while for construction logs that need to capture sequence dependencies, the focus is on inserting adapters into the Attention layer.

[0078] Step 103: Perform singular value decomposition on the weight matrices of the first-parameter efficient adapter and the second-parameter efficient adapter to extract the geometric feature subspaces representing the knowledge distribution, and calculate the orthogonality and conflict correlation between the geometric feature subspaces. Analyze the knowledge structure learned by the model from a mathematical geometry perspective.

[0079] Since the adapter's weight matrix Delta_W is obtained by multiplying low-rank matrices B and A, it essentially maps the high-dimensional input space to a low-dimensional feature subspace. Singular Value Decomposition (SVD) can effectively extract the principal orientation of this subspace.

[0080] Specifically, for the trained weight matrix Delta_W, the decomposition Delta_W≈U×Sigma×V transpose is performed. Here, U and V are the left and right singular vector matrices, respectively, and Sigma is a diagonal matrix containing the singular values. The geometric feature subspace is primarily spanned by the k right singular vectors corresponding to the largest singular values. These vectors represent the feature directions that the adapter is most sensitive to and has the highest activation level when processing input data.

[0081] For example, the geometric feature subspace of the first adapter might primarily correspond to the semantic dimension related to geological structure analysis, while the subspace of the second adapter might correspond to the numerical dimension related to tunneling speed parameters. Calculating orthogonality and conflict correlation is to quantify the degree of overlap between the two knowledge domains in their feature representations. If the angle between the principal feature directions of the first and second adapters in the vector space is close to 90 degrees (cosine similarity close to 0), it indicates that the two types of knowledge are orthogonal and do not interfere with each other; conversely, if the angle is small (cosine similarity close to 1), it indicates a conflict, meaning that the same feature direction is assigned different meanings or processing logics.

[0082] In practical implementations, the calculation of conflict correlation is typically based on the cosine similarity matrix between right singular vectors. Let V1 be the right singular vector matrix of the first adapter, and V2 be the right singular vector matrix of the second adapter. The matrix M is calculated as the transpose of V1 × V2. The elements M in matrix M... ij This represents the cosine similarity between the i-th feature direction of the first adapter and the j-th feature direction of the second adapter.

[0083] In some preferred embodiments, not only directional overlap is considered, but also the magnitude of singular values, i.e., feature importance. Weighted cosine similarity can be used, with singular values ​​as weighting factors, to more accurately assess the conflict between feature directions that have a significant impact on the model output. Furthermore, for deep neural networks, this analysis is performed layer by layer, with each layer generating a corresponding subspace description and conflict metric.

[0084] Step 104: Based on the geometric feature subspace and its conflict correlation degree, construct a unified adaptation mechanism for resource awareness. The mechanism is used to coordinate the activation state or fusion form of the first parameter efficient adapter and the second parameter efficient adapter according to the computational resource constraints in the shield tunneling engineering reasoning task.

[0085] Based on the analysis results of the previous step, this step determines how to integrate the scattered knowledge to serve the reasoning task.

[0086] Resource awareness refers to the mechanism's ability to perceive current hardware environment limitations during operation, such as the amount of GPU memory remaining, the maximum allowed number of floating-point operations (FLOPs), and the requirements for response latency.

[0087] The unified adaptation mechanism is a higher-level concept encompassing two specific implementation paths: static fusion and dynamic routing. In the static fusion path, the fusion form is coordinated. Based on the degree of conflict correlation, the system identifies orthogonal, conflict-free feature subspaces and directly linearly superimposes their corresponding weight parameters. For highly conflicting subspaces, additional fusion coefficients or gradient-based optimization methods are introduced to calculate compromise weight values, which are then merged into a fixed adapter file. This approach requires no additional computational overhead during inference and is suitable for resource-constrained embedded devices. In the dynamic routing path, the activation state is coordinated. During inference, the system dynamically determines which adapters to activate based on the semantics of the input problem and the current resource budget, even refining the activation of specific feature rank slices (RankSlices) within the adapters. For example, when memory is sufficient, activating all features of both adapters simultaneously achieves optimal accuracy; when memory is limited, only the principal feature subspace of the adapter most relevant to the current problem is activated, maximizing performance while meeting resource constraints.

[0088] Specifically, building this mechanism may involve generating a series of configuration files or a lightweight control network. For static paths, the output is a weight matrix file after SVD reconstruction and conflict resolution; for dynamic paths, the output is an index table containing feature signature information and a trained routing policy network.

[0089] In practical applications of tunnel boring machine (TBM) engineering, reasoning tasks are often unpredictable and diverse. For example, during normal tunneling, the system may only need to monitor parameters in a low-frequency, low-resource mode; however, when encountering geological anomaly alarms, the system needs to immediately call upon all knowledge resources for comprehensive analysis. A unified resource-aware adaptation mechanism can adaptively adjust model behavior according to changes in the scenario, ensuring the most reliable intelligent assistance under any working condition.

[0090] Example 2: This example details how to use a teacher model to construct a high-quality labeled dataset and how to design a parallel training architecture to achieve physical isolation of knowledge.

[0091] Step 201: Using a general large language model as the teacher model, text fragments are extracted from academic literature and construction site records respectively, and question-answer pairs are automatically generated.

[0092] To address the issue of unstructured raw data that is difficult to directly use for fine-tuning instructions, and considering the length and deep logical nesting of academic literature, directly truncating it as training data can easily lead to loss of context. Construction records often contain a large number of meaningless formatting symbols or abbreviations. Therefore, a powerful general-purpose large language model (such as GPT-4 or DeepSeek-V2) is introduced as the teacher model to undertake the tasks of data cleaning and generation. Specifically:

[0093] Long documents are segmented into text fragments of moderate length (e.g., 512 or 1024 tokens) using a sliding window or semantic paragraph-based segmentation algorithm. Then, a specific prompt word is constructed and input into the teacher model. This prompt word includes an instruction part and a context part. The instruction part explicitly requires the teacher model to act as a tunnel boring machine expert, conceive several in-depth questions based on the provided text fragments, and generate accurate answers according to the original text.

[0094] For example, given an academic text on the film-forming mechanism of slurry in slurry tunnel boring machines (TBMs), the teacher model might generate the question: In gravel strata, how should the specific gravity and viscosity of the slurry be adjusted to ensure film-forming quality? It would then generate corresponding technical answers. Similarly, given a fault log entry in the construction records, the teacher model might generate the question: What are the possible causes of an abnormal increase in hydraulic pressure in the propulsion system of a TBM in a certain project, and what are the troubleshooting steps? This approach transforms passive reading material into active question-and-answer pairs (Q&A data), improving the efficiency of model learning.

[0095] To ensure the quality of the generated data, a verification mechanism can be introduced. For example, the teacher model can self-evaluate the generated Q&A pairs, or rule-based filters can be used to remove samples with overly short answers or overly broad questions. Furthermore, for chart data, the OCR recognition results of the charts and their captions can be input as text fragments to guide the teacher model in generating question-and-answer pairs involving chart parameter analysis, thus enriching the dimensions of the data.

[0096] Step 202: For each generated question-answer pair, automatically label the task type and shield tunneling stage based on its source text content. The shield tunneling stage label should at least cover the survey, tunneling and maintenance stages.

[0097] By attaching fine-grained semantic labels to the data, a navigation basis is provided for subsequent dynamic routing. Task type labels are mainly used to distinguish the nature of the knowledge and processing logic involved in the problem.

[0098] Specifically, this includes, but is not limited to: theoretical explanations, which focus on explaining physical mechanisms and engineering principles; parameter recommendation categories, which focus on providing specific numerical setting ranges; standard query categories, which focus on referencing national standards or enterprise operating procedures; and risk warning categories, which focus on analyzing potential safety hazards.

[0099] The shield tunneling stage labels are used to define the time and space scope of application of the knowledge. The survey stage corresponds to data such as geological exploration reports and preliminary design drawings, involving knowledge such as lithological analysis and underground pipeline detection; the tunneling stage corresponds to construction logs and real-time monitoring data, involving core operational knowledge such as propulsion speed control, attitude adjustment, and synchronous grouting; the maintenance stage corresponds to equipment maintenance manuals and fault case libraries, involving knowledge such as cutter replacement and hydraulic system maintenance.

[0100] The generation of these labels can also rely on the teacher model. While generating Q&A pairs, the teacher model is required to output a metadata object in JSON format. For example, the structure of a sample might be represented as: {question:..., answer:..., labels:(task_type:parameter_recommendation, stage:boring_phase, source:project_log_001)}.

[0101] For some highly structured source data (such as construction manuals archived by chapter), tags can be directly generated by matching filenames or directory structures according to rules. These tags are not only stored as metadata, but can also be appended to the input text using specific templates, such as adding "[Tunneling Stage] [Parameter Recommendation]" before the question, to explicitly guide the model to establish a connection between tags and knowledge content.

[0102] Step 203: The question-answer pair data carrying task type labels and shield tunneling condition stage labels are organized into basic knowledge sets and professional knowledge sets, respectively, for use in subsequent parallel training and routing network construction.

[0103] This step divides the processed data into two sets for training. The basic knowledge set mainly contains data samples labeled as theoretical explanations and normative queries, sourced from academic literature. The professional knowledge set mainly contains data samples labeled as parameter recommendations and risk warnings, sourced from construction records.

[0104] This classification is not absolutely separate. Some engineering case analyses published in academic papers contain both theoretical and experimental data. They can be classified into one category based on their actual focus, or retained in both sets as bridge data.

[0105] In terms of organization, the data is typically stored in JSONL or Parquet format, with each row representing an independent training sample. To ensure balanced training, statistical analysis of the data with different labels is required. If the sample size for a particular operating condition (such as the maintenance phase) is found to be too small, it is necessary to go back to the data collection phase for targeted supplementation, or use oversampling techniques to increase its frequency of occurrence during training. The constructed dataset will be divided into training and validation sets. The design of the validation set is particularly critical; it must contain representative samples of all labels and maintain a mixed ratio of academic and enterprise data to be used in subsequent steps to evaluate the overall performance of the model and the effectiveness of the routing strategy.

[0106] Step 204: Keep the original weight matrix of the basic pre-trained model frozen and not involved in gradient updates.

[0107] This step establishes the baseline for fine-tuning. After loading the base pre-trained model (which includes models with 7B, 14B, or 72B parameters), the parameter attributes of all its layers, including the Embedding layer, the Attention and FeedForward layers in the TransformerBlock, and the output Head layer, are set to non-trainable (requires_grad=False). This means that during backpropagation, gradients for these parameters are not calculated, nor are their values ​​updated. This operation not only significantly reduces the memory usage required for training (because it eliminates the need to store large optimizer states), but also preserves the general language and logical reasoning capabilities acquired by the base model during pre-training, preventing overfitting or catastrophic forgetting that might occur with fine-tuning on small-scale domain data. The base model acts like a knowledgeable generalist, providing a solid foundation for subsequent specialized training.

[0108] Step 205: Construct the first low-rank matrix pair and the second low-rank matrix pair respectively. Each low-rank matrix pair is formed by multiplying a dimension-reduced matrix and an dimension-increasing matrix, and is initialized as an independent parameter space that does not interfere with each other.

[0109] The initialization process of the LoRA adapter is described. For the target layer weight matrix W (with dimensions d_in×d_out) that needs fine-tuning in the base model, two parallel adapter paths are constructed.

[0110] The first path corresponds to the first low-rank matrix pair (A1, B1), where A1 is a dimension-reduced matrix with dimension r × d_in, and B1 is an dimension-incremented matrix with dimension d_out × r. r is a set rank, usually much smaller than d_in and d_out (e.g., r takes the values ​​8, 16, or 32).

[0111] The second path corresponds to the second low-rank matrix pair (A2, B2), with the same structure as the first path, but with independent storage space. During initialization, the reduced-dimensional matrix A is typically initialized randomly using a Gaussian distribution, and the increased-dimensional matrix B is initialized to all zeros. This design ensures that the adapter output is zero before training begins, i.e., Delta_W = B × A = 0, and the model behaves identically to the original base model, guaranteeing that the training process starts from a stable starting point. The independent parameter spaces mean that A1, B1 and A2, B2 occupy different memory addresses, representing decoupled variables.

[0112] In some preferred embodiments, different ranks can be assigned to different adapters. For example, considering that academic knowledge is more abstract and broad, a larger rank (e.g., r=32) may be needed to capture complex features; while the construction parameters of a specific project are relatively specific and focused, and may only require a smaller rank (e.g., r=8). Differentiated configurations can further save on the number of parameters while ensuring effectiveness.

[0113] In addition, a learnable scaling factor α can be introduced for each adapter to control the contribution weight of the adapter's output to the total output. α / r is usually set to a constant during initialization.

[0114] Step 206: Use the basic knowledge set to perform backpropagation update on the first low-rank matrix pair to obtain the first parameter efficient adapter; use the professional knowledge set to perform backpropagation update on the second low-rank matrix pair to obtain the second parameter efficient adapter, ensuring the physical isolation between academic theoretical features and engineering parameter features on the parameter update path.

[0115] This step describes the parallel training process. Although the two adapters share the same frozen base model forward computation path, they are split during error backpropagation. Specifically, when the input comes from samples in the base knowledge set, the model's forward output is calculated by W×x + B1×A1×x (at this time, the second adapter is not activated or participates in the computation). The resulting prediction error is only used to calculate the partial derivatives with respect to B1 and A1, and the values ​​of B1 and A1 are updated using an optimizer (such as AdamW). Similarly, when the input comes from samples in the expertise set, the forward output is calculated by W×x + B2×A2×x, and the error is only used to update B2 and A2. This mechanism ensures that academic theoretical features are specifically encoded in (A1, B1), while engineering parameter features are specifically encoded in (A2, B2). They are like two independent memory areas in the brain, not interfering with each other.

[0116] In practice, to improve GPU utilization, samples from two datasets are often mixed in a batch for training. This requires a masking mechanism or a special batch organization method. For example, in a batch, for samples belonging to the basic knowledge set, the output of the second adapter is forced to zero (or its gradient flow is cut off in the computation graph); for samples belonging to the professional knowledge set, the output of the first adapter is set to zero. Alternatively, a simpler strategy can be adopted: first update the first adapter on the academic data in the batch, then update the second adapter on the enterprise data in the batch. Regardless of the engineering implementation, the core goal is to ensure the physical isolation of parameter updates. The trained (A1, B1) constitutes the first parameter efficient adapter, and (A2, B2) constitutes the second parameter efficient adapter, which will be saved as independent weight files for subsequent SVD analysis and inference.

[0117] Example 3: This example details how to perform feature extraction, adaptive truncation, and resource cost estimation on the trained adapter during the offline phase, and construct a geometry-resource signature table to guide online inference.

[0118] Step 301: Perform full singular value decomposition on the weight matrix to obtain the complete singular value sequence and the corresponding left and right singular vectors.

[0119] For each trained adapter (e.g., the first adapter), its weight matrix Δ_W at each layer can be explicitly reconstructed by computing B×A. Assume Δ_W has dimensions d_out×d_in. Perform full singular value decomposition (SVD) on it. Mathematically, this is expressed as Δ_W = U×σ×V T Here, U is an orthogonal matrix of d_out × d_out, with its column vectors being left singular vectors; V is an orthogonal matrix of d_in × d_in, with its column vectors being right singular vectors; σ is a diagonal matrix of d_out × d_in, and the elements on the diagonal are the singular values ​​σ_i, satisfying σ_1 ≥ σ_2 ≥ ... ≥ 0. These singular values ​​represent the energy or importance of the corresponding feature direction, i.e., the singular vector, in carrying knowledge information. The larger the singular value, the greater the impact of feature changes in that direction on the model output.

[0120] Since LoRA itself is low-rank (rank r), theoretically only the first r singular values ​​are non-zero (or greater than zero), and the remaining singular values ​​are close to zero within the range of numerical calculation error.

[0121] Therefore, it is only necessary to calculate the first r singular values ​​and their corresponding vectors, reducing computational complexity. In computational libraries such as PyTorch or NumPy, this process can be achieved by directly calling the svd or low_rank_svd functions.

[0122] Step 302: Calculate the sum of squares of the singular value sequence as the total energy, and adaptively determine the minimum truncation rank that can cover the main knowledge information based on the preset cumulative energy threshold.

[0123] Although LoRA sets the rank to r, not all r dimensions are fully utilized during training; the model often automatically learns a lower-dimensional sparse representation. To quantify this, the total energy E_total is defined as the sum of squares of all computed singular values, i.e., ∑(σ _i 2 Then, the cumulative energy percentage P_k is defined as the sum of the squares of the first k singular values ​​divided by the total energy.

[0124] A preset cumulative energy threshold τ_energy is introduced (e.g., set to 0.90 or 0.95). The system iterates from k=1 to find the smallest integer k that satisfies P_k≥τ_energy, and defines it as the minimum truncated rank r_eff. This r_eff represents the minimum number of feature dimensions that can explain more than 90% (or 95%) of the weight changes. r_eff may be different for adapters of different layers and tasks.

[0125] For example, shallow adapters may primarily handle simple syntactic features, resulting in a smaller r_eff; while deep adapters handle complex semantic logic, and their r_eff may be closer to the originally set r. This type of adaptive truncation mechanism enables the precise location of the payload carrying core knowledge for each layer of each adapter, providing a basis for subsequent compression and routing.

[0126] Step 303: Truncate the left and right singular vectors according to the minimum truncation rank, retain the main feature directions of the preceding sequence as the geometric feature subspace, filter out training noise and reduce the computational dimension of the subsequent construction of the unified adaptation mechanism.

[0127] Based on the calculated r_eff, only the first r_eff columns of the U matrix are retained, denoted as U_eff; the r_eff×r_eff sub-block in the upper left corner of the σ matrix is ​​retained, denoted as σ_eff; and the first r_eff columns of the V matrix are retained, denoted as V_eff. This set (U_eff, σ_eff, V_eff) constitutes the geometric feature subspace of the adapter at this layer.

[0128] This subspace is a simplified projection of the original, massive parameter space, eliminating dimensions with extremely small singular values ​​that may only represent random initialization noise or overfitting details. Combining the geometric feature subspaces of all layers forms a highly compressed, geometric knowledge representation. Compared to the original weight matrix, this geometric feature subspace occupies less storage space and better reflects the essential structure of knowledge.

[0129] For example, when calculating the conflict correlation degree later, it is only necessary to calculate the inner product between r_eff dimension vectors instead of the inner product of the original d dimension vectors, which reduces the computational dimension and overhead.

[0130] Step 304: Extract the singular value energy distribution vector and principal subspace direction projection from the first-parameter efficient adapter and the second-parameter efficient adapter after singular value decomposition, respectively, and divide the low-rank matrix of the adapter into several continuous rank slices.

[0131] Furthermore, the geometric features are refined to prepare for dynamic routing, specifically including:

[0132] Extract the singular value energy distribution vector. This is typically obtained by normalizing the singular values ​​on the diagonal of σ_eff, for example, v_energy=[σ_1 / ∑(σ), σ_2 / ∑(σ), ..., σ_reff / ∑(σ)]. This vector intuitively describes the concentration of the adapter's knowledge distribution across different dimensions. The projection onto the principal subspace direction is the V_eff matrix.

[0133] Perform the crucial slicing operation. Divide the r_eff feature dimensions into several consecutive groups, called rank slices. The partitioning strategy can be equal (e.g., every 4 ranks form a slice) or based on energy decay (e.g., the first 2 high-energy ranks form a slice, and the subsequent lower-energy ranks form a slice every 8 ranks).

[0134] Assuming r_eff = 16, it can be divided into 4 slices: Slice_0 contains ranks 0-3, Slice_1 contains ranks 4-7, and so on. Each slice Slice_k corresponds to a specific subset of singular vectors. This slicing design allows the model to avoid loading the entire adapter with or without all of it during inference. Instead, it can select and activate certain slices as needed, like building blocks. For example, for simple problems, it might only be necessary to activate Slice_0, which contains the main energy, thus saving computational resources.

[0135] Step 305: For each rank slice, calculate the corresponding number of floating-point operations and memory usage based on its matrix dimension, and construct a rank slice resource overhead mapping.

[0136] This step assigns a price tag to each slice. To achieve resource-aware routing, it's crucial to know precisely how much hardware resources are required to activate each slice. For the k-th slice, assume it contains n_k ranks. The calculation of its corresponding low-rank matrix during inference is as follows: input vector x (dimension 1 × d_in) × V_slice_k (dimension d_in × n_k), then × U_slice_k (dimension d_out × n_k). Therefore, the additional floating-point operations (FLOPs) introduced by this slice are mainly composed of these two matrix multiplications, approximately 2 × (d_in × n_k + d_out × n_k). If we consider a batch size of B and a sequence length of L, the total FLOPs are 2 × B × L × (d_in + d_out) × n_k. VRAM usage is primarily determined by the number of weight parameters for this slice, approximately (d_in + d_out) × n_k × the number of bytes for the data type (e.g., 2 bytes for FP16). These computational results are stored as key-value pairs, for example: {adapter_id:1, layer:5, slice_id:0, ranks:4, cost:(FLOPs:1.2G, VRAM:12MB)}. This is the rank slice resource cost mapping, which explicitly quantifies the cost required to acquire this part of the knowledge.

[0137] Step 306: Combine the singular value energy distribution vector, the cosine similarity between the principal subspace directional projections, and the rank slice resource overhead mapping to generate a geometry-resource signature table to guide dynamic reasoning decisions.

[0138] All information is aggregated to generate the final signature table, which serves as a comprehensive metadata table and is the basis for the routing network to query during inference.

[0139] For each layer of each adapter, an entry in the table, Signature_Entry, may contain the following fields:

[0140] 1. Energy_Score: Singular value energy distribution vector, representing the distribution of the value of knowledge at this layer;

[0141] 2. Conflict_Risk: The maximum cosine similarity between the main subspace of this layer and the corresponding layers of other adapters, representing the risk of knowledge conflict that may be caused by activating this layer;

[0142] 3. Slice_Map: A list containing the IDs of all slices in this layer and their corresponding resource overhead mappings;

[0143] 4. Semantic_Tag: The semantic tag (e.g., academic, enterprise) of the adapter to which this layer belongs. This geometry-resource signature table is typically very lightweight and can reside in the memory of the inference server. When a new inference request arrives, the routing network does not need to reload a large weight matrix for analysis; it can quickly consult this table and, combined with the current resource budget (e.g., how much GPU memory is left), quickly plan the optimal slice activation combination. This is similar to checking a map and price list before a trip to plan the most cost-effective route.

[0144] Example 4: This example describes in detail how to construct a routing network with self-awareness capabilities. This network can precisely control the activation granularity of knowledge within the model based on real-time fluctuations in hardware resource status, achieving optimal inference performance on resource-constrained shield tunneling site servers.

[0145] Step 401: Receive the natural language input request for the shield tunneling engineering inference task and obtain the real-time resource budget vector of the current inference environment. The real-time resource budget vector defines at least the maximum allowable memory increment and the computation latency level.

[0146] First, the boundary conditions for the inference task are established. The system receives natural language input from the user, such as: when tunneling in soft soil, how should the propulsion speed be adjusted if a sudden change in cutterhead torque occurs? Simultaneously, the system calls the underlying hardware monitoring interface (such as NVML) to collect the current resource status of the device in real time, and combines this with the system administrator's preset strategy to construct a real-time resource budget vector B_budget. In this embodiment, this vector is specifically represented as a triple: [V_max, F_max, L_level]. Wherein, V_max represents the maximum incremental increase in video memory allowed for this inference, in megabytes (MB). For example, if there is currently 24GB of video memory remaining, but the system only reserves 2GB for bursty tasks, then V_max is set to 2048. F_max represents the maximum allowed number of floating-point operations, in gigabytes (GFLOPs). This indicator is directly related to the energy consumption and computational density of inference. L_level is a discrete latency level indicator, for example, ranging from 1 to 3, where 1 represents real-time response (high priority, allowing a small sacrifice in precision for speed), and 3 represents background batch processing (low priority, allowing longer processing time for higher precision). This budget vector B_budget will be used as a constraint and input into the subsequent decision-making process along with the user's question.

[0147] In some optional implementations, the real-time resource budget vector can also include energy consumption constraints (such as maximum power limits) or network bandwidth constraints (for distributed inference scenarios). Furthermore, the latency level L_level can be further refined into specific time thresholds, such as a maximum allowable latency T_max of 500 milliseconds. The system can also dynamically adjust the budget vector based on the urgency of the input request (through keyword matching such as alarms, faults, etc.). For requests under high-risk conditions, the quotas for V_max and L_level are automatically increased to ensure the system responds at full capacity.

[0148] Step 402: Extract the semantic fingerprint of the natural language input request, and input it together with the feature terms in the geometry-resource signature table and the real-time resource budget vector into a pre-trained routing network.

[0149] The construction involves the input features of the router. Among them, the semantic fingerprint S_fingerprint is not a simple word embedding, but a composite vector that integrates contextual information and operating status.

[0150] Specifically, the system uses a lightweight encoder (such as BERT-Tiny or only the first two layers of the base model's embedding) to convert the input text into a fixed-length text feature vector v_text. Simultaneously, the system reads the current status monitoring data of the tunnel boring machine (TBM) and generates a One-Hot encoded vector v_stage representing the working condition (e.g., bits corresponding to the tunneling stage are 1, and the rest are 0). The semantic fingerprint S_fingerprint is obtained by concatenating v_text and v_stage, i.e., S_fingerprint = Concat(v_text, v_stage). Subsequently, the system reads the geometry-resource signature table from memory. To enable the routing network to handle variable-length signature tables (because different layers may have different numbers of slices), an attention mechanism is typically used, or the statistical features of the signature table (such as average energy, total conflict risk) are flattened to form a signature feature vector v_sig. The semantic fingerprint S_fingerprint, the signature feature v_sig, and the resource budget vector B_budget are concatenated to form the total input vector X_input of the routing network. Routing networks R are typically designed as multilayer perceptrons (MLPs), which include an input layer, several hidden layers, and an output layer. The number of parameters is extremely small (e.g., only a few thousand), ensuring that the computational overhead of routing decisions is negligible.

[0151] In some preferred embodiments, to enhance the routing network's sensitivity to resource constraints, the resource budget vector B_budget can be normalized at the input layer, mapping it to the range of 0 to 1. For the geometry-resource signature table, a Transformer-based Encoder structure can be used as the front end of the routing network, leveraging a self-attention mechanism to capture potential correlations between different adapter signatures. Furthermore, feedback information from historical routing decisions can be incorporated into the input features, such as which slices were activated in the previous time step, utilizing the principle of temporal locality to reduce routing jitter.

[0152] Step 403: Utilize the routing network output to generate dynamic activation masks for each rank slice in the first-parameter efficient adapter and the second-parameter efficient adapter. The dynamic activation mask is used to indicate the set of target rank slices that need to be loaded into video memory and participate in forward propagation computation under the current resource budget constraints.

[0153] The number of output layer nodes in the routing network corresponds to the total number of all rank slices in all adapters.

[0154] For example, if there are two adapters, each adapter is divided into 4 slices per layer, and the model has a total of 32 layers, then the number of output nodes is 2 × 4 × 32 = 256. The output layer typically uses the sigmoid activation function, outputting a probability vector P_prob between 0 and 1. To obtain a discrete binary mask, the system can use a threshold truncation method (e.g., if P_prob is greater than 0.5, then Mask is 1) or a Top-K sampling method (selecting the K slices with the highest probabilities). The resulting dynamic activation mask is a sparse 0 / 1 vector, where Mask_i = 1 indicates that the i-th rank slice is selected.

[0155] Based on this mask, the inference engine executes an on-demand loading strategy: it iterates through all slices with Mask_i=1, checking if their weight parameters are already in GPU memory. If not, it quickly loads the corresponding low-rank matrix fragment from main memory (CPU RAM) via the PCIe bus; if it is, it reuses it directly. For slices with Mask_i=0, the system performs no loading or computation operations. This mechanism enables dynamic scaling of GPU memory usage, allowing large models to run in fragmented form on devices with limited GPU memory.

[0156] In some alternative implementations, to ensure the sparsity of the output mask meets hard resource constraints, a verification step can be added after generating the Mask. The system calculates the expected total GPU memory usage V_est and total floating-point operations F_est based on the Mask (using overhead data from the signature table). If V_est > V_max or F_est > F_max, the system automatically sets the Mask of the few slices with the lowest probability to 0 until the budget constraint is met. Alternatively, the routing network can directly output a normalized weight vector. During inference computation, instead of a simple 0 / 1 selection, a weighted sum of the outputs of different slices is performed. However, this usually requires all slices to be in GPU memory and is suitable for scenarios where GPU memory is sufficient but only limited by FLOPs.

[0157] Step 404: The routing network is trained and optimized by minimizing the joint loss function on the hybrid validation set, so that the routing network can meet the computational resource constraints while maintaining prediction accuracy.

[0158] The training of the routing network is typically decoupled from the fine-tuning of the adapter; that is, after the adapter training is complete, the adapter parameters are fixed, and the routing network is trained separately. Training data comes from a mixed validation set. For each sample, a joint loss function L_total is constructed. This function consists of three parts: L_total = L_task + λ_1 × L_ortho + λ_2 × L_res. The first term, L_task, is the prediction error loss for the predefined domain task (such as cross-entropy loss), driving the routing network to select slices that allow the model to answer correctly. The second term, L_ortho, is an orthogonality constraint term, used to penalize slices that simultaneously activate highly conflicting segments. Specifically, if the mask indicates simultaneous activation of two similarity S... ij For slices with high cost, L_ortho increases. The third term, L_res, is a resource penalty term used to constrain cost. L_res is usually defined as ReLU(Cost_current minus Budget), meaning a positive penalty is only incurred when the total cost (Cost_current) of the currently active slice exceeds the budget; otherwise, the penalty is zero. λ_1 and λ_2 are hyperparameters used to balance the weights of various objectives. Through multiple rounds of iterative training on the validation set, the routing network gradually learns to be more cost-effective, prioritizing the selection of rank slices with the highest cost-effectiveness (contributing significantly to accuracy and minimizing conflicts) within the budget.

[0159] Suppose that for a given input, the routing network outputs an activation scheme that is expected to consume 10MB of GPU memory, while the budget constraint is 8MB. In this case, the resource penalty term L_res = ReLU(10-8) = 2. Assuming λ_2 is set to 0.5, this term contributes 1.0 to the total loss. The optimizer will adjust the parameters of the routing network based on this gradient signal, tending to reduce the number of slice activations for that sample in the next iteration. If reducing the number of slices leads to incorrect model predictions, L_task will increase sharply, forcing the network to search for other, more efficient slice combinations, and the network converges to a Pareto optimal state.

[0160] Example 5: This example provides an engineering supplement to the dynamic routing mechanism. This example details how, in high-frequency concurrency scenarios of actual tunnel boring machine (TBM) projects, coarse-grained traffic splitting and cache reuse techniques can further reduce the system's average response latency and computational load.

[0161] Step 501: Parse the natural language input request, identify its corresponding coarse-grained task type and working condition stage label, and make a coarse route determination accordingly.

[0162] A low-cost coarse-screening mechanism is added before fine-grained dynamic routing. The system uses a rule-based matcher or a very lightweight classifier (such as the FastText model) to quickly scan user input requests. It identifies the coarse-grained task type (e.g., academic theory, enterprise practice, hybrid) and the working condition stage label (e.g., in progress, shutdown). If the result is a single knowledge source request, such as a user explicitly querying the rated pressure of the tunnel boring machine's hydraulic cylinders (purely enterprise parameter), the system will directly trigger a bypass mechanism.

[0163] At this point, the system directly activates all rank slices of the corresponding second-parameter high-efficiency adapter (enterprise edition), or the preset high-priority primary slice, skipping the complex routing network calculation process. This type of direct-access strategy can handle approximately 70% to 80% of routine queries in the engineering field, reducing routing overhead. Only when a mixed-type request is determined, such as combining theoretical analysis of abnormal cutterhead wear in the current formation (requiring both theoretical and data analysis), does it enter the subsequent fine-grained routing process.

[0164] In some implementations, coarse-grained classifiers can be trained under supervised supervision using labeled data. To prevent performance degradation due to classification errors, a confidence threshold can be set. Only when the classifier's prediction confidence for purely academic or purely business purposes exceeds 0.95 will it proceed directly; otherwise, it defaults to a hybrid type and is left to a more intelligent routing network for judgment.

[0165] Step 502: If the coarse-grained task type is determined to be a mixed knowledge request, then the pre-built routing cache table is queried using the coarse-grained task type and the working condition stage label as the index key. This step leverages the temporal locality and content repetition of the tunnel boring machine (TBM) engineering problem.

[0166] In actual construction, under the same working conditions (such as three consecutive days of sandy soil excavation), technicians may repeatedly ask similar risk assessment questions. To address this, the system maintains a routing cache table, Cache_Table. The cache key is designed as a hash value of the input features, for example, Key=Hash(Task_Type, Stage_Label, Topic_Keywords). The Topic_Keywords can be obtained from the input question using a keyword extraction algorithm (such as TF-IDF). The cache value is the dynamic activation mask output by the routing network. When a new mixed-type request arrives, the system calculates its key and checks if the same key exists in the cache table. To improve the hit rate, fuzzy matching or semantic vector similarity matching can be used. This involves calculating the cosine similarity between the semantic fingerprint of the current request and the key value in the cache; if the similarity is greater than a preset threshold (e.g., 0.98), it is considered a hit.

[0167] Step 503: If a historical record is found in the routing cache table, the dynamic activation mask stored in the historical record is directly read and reused; if no record is found, the routing network calculation is performed and the newly generated dynamic activation mask is written to the routing cache table.

[0168] If the cache is hit, the system directly retrieves the corresponding Mask from the Cache_Table and immediately loads the adapter slice according to that Mask for inference. This process typically takes microseconds, which is negligible and speeds up the inference response. If the cache is missed, the system systematically calls the routing network, passing in the semantic fingerprint and resource budget, to calculate a new Mask. After inference is complete, the system writes this new (Key, Mask) pair into the cache table for reuse in subsequent similar requests. To prevent the cache table from growing indefinitely, the system uses a Least Recently Used (LRU) strategy to manage it, periodically evicting entries that have not been hit for a long time.

[0169] Furthermore, in some preferred embodiments, the cache table can also store resource budget levels. That is, the same question may have different masks under different resource budgets. Therefore, the key construction can also incorporate discrete resource budget level identifiers (e.g., low resource, high resource). This way, even if a user asks the same question, but the system load increases (budget decreases), the cache can correctly direct (or recalculate after a miss) to the more resource-efficient mask, rather than incorrectly reusing the previous high-resource mask, leading to memory overflow. Through a multi-level caching strategy, the system maximizes throughput while ensuring flexibility and security.

[0170] Example 6: This example is a specific expansion of the static fusion path. This example provides an alternative solution for scenarios with extremely limited resources (such as the inability to run dynamic routing networks) or extreme requirements for inference latency (disallowing dynamic loading overhead). Through offline computation, parallel adapters are merged into a fixed, conflict-free unified adapter, achieving zero additional overhead during the inference phase.

[0171] Step 601: The conflict correlation between geometric feature subspaces is concretized into a cosine similarity matrix of feature directions, and a preset similarity threshold is introduced.

[0172] After completing SVD decomposition and extracting the geometric feature subspace, for each layer, the system obtains the right singular vector matrix V1 of the first adapter and the right singular vector matrix V2 of the second adapter. The system calculates the cosine similarity matrix S_sim between the two. Each element S in the matrix... ij It is equal to the dot product of the i-th column vector of V1 and the j-th column vector of V2. Since singular vectors are usually normalized, this dot product directly represents the cosine of the angle between the two feature directions in high-dimensional space. ij The value of τ ranges from -1 to 1. The closer its absolute value is to 1, the more parallel the two feature directions are, meaning that the two adapters encode similar or conflicting transformation patterns in that dimension. For binarization determination, the system introduces a scalar threshold τ (e.g., 0.6 or 0.8). This threshold is a hyperparameter reflecting the system's tolerance for knowledge conflicts. The lower the threshold, the wider the region identified as conflicting, and the greater the subsequent optimization and fusion efforts; the higher the threshold, the more original features are retained.

[0173] Step 602: Compare the elements in the cosine similarity matrix with the similarity threshold to generate a binary conflict mask matrix. The conflict mask matrix is ​​used to divide the geometric feature subspace into orthogonal independent knowledge subspaces and overlapping knowledge conflict subspaces.

[0174] The system iterates through the similarity matrix S_sim and constructs a binary mask matrix Mask of the same dimension. When element S...ij When the absolute value of Mask is greater than the threshold τ, ij Set the value to 1 to mark a conflict point; otherwise, set it to 0 to mark a safe point. Based on this mask, the feature subspace is divided into two parts. The singular vectors corresponding to the row or column indices where the Mask is 0 constitute an independent knowledge subspace. This subspace is geometrically approximately orthogonal, meaning that the features of the first adapter and the features of the second adapter are functionally independent and can be used like piecing together... Figure 1 The elements are then perfectly joined. Indices where Mask corresponds to 1 form a knowledge conflict subspace. This subspace has highly overlapping directions; simple addition would lead to abnormal amplification of feature values ​​or mutual cancellation of semantics, necessitating special fusion processing.

[0175] Step 603: For the independent knowledge subspace indicated by the conflict mask matrix, directly retain the original projections of the first-parameter efficient adapter and the second-parameter efficient adapter in the corresponding feature directions as the orthogonal retained components of the final unified adapter.

[0176] For independent knowledge subspaces, the system follows the principle of maximizing preservation. Specifically, the system reconstructs the weights using the inverse transformation formula of SVD. For the i-th feature component determined to be independent in the first adapter, its contribution is calculated as the transpose of sigma_1_i×u_1_i×v_1_i. The independent components of the second adapter are handled similarly. Since the two are orthogonal, directly adding these reconstructed matrix components will not introduce interference noise. This part of the superposition result is called the orthogonal-preserving component W_ortho. This approach ensures that the unique abstract logic in academic theory that is independent of enterprise parameters (or vice versa) is completely transferred to the final model, forming the cornerstone of knowledge fusion.

[0177] Step 604: For the knowledge conflict subspace indicated by the conflict mask matrix, introduce the fusion coefficients to be optimized, construct candidate fusion feature projections, and perform gradient optimization using the validation set.

[0178] For conflict subspaces, simple superposition is no longer applicable. The system introduces a set of learnable fusion coefficients γ. For each pair of conflicting feature directions (i, j), a weighted combination is defined: γ_1 × feature i + γ_2 × feature j. The system constructs a candidate fusion model containing these undetermined coefficients and performs forward propagation using a mixed validation set. The gradient of the prediction loss L_task with respect to the coefficients γ is calculated. Using gradient descent, the value of γ is dynamically adjusted to find the optimal balance point, such that the model can answer academic questions to some extent while also taking into account enterprise parameters in this conflict dimension. This process is similar to finding a compromise between two contradictory expert opinions. To prevent overfitting, an orthogonality regularization term can be added to the optimization objective, forcing the new fused feature vector to point as close as possible to the middle ground between the two. When the optimization converges, the weights of this part are reconstructed using the γ coefficients to obtain the optimized fusion component W_opt.

[0179] Step 605: The orthogonal retained components and the optimized fusion components are superimposed to form the weight parameters of the final unified adapter, and then merged into a fixed final unified adapter.

[0180] The system adds W_ortho and W_opt to obtain the final incremental matrix △_W_final for each layer. At this point, △_W_final is a standard compact matrix (or a re-approximation of a low-rank form). During deployment, the system no longer loads two separate adapters or runs a routing network; instead, it directly loads this single △_W_final. Even before inference, △_W_final can be directly added to the frozen weights W_base of the base model, forming new weights W_merged = W_base + Delta_W_final. During inference, the model's computational architecture remains consistent with the original base model, without any additional branches or gating operations, minimizing memory usage (only one copy of the model weights needs to be stored). This is the optimal path to achieving intelligence for older, older tunnel boring machine main control computers with extremely limited computing power.

[0181] Example 7, please refer to Figure 1 This illustrates a preferred embodiment of the method of the present invention, specifically for the vertical field of tunnel boring machine (TBM) engineering. The method includes the following steps:

[0182] Step S1: Collect and construct a hierarchical dataset. In this embodiment, the goal of this step is to construct a hierarchical training course. (a) Collect broad-area academic knowledge on tunnel boring machines (TBMs). This involves collecting academic papers and journal articles from the past decade on TBM construction, geological hazards, and risk management from academic databases such as CNKI and Elsevier, and constructing them into a basic knowledge dataset. (b) Collect core enterprise knowledge. This involves organizing TBM project construction logs, proprietary construction method documents, equipment maintenance manuals, and historical risk contingency plans from the enterprise's internal knowledge base, and constructing them into a professional knowledge dataset.

[0183] Step S2: Constructing a fine-tuned training set. In this embodiment, to enable the model to learn question-and-answer capabilities, the documents collected in S1 are processed into question-and-answer pairs (Q&A). An auxiliary generative model is constructed using a general large model to automatically extract question-and-answer pairs from the dataset in S1, forming the first training set of academic Q&A and the second training set of enterprise Q&A.

[0184] The preferred implementation method for this step is:

[0185] Corpus preprocessing: Text extraction, cleaning, deduplication, and chunking are performed on the academic documents and corporate PDF and Word documents collected in S1 to form structured text fragments.

[0186] Use general-purpose large-scale models to generate Q&A: Use high-performance general-purpose large-scale language models, such as commercial or open-source models with hundreds of billions of parameters, with Doubao and Deepseek-r1 as teacher models.

[0187] The design prompt is a specific instruction for the teacher model. Input a text snippet from S1 and instruct it to generate five high-quality, logically deep question-and-answer pairs based on the following professional text in the field of tunnel boring machine engineering. The questions should be professional, and the answers must strictly adhere to the provided text content.

[0188] The system is automated by using API calls to batch input all text fragments from S1 into the teacher model, automatically generating a massive amount of original Q&A pairs.

[0189] Data filtering and validation involves quality control of the generated Q&A pairs, such as removing entries with overly short or broad answers, and ensuring that the answers are highly relevant to the original text.

[0190] A training set is formed by organizing high-quality Q&A pairs according to their academic or corporate origins, resulting in a first training set corresponding to broad academic knowledge and a second training set corresponding to core corporate knowledge.

[0191] The technical principles of LoRA will be explained.

[0192] The core of a large language model's Transformer architecture consists of multiple layers containing massive pre-trained weight matrices. Traditional full-parameter fine-tuning requires updating the parameters of all these huge matrices, resulting in extremely high computational costs.

[0193] The core idea of ​​LoRA technology is to freeze all the original pre-trained weight matrices so that they remain unchanged during training.

[0194] At a specific layer of the model, two small, trainable low-rank matrices are injected, namely... Figure 3 The Lora_A matrix A and the Lora_B matrix B are shown.

[0195] During fine-tuning, only the parameters of the two small matrices, Lora_A and Lora_B, are updated. For example... Figure 3 As shown, multiplying these two smaller matrices by B×A results in an incremental parameter matrix △W1. The shape of this △W1 matrix is ​​consistent with the original W matrix.

[0196] Therefore, during the actual forward propagation process after fine-tuning, the model's output h is modified as follows:

[0197] h=W·x+△W1·x=W·x+(B·A)·x;

[0198] This is equivalent to adding a learned increment ΔW1, i.e., ΔW1 = B·A, to the original weights W without changing them, thereby adjusting the model's output style and behavior at extremely low parameter cost. This ΔW is the trained LoRA adapter.

[0199] Step S3, Parallel Training of the Adapter. This step is crucial for addressing catastrophic forgetting. (a) Select the base pre-trained large language model Qwen3-32B as the initial weights W.

[0200] We select a large pre-trained language model, Qwen3-32B, whose massive original weight matrix is ​​represented as pre-trained weights W. In this step, these pre-trained weights W are frozen and do not participate in any gradient updates.

[0201] This invention initiates two (or more) independent LoRA training tasks in parallel:

[0202] Task 1: The system loads the first low-rank matrix A. 1 and B 1 The system is trained using the first training set of academic domain Q&A in S2, and this process only updates A. 1 and B 1 The parameters. After training, the first LoRA adapter ΔW1=A is obtained. 1 ·B 1 .

[0203] Task 2: The system simultaneously loads the second set of low-rank matrices A. 2 and B 2 The system is trained using the second training set of enterprise professional Q&A in S2, and this process only updates A. 2 and B 2 The parameters are then obtained. After training, the second LoRA adapter ΔW2 = A is obtained. 2 ·B 2 .

[0204] pass Figure 4 In the architecture shown, although the two training tasks share the same frozen pre-trained weights W, their respective trainable parameters A are different. 1 B 1 A 2 and B 2 It is physically isolated. This ensures that when training enterprise knowledge, the parameters of already trained academic knowledge will never be touched or overridden.

[0205] The parallel branch flowchart clearly shows that Task 1 and Task 2 are parallel, each with its own independent training process and independent LoRA fine-tuning matrix.

[0206] Start two independent training tasks in parallel:

[0207] Task 1 is to efficiently fine-tune the parameters on the base model W using the first training set of academic Q&A in S2, and train the first low-rank adapter LoRA, denoted as △W1.

[0208] Task 2 involves using the second training set of Enterprise Q&A in S2 to efficiently fine-tune the parameters on the same base model W, and training to obtain a second low-rank adapter LoRA, denoted as △W2.

[0209] In this embodiment, the LoRA adapters △W all follow the mathematical principle of low-rank adaptation, where A and B are trainable low-rank matrices. Since △W1 and △W2 are trained independently, parameters are isolated during training, thus preventing parameter overwriting and knowledge forgetting from an architectural perspective.

[0210] Step S4: Construct the final adapter. This step is crucial for achieving non-destructive knowledge fusion. (a) After completing S3, two independent knowledge vectors are obtained: △W1 and △W2.

[0211] (b) The knowledge composition module executes an SVD-based conflict reduction algorithm to construct the final unified adapter △W. final The specific implementation steps are as follows:

[0212] For each Transformer layer's LoRA weight matrix, the system loads the trained ΔW1 and ΔW2. The system then performs truncated singular value decomposition on these two matrices, setting the truncation rank to k. The decomposition formula is as follows:

[0213] △W1≈U1·Σ1·V1 T ; △W2≈U2·Σ2·V2 T ;

[0214] Here, V1 and V2 represent the main activation directions of knowledge in their respective domains in the input feature space.

[0215] To calculate subspace similarity and conflict masking, and to determine whether academic knowledge conflicts with business knowledge, the system calculates the cosine similarity of the right singular vector matrix. A similarity matrix S = |V1| is constructed. T The element values ​​in V2|S are between 0 and 1. Values ​​close to 0 indicate that the feature directions are perpendicular and orthogonal, and they do not interfere with each other; values ​​close to 1 indicate that the feature directions are parallel, and there is a parameter update conflict. A hyperparameter threshold τ is set. A conflict mask matrix M is generated: when S... ij When >τ, M ij =1 indicates a conflict; otherwise, 0 indicates no conflict.

[0216] The final unified adapter △W_final is constructed through a block-based process:

[0217] For the non-collision regions where the mask M is 0, orthogonal preservation is maintained, indicating that the two types of knowledge act on different feature dimensions and are directly superimposed: △W orthogonal =△W1+△W2.

[0218] For the portion where the mask M is 1, the gradient direction that minimizes the loss of the validation set is calculated using the validation set containing mixed data, and the conflict weights are adjusted accordingly. In the simplified form of this embodiment, a weighted average based on the magnitude of the singular values ​​is used:

[0219] △W_conflict=(Σ1·△W1+Σ2·△W2) / (Σ1+Σ2).

[0220] Final merger △W final =△W_orthogonal+△W conflict .

[0221] Through the above SVD processing, the obtained model mainly activates the direction corresponding to V1 when dealing with shield tunneling academic problems and mainly activates the direction corresponding to V2 when dealing with enterprise parameters, without interference between them.

[0222] △W finalWhen applied to the basic model W, a specialized large-scale language model is obtained that simultaneously grasps both broad academic knowledge of tunnel boring machines and core enterprise knowledge, without any forgetting or conflict between the two.

[0223] By employing a parallel training architecture, the parameter update paths of academic and enterprise knowledge are physically isolated, preventing forgetting caused by sequential training. Through the introduction of SVD geometric feature analysis and a conflict masking mechanism, orthogonal independent knowledge subspaces and overlapping conflict subspaces are mathematically identified and separated. For the conflict components, gradient-based optimization fusion is used. The geometric prior-based processing effectively solves the feature interference problem caused by linear merging, ensuring the coexistence of theoretical logic and empirical parameters.

[0224] It should be noted that in some embodiments, some loss functions and optimizers can be implemented using existing technologies, and therefore descriptions are omitted. The routing network can be configured according to actual conditions, such as a 3-layer MLP, with an input layer of ~900 dimensions, hidden layers of 256 / 128 dimensions, an output layer equal to the number of rank slices, hidden layers of ReLU, and an output layer of Sigmoid.

[0225] This embodiment also constructs a geometry-resource signature table containing rank-slice resource overhead mappings. Based on this, a resource-constrained dynamic rank routing (DRK) is implemented. By explicitly using the real-time hardware resource budget (memory increment / FLOPs) as input to the routing network, and in conjunction with coarse routing and caching mechanisms, this scheme dynamically determines which rank slices to activate based on the current hardware state during the inference phase. This solves the problem that existing static models cannot adapt to fluctuating computing power resources at the tunnel boring machine site, achieving optimal performance output under limited resources.

Claims

1. A large language model progressive domain fine-tuning and knowledge fusion method for shield engineering, characterized in that, The method comprises the following steps: constructing a hierarchical shield training course including a wide-area theory and a proprietary method; based on a frozen base pre-training model, using the hierarchical shield training course to respectively and in parallel train first and second parameter efficient adapters to respectively encode shield theory features and engineering parameter features in a physically isolated manner; performing singular value decomposition on the weight matrices of the first and second parameter efficient adapters, extracting geometric feature subspaces representing knowledge distribution, and calculating the orthogonality and conflict correlation between the geometric feature subspaces; based on the geometric feature subspaces and the conflict correlation, constructing a resource-aware unified adaptation mechanism, which is used to coordinate the activation state or fusion form of the first and second parameter efficient adapters according to the calculation resource constraints in a shield engineering reasoning task; extracting geometric feature subspaces representing knowledge distribution, and calculating the orthogonality and conflict correlation between the geometric feature subspaces, including: for the first and second parameter efficient adapters after singular value decomposition, respectively extracting singular value energy distribution vectors and main subspace direction projections, and dividing the low-rank matrices of the adapters into a plurality of continuous rank slices; for each rank slice, calculating the corresponding floating point operation number and memory occupation according to the matrix dimension, and constructing a rank slice resource overhead mapping; combining the cosine similarity between the singular value energy distribution vectors and the main subspace direction projections, and the rank slice resource overhead mapping, to generate a geometric-resource signature table for guiding dynamic reasoning decisions; based on the geometric feature subspaces and the conflict correlation, constructing a resource-aware unified adaptation mechanism, including: receiving a natural language input request of a shield engineering reasoning task, and obtaining a real-time resource budget vector of the current reasoning environment, which defines the maximum allowed memory increment and calculation delay level; extracting the semantic fingerprint of the natural language input request, and inputting it together with the feature items in the geometric-resource signature table and the real-time resource budget vector into a pre-trained routing network; using the routing network to output a dynamic activation mask for each rank slice in the first and second parameter efficient adapters, which is used to indicate the target rank slice set that needs to be loaded into the memory and participate in the forward propagation calculation under the current resource budget constraint.

2. The method of claim 1, wherein, The routing network is trained and optimized by minimizing a joint loss function on a mixed validation set, and the construction of the joint loss function includes: inputting the samples in the mixed validation set into the model configured by the dynamic activation mask, and calculating the prediction error loss of the predetermined domain task; according to the rank slices selected by the dynamic activation mask, combining the rank slice resource overhead mapping in the geometric-resource signature table, calculating the total memory occupation and floating point operation amount under the current activation state, and constructing a resource penalty term based on the degree of exceeding the real-time resource budget vector; weighting and summing the prediction error loss and the resource penalty term to form a joint loss function for updating the parameters of the routing network, so that the routing network meets the calculation resource constraints while maintaining the prediction accuracy.

3. The method of claim 1, wherein, Before inputting the natural language input request into the routing network, further comprising a coarse-grained task routing and cache reuse step: The natural language input request is parsed to identify its corresponding coarse-grained task type and working condition stage label; If the coarse-grained task type is determined to be a single knowledge source request, directly activate all rank slices of the corresponding first parameter efficient adapter or second parameter efficient adapter, skip the calculation of the routing network; If the coarse-grained task type is determined to be a mixed knowledge request, use the coarse-grained task type and the working condition stage label as the index key to query the pre-constructed routing cache table; If a historical record is hit in the routing cache table, directly read and reuse the dynamic activation mask stored in the historical record; If not, perform routing network calculation and write the newly generated dynamic activation mask to the routing cache table.

4. The method of claim 1, wherein, Build a hierarchical shield training course containing a wide range of theories and proprietary methods, the course consists of a basic knowledge set derived from academic literature and a professional knowledge set derived from construction site records, including: Use a general large language model as a teacher model to extract text segments from pre-stored academic literature and construction site records, and automatically generate question and answer pair data; For each generated question and answer pair data, automatically label the task type label and shield working condition stage label according to the source text content, the shield working condition stage label at least covers the survey, tunneling and maintenance stages; Organize the question and answer pair data carrying the task type label and the shield working condition stage label into the basic knowledge set and the professional knowledge set respectively.

5. The method of claim 1, wherein, Build a resource-aware unified adaptation mechanism, including: The conflict correlation degree between geometric feature subspaces is specified as a cosine similarity matrix of feature directions, and a pre-set similarity threshold is introduced; Compare the elements in the cosine similarity matrix with the similarity threshold to generate a binary conflict mask matrix, which is used to divide the geometric feature subspaces into orthogonal independent knowledge subspaces and overlapping knowledge conflict subspaces; Based on the conflict mask matrix, the first and second parameter efficient adapters are merged into a fixed final unified adapter, realizing resource awareness without increasing the inference memory occupancy.

6. The method of claim 5, wherein, Based on the conflict mask matrix, the first and second parameter efficient adapters are merged, including: For the independent knowledge subspaces indicated by the conflict mask matrix, directly retain the original projection of the first and second parameter efficient adapters in the corresponding feature directions as the orthogonal reserved components of the final unified adapter; For the knowledge conflict subspaces indicated by the conflict mask matrix, introduce the fusion coefficient to be optimized to construct the candidate fusion feature projection; Use the validation set containing mixed academic and enterprise data to calculate the prediction loss gradient of the candidate fusion feature projection, and dynamically adjust the fusion coefficient through gradient descent method until the prediction loss converges to obtain the optimized fusion component of the final unified adapter; Superimpose the orthogonal reserved component and the optimized fusion component to form the weight parameters of the final unified adapter.

7. The method of claim 1, wherein, Based on the frozen basic pre-training model, use the hierarchical shield training course to train the first and second parameter efficient adapters in parallel, including: Keep the original weight matrix of the basic pre-training model in a frozen state and do not participate in gradient update; Each of the first and second low-rank matrix pairs is initialized as an independent parameter space without interference, and each low-rank matrix pair is composed of a dimension-reducing matrix and a dimension-increasing matrix; The first parameter efficient adapter is obtained by performing back propagation update on the first low-rank matrix pair using the basic knowledge set, and the second parameter efficient adapter is obtained by performing back propagation update on the second low-rank matrix pair using the professional knowledge set.

8. The method of claim 1, wherein, Performing singular value decomposition on the weight matrix of the first and second parameter efficient adapters to extract a geometric feature subspace representing the knowledge distribution, including: Performing full singular value decomposition on the weight matrix to obtain a complete singular value sequence and corresponding left and right singular vectors; Calculating the sum of squares of the singular value sequence as the total energy, and adaptively determining the minimum truncated rank capable of covering the main knowledge information according to a preset cumulative energy threshold; According to the minimum truncated rank, the left and right singular vectors are truncated, and the main feature directions of the front sequence are retained as the geometric feature subspace.

Images