Cross-domain ship large model training method and device, computer device, storage medium and computer program product

Abstract

The application provides a cross-field ship large model training method and device, computer equipment, a storage medium and a computer program product. The method comprises: acquiring field training data sets of multiple fields involved in a ship design whole process; training each subdivided professional adaptation layer of each field in parallel by using the field training data set of each field, obtaining each trained professional adaptation layer of each field; generating a whole field adapter based on each trained professional adaptation layer of each field, loading the whole field adapter to a basic large language model to form an initial ship large model; acquiring a cross-field training data set and a professional training data set, and fine-tuning the initial ship large model by using the cross-field training data set and the professional training data set until a preset condition is reached, obtaining a final ship large model. The method can accurately and reliably respond to cross-field tasks in the ship design whole process.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a cross-domain method, apparatus, computer equipment, storage medium and computer program product for training large ship models. Background Technology

[0002] Ship design is a complex systems engineering project that encompasses multiple stages, including research and development, construction, and operation. In this process, it is necessary to deal with heterogeneous knowledge from different fields and complete a large number of cross-domain collaborative tasks.

[0003] Currently, large language models are often used to assist in the ship design process to improve efficiency and quality. However, due to the independent professional knowledge systems, vastly different data formats, and diverse task objectives of various shipbuilding fields, directly using massive amounts of data covering the entire process to train a single large language model will make it difficult for the model to form a stable, clear, and collaborative multi-domain knowledge representation within the parameter space. As a result, when dealing with complex cross-domain tasks that require deep integration of multi-domain knowledge, the model's output answers are often inaccurate and lack logical coherence, making it difficult to meet the reliability and professional requirements of practical engineering applications.

[0004] Therefore, traditional technologies suffer from the problem that existing large language models cannot accurately and reliably respond to cross-domain tasks throughout the entire ship design process. Summary of the Invention

[0005] Based on this, the purpose of this application is to at least solve one of the above-mentioned technical defects, in particular the technical defect that the existing large language model cannot accurately and reliably respond to cross-domain tasks in the entire process of ship design. This application provides a cross-domain ship large model training method, apparatus, computer equipment, computer-readable storage medium and computer program product that can accurately and reliably respond to cross-domain tasks in the entire process of ship design.

[0006] Firstly, this application provides a cross-domain method for training large ship models, the method comprising:

[0007] Obtain domain training datasets for multiple fields involved in the entire ship design process;

[0008] For any domain, the domain training dataset is used to train each sub-professional adaptation layer of the domain in parallel, resulting in each trained professional adaptation layer of the domain.

[0009] Based on the post-training professional adaptation layers of each domain, a full-domain adapter is generated and loaded into the basic large language model to form the initial large ship model.

[0010] Obtain cross-domain training datasets covering various fields and professional training datasets for stabilizing the performance of each sub-specialty task. Fine-tune the initial large ship model using the cross-domain training datasets and professional training datasets until the preset conditions are met to obtain the final large ship model. The final large ship model is used to respond to cross-domain tasks in the entire ship design process and output response text for cross-domain tasks.

[0011] In an exemplary embodiment, each sub-specialty adaptation layer includes a first parameter matrix with trainable parameters and a second parameter matrix with fixed parameters; the first parameter matrix is ​​used to learn the specialty task features of the domain; the second parameter matrix is ​​used to project the specialty task features of the domain learned by the first parameter matrix onto the parameter space of the basic large language model; using the domain training dataset, each sub-specialty adaptation layer of the domain is trained in parallel to obtain each trained specialty adaptation layer of the domain, including:

[0012] The first parameter matrix of each sub-specialty adaptation layer in the domain is initialized to obtain the initialized first parameter matrix of each sub-specialty adaptation layer. The second parameter matrix of each sub-specialty adaptation layer in the domain is orthogonalized to obtain the orthogonalized second parameter matrix of each sub-specialty adaptation layer. Different orthogonalized second parameter matrices are pairwise orthogonal.

[0013] Using the domain training dataset, we train each sub-professional adaptation layer in parallel. During the parallel training process, we fix the orthogonalized second parameter matrix of each sub-professional adaptation layer and only update the initialized first parameter matrix of each sub-professional adaptation layer until the convergence condition is met, thus obtaining each trained professional adaptation layer of the domain.

[0014] In one exemplary embodiment, a domain-wide adapter is generated based on each trained professional adaptation layer in each domain, including:

[0015] Determine the domain weight of each domain and the professional weight of each sub-specialty under each domain, and based on the domain weight of each domain and the professional weight of each sub-specialty under each domain, determine the integration weight of each sub-specialty under each domain;

[0016] By using the fusion weights of each sub-specialty in each domain, the parameter matrices of each trained professional adaptation layer in each domain are weighted and fused to obtain the domain parameter matrix corresponding to each domain.

[0017] Based on the domain weights of each domain, the domain parameter matrices corresponding to each domain are weighted and fused to obtain the full domain parameter matrix, thereby generating the full domain adapter.

[0018] In an exemplary embodiment, the cross-domain training dataset includes task samples of multiple cross-domain tasks; each task sample includes the cross-domain task and its standard response text; the professional training dataset includes task samples of multiple sub-professional tasks; each sub-professional task includes the sub-professional task and its standard response text; the initial large-scale ship model is fine-tuned using the cross-domain training dataset and the professional training dataset until preset conditions are met, resulting in the final large-scale ship model, including:

[0019] Each cross-domain task is input into the initial large ship model to obtain the predicted response text of each cross-domain task; and each sub-specialty task is input into the initial large ship model to obtain the predicted response text of each sub-specialty task.

[0020] Based on the predicted response text and standard response text of each cross-domain task, determine the cross-domain loss information, and based on the predicted response text and standard response text of each sub-specialty task, determine the sub-specialty loss information.

[0021] Based on cross-domain loss information and subdivided professional loss information, the total loss information is determined;

[0022] Based on the total loss information, the initial large ship model is fine-tuned until the preset conditions are met, resulting in the final large ship model.

[0023] In one exemplary embodiment, the method further includes:

[0024] Natural language tasks that acquire user input; natural language tasks can be categorized into specialized tasks or cross-domain tasks.

[0025] Retrieve terms and their attributes related to the natural language task from a pre-built knowledge base of ship engineering terminology, and generate contextual prompts adapted to the natural language task based on the terms and their attributes.

[0026] The contextual prompts are combined with the natural language task and then input into the final large-scale ship model to obtain the response text for the natural language task.

[0027] In one exemplary embodiment, the method further includes:

[0028] Monitor the update status of the ship engineering rules knowledge base associated with the final ship master model;

[0029] When the update state meets the preset model training triggering conditions, acquire each update rule and the knowledge data associated with each update rule, and construct an incremental training dataset based on each update rule and the knowledge data associated with each update rule.

[0030] The final large ship model is trained using an incremental training dataset until it meets the preset conditions.

[0031] Secondly, this application provides a cross-domain ship large model training device, the device comprising:

[0032] The data acquisition module is used to acquire domain training datasets from multiple fields involved in the entire ship design process;

[0033] The parallel training module is used to train each sub-specialty adaptation layer of the domain in parallel using the domain's domain training dataset, thereby obtaining each trained specialty adaptation layer of the domain.

[0034] The adapter loading module is used to generate a domain-wide adapter based on each trained professional adaptation layer in each domain, and load the domain-wide adapter into the basic large language model to form the initial large ship model.

[0035] The model fine-tuning module is used to acquire cross-domain training datasets covering various fields and professional training datasets for stabilizing the performance of each sub-specialty task. The initial large ship model is then fine-tuned using the cross-domain training datasets and professional training datasets until preset conditions are met, resulting in the final large ship model. The final large ship model is used to respond to cross-domain tasks in the entire ship design process and output response text for the cross-domain tasks.

[0036] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0037] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0038] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0039] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0040] The cross-domain ship large model training method, apparatus, computer equipment, storage medium, and computer program product provided in this application acquire domain training datasets from multiple domains involved in the entire ship design process. For any domain, the domain training dataset is used to train the adaptation layers of each sub-specialty within that domain in parallel, resulting in each trained professional adaptation layer. Based on the trained professional adaptation layers of each domain, a global adapter is generated and loaded into the basic large language model to form the initial ship large model. A cross-domain training dataset covering all domains and a professional training dataset used to stabilize the performance of each sub-specialty task are acquired. The initial ship large model is then fine-tuned using both the cross-domain and professional training datasets until preset conditions are met, resulting in the final ship model. The final large-scale ship model is used to respond to cross-domain tasks throughout the entire ship design process, outputting response texts for these tasks. This efficiently injects relevant domain expertise into different sub-specialty adaptation layers, constructing a comprehensive adapter specifically for ship design. This comprehensive adapter is then combined with the basic large-scale language model, and through joint fine-tuning with cross-domain and specialized training data, the final large-scale ship model is obtained. This overcomes the limitations of general-purpose large-scale language models lacking ship-specific expertise and single-specialty models being unable to perform cross-domain collaboration. Furthermore, it significantly optimizes the model's ability to handle complex and comprehensive tasks, enabling the final large-scale ship model to understand and accurately respond to complex ship design tasks involving multi-disciplinary collaboration, significantly improving the accuracy, reliability, and engineering practicality of the response content. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart illustrating a cross-domain large ship model training method provided in this application embodiment;

[0043] Figure 2 This is a flowchart illustrating a cross-domain large ship model training method in another embodiment of this application;

[0044] Figure 3 A structural block diagram of a cross-domain large ship model training device provided in this application embodiment;

[0045] Figure 4 This is an internal structural diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0046] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0047] The rapid iteration of AI technology provides core support for the intelligent transformation of the shipbuilding and marine engineering industry. Relying on the diverse capabilities of technologies such as RAG cross-modal retrieval, LoRA fine-tuning, and large visual models, it can address key needs such as knowledge integration, efficiency improvement, and semantic adaptation in the entire ship design process, injecting new momentum into the industry's breakthrough of traditional model bottlenecks and its move towards green and intelligent development.

[0048] As a typical discrete manufacturing industry, the shipbuilding and ocean engineering industry is characterized by high technical complexity, frequent design changes, small production batches, and dispersed professional knowledge. Traditional design models face multiple pain points: First, knowledge acquisition efficiency is low, relying on manual review of thousands of pages of massive specification manuals, resulting in insufficient knowledge acquisition efficiency; second, modeling and collaboration cycles are lengthy, with 2D drawings taking 6-18 months to convert to 3D models and cross-departmental collaborative development of new ship type software taking 6-12 months; third, there are gaps in semantic expression, with highly domain-specific professional terms such as "roll period" and "structural redundancy" being difficult for general semantic embedding models to accurately capture their physical connotations, causing semantic disconnect in tasks such as specification question answering and drawing classification by AI systems; fourth, rule adaptation is severely lagging, with engineering rules such as welding process parameters and structural strength formulas undergoing frequent dynamic iterations, requiring traditional models to be fully retrained to adapt to new rules; fifth, cross-domain task interference is significant, with large differences in data distribution across ship R&D design, construction management, and operation management, and traditional LoRA not being professionally segmented, easily leading to parameter interference between different tasks and low convergence efficiency.

[0049] Addressing the core pain points of the industry, this application focuses on tackling three key technical challenges and incorporates a domain-specific LoRA innovation solution in the process of fine-tuning efficiency optimization with small samples. This application improves the efficiency of small-sample fine-tuning and cross-task compatibility. Addressing the issues of redundancy in fixed-rank parameters and cross-domain task interference in traditional LoRA, it proposes a two-level subdivided LoRA architecture based on the "shipbuilding domain - specialty," combined with matrix orthogonalization design to achieve efficient training. This application bridges the semantic gap in professional terminology by constructing a shipbuilding-specific terminology dictionary and a dynamic prompt generation mechanism, accurately mapping professional terms such as "structural redundancy" and "roll period" to a low-dimensional semantic space. This adapts to the long-tail application scenarios of new environmentally friendly ship standards, solving the problem of general semantic embedding models failing to capture the physical connotation of domain terms. This application also addresses the problem of lagging dynamic rule adaptation by designing a rule-constrained loss function and an incremental learning framework. Through KL divergence (Kull-back-Leibler Divergence), it embeds engineering rules such as welding process parameters and structural strength formulas into the model training objective, supporting local updates of the rule base instead of full retraining. This reduces the model's response time to rule iteration from the traditional 1-2 months to within 1 week.

[0050] In one exemplary embodiment, such as Figure 1 As shown, a cross-domain method for training large ship models is provided. Taking the application of this method to a server as an example, the method includes the following steps S102 to S108. Wherein:

[0051] Step S102: Obtain domain training datasets for multiple fields involved in the entire ship design process.

[0052] The entire ship design process refers to the complete ship engineering stages from conceptual design, preliminary design, detailed design to production design.

[0053] Among them, "multiple fields" refers to the areas involved in the design process of shipbuilding engineering, including research and development design, construction management, and operation management.

[0054] Among them, the domain training dataset is a collection of question-and-answer or instruction data for sub-specialty tasks under multiple sub-specialties within a specific domain.

[0055] Optionally, the server first collects and organizes domain training datasets for various aspects of the ship.

[0056] Step S104: For any domain, use the domain training dataset to train each sub-specialty adaptation layer of the domain in parallel, and obtain each trained specialty adaptation layer of the domain.

[0057] Among them, the sub-specialty adaptation layer is a lightweight neural network module used to adapt the general capabilities of the basic large language model to a specific shipbuilding sub-specialty task.

[0058] Optionally, for each sub-specialty adaptation layer in each field, each sub-specialty adaptation layer can be trained in parallel using only the field training dataset of that field, so that each sub-specialty adaptation layer can quickly learn the field-specific features.

[0059] In practical applications, based on the business scenarios and data characteristics of the shipbuilding industry, the three major areas can be divided into multiple sub-specialties, with each area including multiple sub-specialties.

[0060] For example, the R&D and design field includes different sub-disciplines such as overall design, hull, marine engineering, electrical engineering, and outfitting, corresponding to core tasks such as overall scheme design, hull structure calculation, marine engineering system selection, electrical wiring design, and outfitting equipment layout. The construction management field includes different sub-disciplines such as production planning, quality inspection, and cost management, covering construction process planning, welding quality inspection, and construction cost accounting scenarios. The operation management field includes different sub-disciplines such as maritime inspection, operation and maintenance management, operation management, and route planning, supporting functions such as maritime compliance judgment, equipment fault diagnosis, operation data management, and optimal route generation. The sub-discipline adaptation layer corresponding to each sub-discipline can be independently initialized and fine-tuned to ensure the accuracy of sub-discipline tasks.

[0061] For the research and development design field, multiple professional LoRAs are set up, namely overall LoRA, marine engineering LoRA, electrical LoRA and outfitting LoRA. The input data for the overall LoRA is the ship type (bulk carrier, LNG carrier, etc.) and design loads. The output data is the general arrangement drawing and tables of main dimensional parameters such as length, beam, and draft. The training data includes more than 5,000 overall scheme drawings and more than 3,000 code clauses. The input data for the hull LoRA is the hull structure drawings and material strength parameters (such as AH36 steel). The output data includes stress calculation reports. The training data covers more than 8,000 structural drawings and more than 5,000 strength formulas. The input data for the engine LoRA is the engine parameter table (power, speed) and operation and maintenance records. The output data includes engine selection recommendations and fault warning results such as bearing wear. The training set contains more than 4,000 parameter tables and more than 2,000 fault cases. The input data for the electrical LoRA is electrical load data. The output data is electrical calculation sheets. The training data includes more than 3,000 electrical load records. The input data for the outfitting LoRA is the outfitting equipment list. The output data is outfitting calculation sheets. The training data includes more than 4,000 equipment lists.

[0062] For the construction management field, multiple specialized LoRaAs are set up: Production Planning LoRa, Quality Inspection LoRa, and Cost Management LoRa. The Production Planning LoRa takes component processing technology and personnel / equipment resource allocation information as input data, and outputs a Gantt chart of production progress and project duration forecast (in days). The training data includes over 6,000 schedules and over 3,000 resource data points. The Quality Inspection LoRa takes welding parameters (current, voltage) and quality inspection images as input data, and outputs quality pass / fail criteria and defect types such as porosity and cracks. The training set includes over 3,000 quality inspection reports and over 2,000 welding data points. The Cost Management LoRa takes material prices and processing time data as input data, and outputs cost accounting tables and material substitution suggestions. The training data includes over 5,000 cost accounting records and over 3,000 material price fluctuation data points.

[0063] For the operations management field, several specialized LoRays (Local Area Recognitions) are set up: Marine Inspection LoRay, Operations and Maintenance Management LoRay, Operations Management LoRay, and Route Planning LoRay. The input data for Marine Inspection LoRay consists of vessel technical files and IMO regulations, while the output data includes compliance assessment results and PDF inspection reports. Training data includes over 4,000 technical files and over 3,000 regulations. The input data for Operations and Maintenance Management LoRay consists of equipment operating data (temperature, vibration) and spare parts consumption records, while the output data includes fault diagnosis reports and spare parts replacement reminders. Training data includes over 5,000 equipment operating data points and over 2,000 spare parts records. The input data for Operations Management LoRay consists of operational data (such as voyage revenue and fuel costs), while the output data is operational analysis reports. Training data includes over 3,000 voyage operational records. The input data for Route Planning LoRay consists of route origin and destination, as well as meteorological data such as wind speed and waves. Output data includes route planning maps and fuel consumption prediction results (unit: tons / nautical mile). Training data includes over 7,000 route data points and over 4,000 meteorological records.

[0064] Step S106: Based on each trained professional adaptation layer in each domain, generate a domain-wide adapter and load the domain-wide adapter into the basic large language model to form the initial large ship model.

[0065] Among them, the full-domain adapter is an adaptation module that integrates knowledge from all sub-specialties across all domains, aiming to empower the basic large language model to understand different sub-specialty tasks in multiple domains of the ship.

[0066] Among them, the basic large language model refers to a general model that has been pre-trained on a massive amount of general text and has powerful language understanding and generation capabilities.

[0067] Optionally, the server integrates the professional knowledge from all the subdivided professional adaptation layers after training to form a unified global adapter, and loads the global adapter into the basic large language model to construct an initial large ship model with the rudiments of ship knowledge.

[0068] Step S108: Obtain a cross-domain training dataset covering various fields and a professional training dataset for stabilizing the performance of each sub-specialty task. Fine-tune the initial large ship model using the cross-domain training dataset and the professional training dataset until the preset conditions are met, and obtain the final large ship model. The final large ship model is used to respond to cross-domain tasks in the entire ship design process and output the response text for the cross-domain tasks.

[0069] Among them, cross-domain training datasets contain datasets of complex tasks that require the integration of knowledge from multiple domains to solve, such as "optimizing the power system configuration of a certain type of ship, which requires comprehensive consideration of engine performance, fuel economy, overall layout and structural strength."

[0070] The specialized training dataset is a refined dataset used to solidify the model's accuracy on a single specialized task.

[0071] The final large-scale ship model is an artificial intelligence model that has been trained and is capable of handling cross-disciplinary tasks in ship design as well as individual specialized tasks.

[0072] Optionally, the server uses more complex cross-domain task data and subdivided professional task data to consolidate the foundation to jointly fine-tune the initial large ship model, so that it can not only master each profession, but also solve cross-domain comprehensive problems in a comprehensive way, until the model performance meets the standards, and then deploy it as the final large ship model.

[0073] In the aforementioned cross-domain ship large-scale model training method, domain training datasets from multiple domains involved in the entire ship design process are obtained. For any domain, the domain training dataset is used to train the adaptation layers of each sub-specialty within that domain in parallel, resulting in each trained professional adaptation layer. Based on the trained professional adaptation layers of each domain, a full-domain adapter is generated and loaded into the basic large-scale language model to form the initial ship large-scale model. A cross-domain training dataset covering all domains and a professional training dataset used to stabilize the performance of each sub-specialty task are obtained. The initial ship large-scale model is then fine-tuned using both the cross-domain and professional training datasets until preset conditions are met, resulting in the final ship large-scale model. The final ship large-scale model is then used... This system responds to cross-domain tasks throughout the entire ship design process, outputting response texts for these tasks. This efficiently injects relevant domain expertise into different sub-specialty adaptation layers, constructing a comprehensive adapter specifically for ship design. The comprehensive adapter is then combined with a basic large-scale language model, and through joint fine-tuning with cross-domain and specialized training data, a final large-scale ship model is obtained. This overcomes the limitations of general-purpose large-scale language models lacking ship-specific expertise and single-specialty models being unable to perform cross-domain collaboration. Furthermore, it significantly optimizes the model's ability to handle complex and comprehensive tasks, enabling the final large-scale ship model to understand and accurately respond to complex ship design tasks involving multi-disciplinary collaboration, significantly improving the accuracy, reliability, and engineering practicality of the response content.

[0074] In an exemplary embodiment, each sub-specialty adaptation layer includes a first parameter matrix with trainable parameters and a second parameter matrix with fixed parameters. The first parameter matrix is ​​used to learn the professional task features of the domain. The second parameter matrix is ​​used to project the professional task features of the domain learned by the first parameter matrix onto the parameter space of the basic large language model. Using the domain training dataset, each sub-specialty adaptation layer of the domain is trained in parallel to obtain each trained professional adaptation layer of the domain. This includes: initializing the first parameter matrix of each sub-specialty adaptation layer of the domain to obtain the initialized first parameter matrix of each sub-specialty adaptation layer, and orthogonalizing the second parameter matrix of each sub-specialty adaptation layer of the domain to obtain the orthogonalized second parameter matrix of each sub-specialty adaptation layer. Different orthogonalized second parameter matrices are pairwise orthogonal. Using the domain training dataset of the domain, each sub-specialty adaptation layer of the domain is trained in parallel. During the parallel training process, the orthogonalized second parameter matrix of each sub-specialty adaptation layer is fixed, and only the initialized first parameter matrix of each sub-specialty adaptation layer is updated until the convergence condition is met to obtain each trained professional adaptation layer of the domain.

[0075] The first parameter matrix is ​​a learnable weight matrix used in the adaptation layer to extract and learn ship-specific task-specific patterns from the input. In this application, the first parameter matrix corresponds to the A matrix of the professional LoRA layer.

[0076] The second parameter matrix is ​​a fixed mapping matrix responsible for translating the learned specialized features and integrating them into the general feature space that the basic large language model can understand. In this application, the second parameter matrix corresponds to the B matrix of the specialized LoRA layer.

[0077] Optionally, the server defines a trainable low-rank matrix (i.e., the first parameter matrix) and a fixed parameter matrix (i.e., the second parameter matrix) for each sub-specialty adaptation layer. Before training begins, the server initializes the first parameter matrices of all sub-specialty adaptation layers to obtain the initialized first parameter matrices of each sub-specialty adaptation layer. The server also performs orthogonalization on the second parameter matrices of all sub-specialty adaptation layers to ensure that the second parameter matrices of different sub-specialty adaptation layers are orthogonal to each other. During training, these orthogonal second parameter matrices are fixed, and the first parameter matrix of each sub-specialty adaptation layer is updated using only the domain training dataset of its respective field.

[0078] In practical applications, the shipbuilding industry is divided into three major areas based on data distribution: R&D design, construction management, and operation management. Each area has its own dedicated professional-level LoRA (R&D design includes different professional-level LoRAs such as overall design, hull design, marine engineering, electrical engineering, and outfitting; construction management includes different professional-level LoRAs such as production planning, quality inspection, and cost management; and operation management includes different professional-level LoRAs such as marine inspection, maintenance management, and route planning). This application achieves rapid model convergence and cross-task non-interference through all-zero A-matrix initialization, domain-professional-level orthogonal B-matrix construction, and domain-specific training. The specific design process includes all-zero A-matrix construction, fully orthogonal B-matrix construction, and the construction of a domain-specific A-matrix training and B-matrix freezing mechanism.

[0079] In constructing the all-zero matrix A, the A matrix, as a trainable low-rank matrix for LoRA, should have its dimensions designed in conjunction with the data complexity of each domain (such as the number of paper components and parameter dimensions). The A matrix should be uniformly initialized as an all-zero matrix to ensure that there is no parameter cross-interference between the LoRAs of different domains before training. The formula and dimensions are set as follows: , characterizing the The first in the field The initialization method of the A matrix for each professional LoRA is given by the formula, where, The hidden layer dimension of the base model (uniformly set to 768 or 1024 to ensure cross-domain compatibility). Dynamic rank for LoRA in this domain-specific field (adapted to the complexity of domain-specific data to avoid parameter redundancy).

[0080] In constructing the all-zero A matrix, dynamic ranks are designed hierarchically by domain based on the data distribution characteristics of each professional field (e.g., ship hull design involves structural drawings of over 200 components, while maritime inspection in operation management only handles standard texts). For example, in the R&D design field (highly complex data, including drawings and structural parameters), its professional LoRA dynamic rank... Shipbuilding and marine engineering majors study complex mappings such as structural strength and power matching, and dynamic rank. The overall, electrical, and outfitting professional data dimensions are relatively simplified, and the dynamic order is... In the field of construction management (medium-complexity data, including process and quality inspection data), its professional LoRA dynamic rank... Production planning involves resource matching for over 50 processes, with dynamic order management. The data dimensions for quality inspection and cost management are relatively singular, and the dynamic order is limited. In the field of operations management (multimodal data, including text, time series, and meteorological data), its professional LoRA dynamic rank... Flight route planning processes meteorological data from more than 15 dimensions, with dynamic order. Maritime inspection and operation management primarily rely on text and time-series data, with dynamic order... .

[0081] After constructing a completely zero A matrix, a domain-specific training logic is adopted. That is, after the A matrix is ​​initialized, each domain's professional LoRA uses only the data distribution of its own domain for A matrix training (e.g., the ship hull LoRA in the R&D and design domain uses only ship structure drawings and strength formula data, and the quality inspection LoRA in construction management uses only welding parameters and quality inspection image data). The B matrix is ​​frozen throughout the training process to avoid interference from cross-domain data on parameter updates, ensuring that each professional LoRA can quickly learn the domain-specific features, and improving the convergence speed by more than 40%.

[0082] In constructing a fully orthogonal B matrix, the B matrix serves as a fixed low-rank matrix in LoRA, ensuring complete orthogonality between B matrices from different domains and between different specialties within the same domain (i.e., the inner product of any two B matrices is 0). This is achieved through a "domain characteristic matching orthogonality method," which ensures that cross-task interactions do not interfere with each other while adapting to the learning needs of data from different domains. Specifically, different B matrix orthogonalization methods are used for different domains.

[0083] For professional LoRA in the R&D and design field, the Gram-Schmidt orthogonalization method is adopted. The hull and marine engineering specialties in R&D and design process high-dimensional structural drawings (e.g., hull drawings contain more than 200 component parameters), and the data distribution exhibits a "high-dimensional sparse" characteristic. Therefore, a general orthogonal method can be used to ensure the accuracy of parameter learning. The steps include: Step 1, Initial vector set generation. For the five professional LoRAs in the R&D and design field, initial vector sets are generated separately. A set of random vectors of dimension Element obeys distributed( To ensure the variance and dimension of the initial vector are well-matched and to avoid numerical overflow, the following steps are performed: Step 2: Orthogonal calculations are performed by specialty. For the first specialty LoRA (e.g., hull LoRA), the initial vector is directly taken as the orthogonal vector. For subsequent specializations (such as marine engineering and overall LoRA), orthogonality is achieved by subtracting the projection of the already orthogonal vectors. The model is as follows: Step 3: Normalization and verification for each major. Perform L2 normalization ( ), verify the inner product of any two professional B matrices within the same field. (Error < 1e-6), while verifying the orthogonality of B matrices with other domains (e.g., the inner product of the B matrix with the production plan LoRA of construction management < 1e-6).

[0084] For the LoRA (Local Area of ​​Reference) in the construction management field, the stochastic orthogonal matrix QR decomposition method is adopted. In the construction management field, data such as production planning and quality inspection mainly consist of "process parameter tables and quality inspection texts." Therefore, a moderately dimensional but highly stable orthogonality guarantee can be adopted (to avoid parameter interference from resource matching and defect classification tasks). That is, the QR decomposition method is used to construct the B matrix. The steps include: Step 1, generating domain-specific stochastic matrices. For the three professional LoRAs in the construction management field, generate... dimensional random matrix ( (For the dynamic rank of this major), the elements obey... The distribution is such that the random matrix seed for each major is independent (to avoid initial correlation); Step 2, QR decomposition and orthogonal matrix extraction, for each Perform QR decomposition, decompose into In the formula, It is an orthogonal matrix (satisfying) , for The identity matrix (of order 1) is directly used as the B matrix of the LoRA for this major. The matrix is ​​an upper triangular matrix (used only for decomposition and not involved in subsequent calculations); Step 3: Cross-domain orthogonality verification, in addition to verifying the orthogonality of the three professional B matrices within the same domain ( In addition, it also verifies the orthogonality with the B matrix in the fields of R&D design and operations management (such as...). This ensures that cross-domain tasks are free from parameter interference.

[0085] For the LoRA (Local Area of ​​Reference) in the operations management domain, the Hadamard matrix construction method is adopted. The operations management domain contains four professional LoRAs, with data primarily consisting of "time-series operation and maintenance records, scheduling time-series data, etc." Balancing orthogonality and time-series feature learning efficiency, and given the number of professional domains is four (close to an integer power of 2), the Hadamard matrix construction method is used. The steps include: Step 1, generating a domain-level Hadamard matrix, generating an 8th-order Hadamard matrix using a recursive formula (completing to 2³ to accommodate the four professional domains and four spare dimensions, avoiding dimension waste): Step 2: Professional-level adaptation, ... Divide into 8 row vectors. The first four vectors are selected and assigned to the four professional LoRAs in operations management. Each vector is expanded to... Dimension (achieved by reusing vector elements in a loop to ensure adaptation to the periodic features of time series data); Step 3, orthogonality scaling and verification: scaling transformation is performed on each expanded vector to ensure the orthogonality of the B matrix. The verification criterion is that any two professional B matrices within the same field satisfy... (Error < 1e-6), and the inner product with the B matrix in the fields of R&D design and construction management is < 1e-6, completely blocking cross-task parameter interference.

[0086] In constructing the domain-specific A-matrix training and B-matrix freezing mechanism, a training strategy of "domain-specific parallel training, complete cross-domain isolation, and full-process B-matrix freezing" is adopted to achieve rapid convergence and cross-task isolation. To achieve domain-specific data isolation, independent training datasets are constructed for each of the three domains (the R&D design dataset contains over 12,000 structural drawings and specification texts; the construction management dataset contains over 8,000 process and quality inspection data; and the operation management dataset contains over 10,000 time-series and meteorological data). Each domain's professional LoRA only uses its own domain's training dataset for training, avoiding cross-domain data noise interference. To achieve B-matrix freezing, the B-matrix parameters of all domain-specific LoRAs are frozen using PyTorch's `requires_grad=False` attribute. During training, only the A-matrix is ​​updated, and the gradient constraint model is as follows: , To ensure the orthogonality of the B matrix is ​​not compromised, the training loss for this domain-specific LoRA is designed for this purpose. Because the A matrix is ​​initially all zeros and only learns features specific to this domain, parameter updates are more focused (e.g., the hull LoRA only optimizes parameters related to structural stress prediction). Compared to traditional full-scale fine-tuning, the number of convergence rounds for each domain's LoRA is reduced by 40% (e.g., the hull LoRA is reduced from 150 rounds to 90 rounds), and the parameter interference rate across tasks (e.g., "hull design" and "welding quality inspection") is reduced from 15% to <3%.

[0087] Furthermore, this application provides a professional LoRA differential fine-tuning parameter design method to balance model training efficiency and accuracy. By considering the complexity of various professional tasks (the level of detail in drawing design, the density of standardized terminology, and the difficulty of cross-disciplinary collaboration), differential fine-tuning parameters are designed, specifically providing dynamic rank... The design involves three parts: adapting the model, fine-tuning the parameters and loss function, and fine-tuning the core constraints.

[0088] (1) Dynamic Rank Adapt the model. Define task complexity metrics. , Task complexity metrics It combines the complexity of drawings and text. The number of components on a single drawing (unit: pieces). To standardize the term density of text (unit: terms / 100 words); according to The value of is used to classify the task complexity level, corresponding to different dynamic rank. :when Time (simple tasks, such as electrical wiring diagrams with 30 components and a terminology density of 5 per 100 words). A score of 6-8 is suitable for electrical, outfitting, and cost management professionals, such as those working with electrical LoRA. Set to 8; when Time (medium-level task, such as an overall scheme drawing containing 80 components and a terminology density of 10 terms per 100 words). A value of 10-14 is applicable to overall, production planning, and operations management specialties, such as overall LoRA. Set to 12; when (For complex tasks, such as ship structure drawings containing 200 components and a terminology density of 12 per 100 words). A range of 16-20 is suitable for disciplines such as hull design, marine engineering, and route planning, for example, hull LoRA. Set it to 18.

[0089] Dynamic rank-fit validation is performed every 20 rounds to calculate the rate of decrease in training loss. Improvement in validation accuracy , .like (Parameter redundancy), then proceed as follows Lower the rank value; if (Insufficient fit), then proceed as follows Increase the rank value to ensure Always adapt to task complexity ( For the first Training losses For the first (Round verification accuracy).

[0090] The essence of dynamic rank-fit verification is to transform "static rules" into "dynamic training process optimization", which not only ensures the accuracy of LoRA fine-tuning of different disciplines (matching task complexity), but also avoids the waste of efficiency caused by parameter redundancy, and achieves the dual advantages of "accuracy-efficiency" in LoRA fine-tuning of multiple disciplines such as electrical, hull, and overall systems.

[0091] (2) Fine-tuning parameters and loss function (subdivided by domain).

[0092] For the R&D design field, the overall LoRA dynamic rank is 12-16, and the optimizer used is AdamW (learning rate). Weight decay , , The loss function is ( Cross-entropy loss is used for drawing classification; The mean squared error loss (MSE) is used for principal scale parameter prediction. Training runs are 80-100 rounds, with acceptance criteria of drawing classification accuracy ≥85% and parameter prediction MAE <5%. The dynamic rank of the hull LoRA is 16-20, and the optimizer parameters are... Weight decay , , The loss function is ( The KL divergence loss is used for structural stress rule constraints. The training runs for 100-120 rounds, with acceptance criteria of stress prediction error <3% and rule compliance rate ≥92%. The dynamic rank of the engine LoRA is 14-18, and the optimizer parameters are... Weight decay , , The loss function is ( Used for fault classification (Used for engine power optimization), training rounds 90-110, acceptance criteria are fault classification accuracy ≥88% and power optimization error <4%; dynamic rank of electrical / outfitting LoRA 8-12, optimizer parameters are... Weight decay , , The loss functions are all (For drawing classification), 60-80 training rounds, with the acceptance criterion being a drawing classification accuracy rate of ≥85%.

[0093] For the construction management domain, the dynamic rank of LoRA in production planning is 10-14, and the optimizer parameters are... Weight decay , , The loss function is ( Used for project schedule prediction (used for resource matching), training rounds 70-90, acceptance criteria are: project duration prediction error <7%, resource matching accuracy ≥86%; quality check: LoRA dynamic rank 6-10, optimizer parameters are... Weight decay , , The loss function is ( Used for defect classification (For welding parameter compliance constraints), training rounds 50-70, acceptance criteria are defect classification accuracy ≥88%, welding compliance ≥93%; cost management LoRA dynamic rank 8-12, optimizer parameters are... Weight decay , , The loss function is (For cost prediction), 60-80 training rounds, with the acceptance criterion being a cost prediction error of <5%.

[0094] For the operations management field, the dynamic rank of LoRA in maritime inspection is 8-12, and the optimizer parameters are: Weight decay , , The loss function is (For compliance assessment), training rounds 60-80, acceptance metric: compliance assessment accuracy ≥ 85%; dynamic rank of LoRA for operations and maintenance management 10-14, optimizer parameters are as follows. Weight decay , , The loss function is ( Used for fault prediction (used for spare parts replacement cycle prediction), training rounds 70-90, acceptance criteria are fault prediction accuracy ≥85%, replacement cycle prediction error <6%; dynamic rank of LoRA for operations management is 8-12, optimizer parameters are... Weight decay , , The loss function is (Used for operational indicator prediction), training rounds 60-80, acceptance criterion is operational indicator prediction error <5%; dynamic rank of LoRA for route planning 12-16, optimizer parameters are as follows. Weight decay , , The loss function is ( Used for fuel consumption prediction (Used for route risk classification), with 80-100 training rounds, and the acceptance criteria are fuel consumption prediction error <6% and route risk identification accuracy ≥87%.

[0095] (3) Fine-tuning core constraints. The B-matrix freezing constraint refers to freezing the B-matrix parameters throughout training using PyTorch's `requires_grad=False` to ensure orthogonality is not violated. Mathematically, this is expressed as... ( For the total loss of professional LoRA, For the first The system uses a B matrix for each specialty, updating only the A matrix to reduce computation. Overfitting control is also employed, combining an early stopping strategy with Dropout. The early stopping strategy halts training when the validation set accuracy shows no improvement for five consecutive rounds. The Dropout probability is set to 0.1, and the A matrix parameters are randomly deactivated. , To reduce overfitting, a mask matrix that follows a Bernoulli (0.1) distribution is used, ensuring that the training-validation loss difference is less than 5%.

[0096] In this embodiment, by designing a learnable feature extractor (first parameter matrix) and a fixed orthogonalization mapper (second parameter matrix) for each professional adaptation layer, and fixing the mapper and training only the feature extractor during training, high-efficiency and low-interference parallel professional knowledge injection is achieved. The orthogonalization of the second parameter matrix ensures that the representation directions of knowledge learned from different professions are decoupled and complementary when integrated into the base model, rather than chaotic or conflicting. This fundamentally solves the common problem of knowledge forgetting or knowledge confusion when injecting multi-domain knowledge into a large language model at the same time. It lays a solid foundation for the subsequent generation of a full-domain adapter that can evenly master professional knowledge from various domains and has a clear internal knowledge structure. This is a key design for improving the cross-domain response accuracy of the final model.

[0097] In an exemplary embodiment, a full-domain adapter is generated based on each trained professional adaptation layer in each domain, including: determining the domain weights of each domain and the professional weights of each sub-specialty within each domain; determining the fusion weights of each sub-specialty within each domain based on the domain weights of each domain and the professional weights of each sub-specialty within each domain; using the fusion weights of each sub-specialty within each domain to perform weighted fusion of the parameter matrices of each trained professional adaptation layer in each domain to obtain the domain parameter matrix corresponding to each domain; and performing weighted fusion of the domain parameter matrices corresponding to each domain according to the domain weights of each domain to obtain the full-domain parameter matrix, thereby generating the full-domain adapter.

[0098] The domain weight reflects the importance of that domain in the overall ship design.

[0099] Among them, the professional weight reflects the training accuracy and data volume ratio of a specific professional field.

[0100] Among them, the fusion weight is the final weighted coefficient assigned to each sub-professional adaptation layer after combining the domain weight and the professional weight.

[0101] Optionally, the server assigns a domain weight to each domain and a professional weight to each sub-specialty within each domain. Then, by multiplying the domain weight by the professional weight, the fusion weight of each sub-specialty adaptation layer is calculated. Next, using these fusion weights, the first parameter matrices of all sub-specialty adaptation layers within the same domain are weighted, and the second parameter matrices of all sub-specialty adaptation layers within the same domain are weighted to generate a domain first parameter matrix and a domain second parameter matrix representing that domain. Finally, using the domain weights of each domain, all domain first parameter matrices are weighted to ultimately fuse them into a single full-domain first parameter matrix, and using the domain weights of each domain, all domain second parameter matrices are weighted to ultimately fuse them into a single full-domain second parameter matrix. The full-domain first parameter matrix and the full-domain second parameter matrix are the core of the full-domain adapter.

[0102] This embodiment provides a parameter fusion strategy. This strategy, based on the successful training of LoRA models across various sub-domains (acceptance accuracy ≥85% for R&D and design, ≥86% for construction management, and ≥85% for operation management), employs a two-step approach—from domain-level fusion to full-domain fusion—to fuse the parameters of 12 professional LoRA models into a unified industry model. Simultaneously, it preserves the orthogonality of each domain's professional characteristics, ensuring that the fused model maintains its "cross-task non-interference" feature. Specifically, it includes:

[0103] Step 1: Integration weight calculation, with a dual weighting mechanism of domain and professional level. The integration weight considers both "professional performance" and "domain importance" to avoid a single domain dominating the integration result and ensure the coordinated balance of R&D design (core design stage), construction management (implementation stage), and operation management (full life cycle stage).

[0104] Specifically, the first step is to calculate the professional-level base weights, which are the domain-specific training accuracy based on LoRA for each profession. With the proportion of data volume Calculate professional-level basic weights ,in, For the number of professionals in this field (R&D design) Construction Management ,operations management ), ( (This refers to the number of training samples for this major).

[0105] The second step involves adjusting domain weights, which means setting domain weights based on the importance of each domain throughout the entire ship design process. (Research and development design) Construction Management ,operations management ), the fusion weight is .

[0106] For example, if the research and design field of ship hull LoRA ( , 10,000 items, within the field LoRA (Quality Inspection) in the Construction Management Field , 10,000 items, within the field Then the professional-level basic weight , The fusion weight is , In this way, it not only reflects the high accuracy of the hull specialization, but also takes into account the importance of implementation in the construction process (the quantitative benchmark of matrix fusion effect is used to calibrate the feature contribution ratio of each specialty and the performance fluctuation of the attribution model (for example, when the accuracy of a certain task decreases, the specialty with insufficient contribution can be located by fusion weight)).

[0107] Step 2: Matrix fusion calculation to preserve orthogonality and transfer domain characteristics. Matrix fusion consists of two steps: "domain-level fusion → full-domain fusion," and the orthogonality of the B matrix is ​​maintained throughout the process. This ensures that the fused model can still distinguish the task characteristics of each domain and avoid cross-task interference.

[0108] Specifically, the first step is to perform domain-level matrix fusion, that is, within each domain, the A / B matrix of all professional LoRAs in that domain is fused according to the professional-level basic weights. The fusion yields a domain-level fusion matrix, and the model is as follows: , ,in This is the A matrix trained using LoRA for this specific field. This is the initially constructed orthogonal B matrix. Since the B matrices of various specialties within the same domain are already orthogonal, the merged domain-level B matrix... It still maintains orthogonality with B matrices from other domains (such as...) This ensures feature isolation between domains, meaning that orthogonality can still be guaranteed.

[0109] The second step is to perform full-domain matrix fusion, which involves merging the fusion matrix of the three major domains according to their domain weights. By merging, a matrix of industry models across all domains is obtained, and the model is... , After fusion It still satisfies orthogonality (the inner product of the eigenvectors of any two domains is <1e-6), and because the A matrix integrates the best features of each domain, the full-domain model achieves an accuracy of 82% in cross-domain tasks (such as "hull design → welding quality inspection → operation and maintenance cycle prediction"), while the performance fluctuation of single-domain tasks (such as hull stress prediction) is <2%, achieving the goal of "cooperation without interference and convergence without deceleration".

[0110] In this embodiment, a two-layer weighted fusion mechanism is introduced, and weights are determined based on domain importance and professional representativeness. This enables the intelligent generation of a unified and balanced full-domain adapter from numerous discrete professional adaptation layers. This solves the problems that may arise from simple splicing or average fusion, such as "most professional fields overshadowing key niche fields" or "high-frequency tasks dominating the model while ignoring low-frequency important tasks." Weighted fusion ensures that the final full-domain adapter not only includes all necessary professional knowledge, but also that the weight distribution of the knowledge structure conforms to the actual needs of naval architecture. This allows the large model loaded with this adapter to more rationally call upon and weigh knowledge from different domains when dealing with cross-domain problems, thereby outputting a more accurate response that better aligns with engineering practice priorities.

[0111] In an exemplary embodiment, the cross-domain training dataset includes task samples of multiple cross-domain tasks; each task sample of a cross-domain task includes the cross-domain task and its standard response text; the professional training dataset includes task samples of multiple sub-professional tasks; each task sample of a sub-professional task includes the sub-professional task and its standard response text; the initial large-scale ship model is fine-tuned using the cross-domain training dataset and the professional training dataset until preset conditions are met to obtain the final large-scale ship model, including: inputting each cross-domain task into the initial large-scale ship model to obtain the predicted response text of each cross-domain task, and inputting each sub-professional task into the initial large-scale ship model to obtain the predicted response text of each sub-professional task; determining cross-domain loss information based on the predicted response text and standard response text of each cross-domain task, and determining sub-professional loss information based on the predicted response text and standard response text of each sub-professional task; determining total loss information based on the cross-domain loss information and sub-professional loss information; and fine-tuning the initial large-scale ship model based on the total loss information until preset conditions are met to obtain the final large-scale ship model.

[0112] Cross-domain loss information measures the difference between the model's output and the standard answer when dealing with complex tasks that require the integration of knowledge from multiple domains. It focuses on evaluating the model's comprehensive reasoning and collaborative abilities.

[0113] Among them, the subdivided professional loss information is a loss value that measures the accuracy of the model's output when processing standard tasks in a single professional field, focusing on consolidating and stabilizing the model's professional basic capabilities.

[0114] Among them, the total loss information is an overall optimization goal that integrates cross-domain capabilities and professional basic capabilities.

[0115] Optionally, during the fine-tuning phase, the server samples from two datasets to form training batches. For each batch, the cross-domain tasks and sub-specialty tasks contained therein are input into the initial large ship model to obtain the corresponding predicted answers. Then, two losses are calculated in parallel: one based on the prediction and standard answer of the cross-domain task, and the other based on the prediction and standard answer of the specialty task. Next, the two losses are weighted and summed according to a preset ratio to obtain the total loss. By iteratively optimizing this total loss, the model is simultaneously driven to improve its ability to solve complex and comprehensive problems while maintaining the accuracy of its specialty tasks.

[0116] This embodiment provides a collaborative training method aimed at improving the accuracy of cross-domain tasks while maintaining stable performance in individual disciplines. The training objective is an accuracy of ≥82% for cross-domain tasks (e.g., "hull component thickness → welding current → maintenance cycle") and a performance fluctuation of <2% for individual discipline tasks (to avoid loss of professional accuracy due to fusion). The loss function used is a hybrid loss function, which can be expressed as: ,in, For cross-domain task losses (such as "hull → welding" matching tasks), adopt ), For the first For each specialty's single-task loss, a weight allocation of 0.7 and 0.3 ensures a balance between cross-domain collaboration and specialty accuracy. When updating parameters, only the initial A matrix across the entire domain is updated. ,freeze To avoid parameter drift affecting individual subject performance, the original A matrix for each subject is used. The training epochs are set to 20-30 epochs (shorter epochs to avoid overfitting), and validation is performed every 5 epochs. If the performance of an individual subject decreases by more than 2%, the frequency is reduced. Increase the weight to 0.5. If the cross-domain accuracy is less than 80%, the cross-domain data ratio is increased to 40%, and training is restarted to ensure that the training direction is in line with the goal.

[0117] In this embodiment, by designing a joint fine-tuning objective that simultaneously incorporates cross-domain losses and subdivided professional losses, the final large-scale ship model is ensured to be not only an "expert" in various fields, but also a "chief engineer" capable of efficient inter-domain collaboration. This solves the problem that fine-tuning using only cross-domain data may lead to a weak professional foundation for the model, or that fine-tuning using only professional data may lead to the model being unable to handle complex problems. This enables the model to provide a reliable response with sufficient professional depth and comprehensive global considerations when facing real, ambiguous cross-domain tasks in the entire ship design process, thereby significantly improving the accuracy of the response and its engineering value.

[0118] In an exemplary embodiment, the method further includes: obtaining a natural language task input by the user; the natural language task is a subdivided professional task or a cross-domain task; retrieving terms associated with the natural language task and term attributes of the terms from a pre-built ship engineering terminology knowledge base, and generating contextual prompts adapted to the natural language task based on the terms and term attributes; and inputting the contextual prompts and the natural language task into the final ship large model to obtain the answer text for the natural language task.

[0119] Among them, the Marine Engineering Terminology Knowledge Base is a structured database that stores professional terms in the field of ship design, their standard definitions, parameter ranges, related systems, commonly used units and other attribute information.

[0120] Among them, the contextual hints are background knowledge texts that are dynamically retrieved and organized based on the user's question, used to supplement the professional context that may be implicit or missing in the question for the large model.

[0121] Optionally, when a user asks a question, the server first performs natural language understanding on the question to extract key entities and intents. Then, it retrieves relevant terms and their detailed attributes from the ship engineering terminology knowledge base. Next, it organizes these retrieved terms and attributes into a coherent explanatory text as augmented input. Finally, it feeds this augmented input into the final ship model, which, guided by a fully professional context, can generate a more accurate answer.

[0122] In this embodiment, by introducing a context enhancement mechanism based on a professional terminology knowledge base, contextual prompts can be dynamically generated for the natural language task input by the user. This completes the professional background and implicit constraints of the problem for the model, greatly improving the model's understanding of the user's intent and the accuracy of its grasp of professional concepts. As a result, the final response text is more accurate, reliable and in line with industry standards, further consolidating the model's ability to solve the core problem of inaccurate response to cross-domain tasks.

[0123] This application provides a semantic embedding method for professional terminology to address the semantic gap problem, specifically including the construction of a dedicated terminology dictionary, the establishment of a dynamic Prompt generation mechanism, and accurate semantic mapping of marine professional terminology.

[0124] First, a specialized terminology dictionary for the shipbuilding industry needs to be constructed. This construction involves three steps: terminology collection, structured terminology annotation, and dictionary updates to ensure comprehensive coverage, complete attributes, and timely updates. In terminology collection, a combination of automatic extraction and manual screening is used. Automatic extraction, based on the TF-IDF+TextRank algorithm, extracts candidate terms (such as roll period, structural redundancy, and slam load) from over 100,000 shipbuilding specification texts (IMO conventions, GB / T standards, and enterprise process documents). Manual screening is performed by shipbuilding engineers with over 5 years of experience to eliminate ambiguous terms (such as terms where the period is clearly defined as roll period rather than construction period), resulting in a terminology database containing over 1200 core terms. In the structured terminology annotation, each term is annotated with 6 attributes to ensure semantic integrity. Attributes include term ID (e.g., RD-HULL-001), term name (e.g., structural redundancy), domain affiliation (e.g., R&D design - hull), and object... The dictionary includes the following parameters: theoretical connotation (e.g., the degree to which the remaining structure can maintain its load-bearing capacity after partial failure of the ship structure, usually expressed by the redundancy coefficient K = actual load-bearing capacity / design load-bearing capacity), related parameters (e.g., ["structural material strength", "component cross-sectional dimensions", "load distribution"]), application scenarios (e.g., "ship structure anti-sinking design, bulkhead strength verification"), and long-tail adaptation tags (e.g., "new environmentally friendly ships (e.g., LNG ships) additionally consider the impact of low temperature on the redundancy coefficient, and the K value is reduced by 5%-8%)). In dictionary updates, industry standards are updated quarterly. New terms are automatically added to the database after expert review (pass rate ≥90%), simultaneously triggering an update to the semantic mapping module to ensure the timeliness of the terminology database and cover terms from emerging fields such as carbon capture systems and ammonia fuel power.

[0125] Secondly, a dynamic Prompt generation mechanism is established. Specifically, the dynamic Prompt generator is based on a small Transformer encoder-decoder architecture (256 hidden layer dimensions, 3 layers), with training data consisting of over 5000 "task-terminology-high-quality Prompt" triples. The generation architecture includes: an input layer, which receives the user's natural language task (e.g., please explain the structural redundancy requirements of an LNG ship) and the associated attributes (physical connotation, long-tail tags, application scenarios) of the terminology dictionary, converting the text into vector representations using a BERT pre-trained in the shipbuilding domain; and a generation layer, which generates a structured Prompt containing the physical connotation of the terminology, associated parameters, and scenario constraints based on the task and terminology attributes. An example is: "In the hull specialty of ship research and design, structural redundancy refers to [physical connotation], its related..." The parameters include [Association Parameters]. For the new environmentally friendly LNG ship scenario, the impact of low temperature on the redundancy coefficient K is additionally considered (K is reduced by 5%-8%). Please explain the structural redundancy requirements of LNG ships in conjunction with the new environmental regulations: [User Task]”, ensuring that the Prompt supplements the semantics of the terms without introducing redundant information; the output layer is used to adaptively adjust the length of the Prompt according to the complexity of the task. The length of the Prompt for simple tasks (such as terminology definition query) is controlled within 50 characters, and the length of the Prompt for complex tasks (such as specification interpretation) is controlled within 150 characters. It also supports the generation of multi-term associated Prompts (such as “explain the relationship between structural redundancy and welding process”, associating the attributes of two terms).

[0126] The generator provides adaptive adjustment logic for generating Prompts, mainly targeting two scenarios: If the input task contains long-tail terms (such as "carbon capture system redundancy"), the generator automatically extracts the "long-tail adaptation tag" of the term from the dictionary and adds it to the end of the Prompt (such as "carbon capture system redundancy considers the impact of CO2 corrosion on structural strength"); If the task involves the association of multiple terms (such as "the relationship between structural redundancy and slam load"), the generator automatically associates the physical connotations and association parameters of the two terms (such as "the K value of structural redundancy combined with the peak value and frequency adjustment of slam load"), constructs an associative Prompt, and clarifies the logical relationship between the terms.

[0127] Finally, semantic mapping of terms is performed. Based on the pre-trained BERT in the shipbuilding domain and contrastive learning, accurate semantic mapping of professional terms is achieved. The process is as follows: Step 1, Initial semantic vector generation: Input the terms in the terminology dictionary into the pre-trained BERT in the shipbuilding domain (with a base semantic matching accuracy of 75%) to obtain the initial semantic vector. Step 1: Capture the basic semantics of terms; Step 2: Prompt vector fusion, using context pairs generated by dynamic Prompt. Fine-tuning is performed to obtain the adaptation vector. The model is In the formula, (Prompt weighting, balancing the term itself with the semantics of the context). The vector representation of the dynamic prompt (generated through the same BERT model) is fused to make the semantics of the terminology more relevant to the maritime domain; Step 3, contrastive learning optimization, uses contrastive learning to separate the semantics of specialized terms from general vocabulary, and adopts InfoNCE loss as the loss function. In the formula, (Cosine similarity) A positive sample vector (a term in the same field, such as "structural redundancy" corresponding to "sink resistance"). For negative sample vectors (common terms such as "redundancy" and "period") ), (Temperature parameter), the target is to make , This ensures that specialized semantics are not interfered with by general semantics.

[0128] Long-tail scenario adaptation optimizes for new environmentally friendly ship terminology (such as "thermal efficiency of ammonia-fueled ships" and "pressure loss of carbon capture systems"). Only 50-100 pieces of labeled data (such as mapping samples between terms and regulatory clauses) are used to fine-tune the semantic mapping layer parameters, updating the model accordingly. ( Output projection layer parameters for BERT. To adapt to the learning rate, the adaptation time is less than 24 hours, and the long-tail term matching accuracy is ≥85%, meeting the needs of emerging fields in the industry.

[0129] In an exemplary embodiment, the method further includes: monitoring the update status of the ship engineering rules knowledge base associated with the final ship large model; when the update status meets the preset model training triggering conditions, acquiring each update rule and the knowledge data associated with each update rule, and constructing an incremental training dataset based on each update rule and the knowledge data associated with each update rule; and training the final ship large model using the incremental training dataset until the preset conditions are met.

[0130] Among them, the Naval Engineering Rules Knowledge Base is a knowledge system that stores mandatory or recommended rules and their interpretations, such as international conventions, classification society rules, and industry standards.

[0131] The update status refers to the addition and revision status of rules in the knowledge base.

[0132] The incremental training dataset is a set of training samples constructed based on updated rule clauses and related technical interpretations, used to update the knowledge of existing models.

[0133] Optionally, the server continuously monitors version changes of the ship engineering rules knowledge base. When an important rule update is detected, it automatically extracts the newly added or modified rule clauses, their official interpretations, and application cases. Then, using these new materials, it automatically constructs question-and-answer pairs and instruction-following samples to form an incremental training dataset. Finally, it uses this incremental training dataset to incrementally train the deployed final ship model, enabling the model to quickly learn and master the latest regulatory requirements. After the update is completed, it can be put into use without retraining from scratch.

[0134] In this embodiment, by establishing a linkage mechanism between rule knowledge base updates and model capability updates, the sustainable evolution of the large ship model is achieved. This solves the problem that traditional large models, once trained, have their knowledge frozen due to frequent updates to ship design standards, making them unable to adapt to new rule requirements and resulting in outdated or even erroneous responses. This ensures that the model's knowledge system can be synchronized with the latest industry standards, enabling it to reason and respond based on currently valid rules at any time, maintaining the accuracy and reliability of its output over the long term. This is a key operational strategy to ensure that the large ship model can always provide accurate and reliable responses to cross-domain tasks in a dynamically changing engineering environment.

[0135] This embodiment provides a continuous pre-training method for rule embedding to address the lag problem in dynamic rule adaptation. Considering the dynamic iterative characteristics of engineering rules, this embodiment designs a trigger-based continuous pre-training mechanism to achieve rapid rule embedding and model adaptation. Specifically, it includes:

[0136] Step 1: Construct a knowledge base for shipbuilding engineering rules. The knowledge base covers rules for the entire shipbuilding industry process and adopts a structured storage and dynamic update mechanism.

[0137] The rules are sourced from various sources, including the SOLAS Convention, MARPOL Convention, GB / T shipbuilding standards (such as GB / T3354-2014), and internal enterprise process documents (such as welding operation instructions and quality inspection manuals), covering the entire process of design, construction, and operation. The storage architecture adopts a dual storage approach of "MySQL + Elasticsearch". MySQL stores structured rule fields (rule ID, type, parameter range, etc.), while Elasticsearch supports full-text search of rules (such as searching for rules related to "LNG ship welding").

[0138] The rule fields include rule ID (e.g., WELD-2025-001), rule type (enumerated values: welding process, structural strength, maritime compliance, etc.), clause content (e.g., "When the welding ambient temperature of an LNG ship is ≤-10℃, the welding current shall be reduced by 15%), parameter range (JSON format, e.g., {"current":"original current × 0.85","temperature":"≤-10℃"}), associated professional LoRA (array format, e.g., ["hull LoRA","quality inspection LoRA"]), effective date (e.g., 2025-01-01), and priority, which indicates the importance of the ship rule (1 is the highest, 3 is the lowest), and is used in the loss function to adjust the constraint weights of the new rule. (Priority 1) When priority is 3 This allows the model to focus more on adapting high-priority rules.

[0139] The update mechanism combines automatic and manual updates. During automatic updates, the system connects to the IMO official website API, captures new regulation texts every quarter, and automatically adds them to the database after being reviewed by domain experts with more than 5 years of experience (pass rate ≥80%). During manual updates, enterprise users submit custom rules (such as internal welding process standards) through the web interface, fill in the parameter range and related professional fields, and trigger the pre-training process after the review is approved.

[0140] Step 2: Construct a trigger-based continuous pre-training process.

[0141] First, set pre-training trigger conditions. Continuous pre-training will automatically start when any of the following conditions are met to ensure timely rule embedding: rule update trigger, i.e., the number of newly added or modified rules in the rule base is ≥5 (e.g., adding 5 new rules for cryogenic welding of LNG ships), avoiding frequent triggers that lead to resource waste; data increment trigger, i.e., adding ≥1000 new cross-domain labeled data (referring to the labeling of linked task samples of different ship types) (e.g., adding "ammonia fuel ship R&D-construction" related data), supplementing the samples needed for rule adaptation; performance degradation trigger, i.e., the accuracy of core tasks (standard Q&A, parameter prediction) decreases by ≥3% (e.g., the accuracy of welding parameter prediction drops from 92% to 88%), restoring performance through pre-training.

[0142] Secondly, input pre-training data. The pre-training data contains four categories, ensuring the accuracy of rule embedding and preventing the forgetting of historical knowledge. The pre-training data includes initial model parameters across all domains, i.e., the fine-tuned parameters after mixing. and The system includes: a pre-trained foundational model to ensure cross-domain collaboration capabilities; new rule data (5-20 rules, including embedded new rules and their corresponding parameter ranges, and related professional information, such as the rule "LNG ship welding current reduced by 15%" associated with "hull LoRA + quality inspection LoRA"); a historical data subset (10% of the LoRA training sets of each profession, approximately 3000 rules, to prevent the model from forgetting historical knowledge, such as existing welding process rules); and rule association data (500-1000 rules, which are the labeled data corresponding to the new rules, such as low-temperature welding parameter samples, annotating the mapping relationship between the new rule parameters and professional tasks, such as -10℃ ambient temperature corresponding to 180A welding current, improving the accuracy of new rule adaptation).

[0143] Finally, rule embedding is performed. Using the "Related Specialty LoRA" tag of the rule, the updated sub-region of the A matrix is ​​located. Parameter updates are performed only on related regions to avoid interference with parameters from unrelated specialties. This can be represented as follows: In the formula, , ( For learning rate, (This refers to the total loss during continuous pre-training). The continuous pre-training loss function employs a multi-objective loss function, balancing new rule constraints, cross-domain collaboration, and parameter consistency, and is expressed as follows: In the formula, (New regulations constrain weights) (Cross-domain collaborative weighting) (Parameter consistency weight); Adjustments to the KL divergence loss under the new rules, based on rule priority. (Priority 1) When priority is 3 (Only adjusting the constraint weights of the new regulations) Cross-domain collaborative weights Consistency weights with parameters (Keep it fixed to ensure cross-domain capabilities and parameter stability) ( To increase the number of rules, For the new rule probability distribution, (Predicting distribution for the model). To mitigate cross-domain task losses, ensure that cross-domain capabilities do not decline after pre-training; To account for parameter consistency loss, the difference between the A matrix before and after the update is calculated using the L2 norm: ,Target To avoid parameter drift.

[0144] The iterative training parameter optimizer uses Adam, with an initial learning rate of... If every 2 rounds of verification If the decline rate is less than 1%, then The training rounds are set to 5-10 rounds (shorter rounds for faster adaptation). An early stop strategy is used during training. and Stop training when necessary to balance efficiency and accuracy.

[0145] To facilitate understanding by those skilled in the art, the overall technical architecture of this application is described below. The overall technical architecture of this application includes a professional LoRA layer, a hybrid fusion layer, and a rule-based persistent embedding layer.

[0146] The specialized LoRA layer refers to different specialized LoRA sub-modules for various fields, subdivided based on the characteristics of ship data distribution. The R&D and design field includes different specialized LoRAs such as overall design, hull structure calculation, engine system selection, electrical wiring design, and outfitting, corresponding to core tasks such as overall scheme design, hull structure calculation, engine system selection, electrical wiring design, and outfitting equipment layout. The construction management field includes different specialized LoRAs such as production planning, quality inspection, and cost management, covering construction process planning, welding quality inspection, and construction cost accounting scenarios. The operation management field includes different specialized LoRAs such as maritime inspection, operation and maintenance management, operation management, and route planning, supporting maritime compliance judgment, equipment fault diagnosis, operation data management, and optimal route generation functions. Each specialized LoRA is independently initialized and fine-tuned to ensure the accuracy of specialized tasks.

[0147] The hybrid fusion layer is responsible for cross-disciplinary and cross-domain parameter coordination and performance optimization, and includes three core modules. The cross-disciplinary parameter fusion module calculates the fusion weights based on the fine-tuning accuracy and data volume of each discipline's LoRA, realizing the weighted integration of parameters from multiple disciplines within the same domain; the cross-domain collaborative training module constructs cross-domain related datasets (such as the "hull components → welding process → maintenance cycle" triples), and optimizes the cross-domain collaborative capability of the full-domain model through a hybrid loss function; the performance verification module monitors the cross-domain task accuracy and single-discipline performance fluctuations in real time, ensuring that the fused model satisfies the collaborative requirements without losing professional accuracy.

[0148] The continuous rule embedding layer, consisting of three modules, enables dynamic updates and model adaptation of engineering rules. The rule base management system employs a dual-storage architecture of MySQL and Elasticsearch to store structured rules such as GB / T shipbuilding standards and enterprise process documents, supporting both automatic (quarterly updates via IMO's official API) and manual (user web interface submissions) updates. The trigger-based pre-training module automatically initiates pre-training when the number of rule updates is ≥5, the number of newly added cross-domain data entries is ≥1000, or the accuracy of core tasks decreases by ≥3%. The parameter consistency maintenance module constrains the differences in model parameters before and after updates using the L2 norm, preventing the forgetting of historical knowledge due to rule embedding.

[0149] In another embodiment, such as Figure 2 As shown, a cross-domain method for training large ship models is provided. Taking the application of this method to a server as an example, the method includes the following steps:

[0150] Step S202: Obtain domain training datasets for multiple fields involved in the entire ship design process.

[0151] Step S204: For any domain, use the domain training dataset to train each sub-specialty adaptation layer of the domain in parallel, and obtain each trained specialty adaptation layer of the domain.

[0152] Step S206: Determine the domain weight of each domain and the professional weight of each sub-specialty under each domain, and determine the fusion weight of each sub-specialty under each domain based on the domain weight of each domain and the professional weight of each sub-specialty under each domain.

[0153] Step S208: Using the fusion weights of each sub-specialty under each domain, the parameter matrices of each trained professional adaptation layer in each domain are weighted and fused to obtain the domain parameter matrix corresponding to each domain.

[0154] Step S210: According to the domain weights of each domain, the domain parameter matrices corresponding to each domain are weighted and fused to obtain the full domain parameter matrix, so as to generate the full domain adapter, and load the full domain adapter into the basic large language model to form the initial ship large model.

[0155] Step S212: Obtain cross-domain training datasets covering various fields and professional training datasets used to stabilize the performance of each sub-specialty task. Fine-tune the initial large ship model using the cross-domain training datasets and professional training datasets until the preset conditions are met, and obtain the final large ship model. The final large ship model is used to respond to cross-domain tasks in the entire ship design process and output response text for cross-domain tasks.

[0156] It should be noted that the specific limitations of the above steps can be found in the specific limitations of a cross-domain ship large model training method described above.

[0157] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0158] The cross-domain ship large model training device provided in the embodiments of this application is described below. The cross-domain ship large model training device has the same inventive concept as the cross-domain ship large model training method described above. The solution to the problem provided by the device is similar to the solution described in the above method. Therefore, the specific limitations of one or more cross-domain ship large model training device embodiments provided below can be referred to the limitations of the cross-domain ship large model training method above. The cross-domain ship large model training device described below and the cross-domain ship large model training method described above can be referred to each other, and will not be repeated here.

[0159] In one exemplary embodiment, Figure 3 A schematic diagram of a cross-domain large-scale ship model training device provided in this application embodiment is shown below. Figure 3 As shown, this cross-disciplinary ship large model training device includes: a data acquisition module 302, a parallel training module 304, an adapter loading module 306, and a model fine-tuning module 308, wherein:

[0160] The data acquisition module 302 is used to acquire domain training datasets for multiple fields involved in the entire ship design process;

[0161] Parallel training module 304 is used to train each sub-professional adaptation layer of the domain in parallel using the domain training dataset for any domain, and obtain each trained professional adaptation layer of the domain.

[0162] The adapter loading module 306 is used to generate a full-domain adapter based on each trained professional adaptation layer in each domain, and load the full-domain adapter into the basic large language model to form the initial large ship model.

[0163] The model fine-tuning module 308 is used to acquire cross-domain training datasets covering various fields and professional training datasets used to stabilize the performance of each sub-specialty task. The initial large ship model is fine-tuned using the cross-domain training datasets and professional training datasets until the preset conditions are met, resulting in the final large ship model. The final large ship model is used to respond to cross-domain tasks in the entire ship design process and output response text for the cross-domain tasks.

[0164] In an exemplary embodiment, each sub-specialty adaptation layer includes a first parameter matrix with trainable parameters and a second parameter matrix with fixed parameters. The first parameter matrix is ​​used to learn the professional task features of the domain. The second parameter matrix is ​​used to project the professional task features of the domain learned by the first parameter matrix onto the parameter space of the basic large language model. The parallel training module 304 is specifically used to initialize the first parameter matrix of each sub-specialty adaptation layer in the domain to obtain the initialized first parameter matrix of each sub-specialty adaptation layer, and to orthogonalize the second parameter matrix of each sub-specialty adaptation layer in the domain to obtain the orthogonalized second parameter matrix of each sub-specialty adaptation layer. Different orthogonalized second parameter matrices are pairwise orthogonal. Using the domain training dataset of the domain, each sub-specialty adaptation layer in the domain is trained in parallel. During the parallel training process, the orthogonalized second parameter matrix of each sub-specialty adaptation layer is fixed, and only the initialized first parameter matrix of each sub-specialty adaptation layer is updated until the convergence condition is met, thus obtaining each trained specialty adaptation layer in the domain.

[0165] In an exemplary embodiment, the adapter loading module 306 is specifically configured to determine the domain weight of each domain and the professional weight of each sub-specialty under each domain, and based on the domain weight of each domain and the professional weight of each sub-specialty under each domain, determine the fusion weight of each sub-specialty under each domain; use the fusion weight of each sub-specialty under each domain to perform weighted fusion of the parameter matrices of each trained professional adaptation layer in each domain to obtain the domain parameter matrix corresponding to each domain; and perform weighted fusion of the domain parameter matrices corresponding to each domain according to the domain weight of each domain to obtain the full domain parameter matrix, so as to generate the full domain adapter.

[0166] In an exemplary embodiment, the cross-domain training dataset includes task samples of multiple cross-domain tasks; each task sample of a cross-domain task includes the cross-domain task and its standard response text; the professional training dataset includes task samples of multiple sub-professional tasks; each task sample of a sub-professional task includes the sub-professional task and its standard response text; the model fine-tuning module 308 is specifically used to input each cross-domain task into the initial ship large model to obtain the predicted response text of each cross-domain task, and to input each sub-professional task into the initial ship large model to obtain the predicted response text of each sub-professional task; to determine cross-domain loss information based on the predicted response text and standard response text of each cross-domain task, and to determine sub-professional loss information based on the predicted response text and standard response text of each sub-professional task; to determine total loss information based on the cross-domain loss information and sub-professional loss information; and to fine-tune the initial ship large model based on the total loss information until a preset condition is met to obtain the final ship large model.

[0167] In an exemplary embodiment, the apparatus further includes: a prompt information generation module, used to acquire a natural language task input by the user; the natural language task is a subdivided professional task or a cross-domain task; retrieve terms associated with the natural language task and term attributes of the terms from a pre-built ship engineering terminology knowledge base, and generate contextual prompt information adapted to the natural language task based on the terms and term attributes; combine the contextual prompt information with the natural language task and input it into the final ship large model to obtain the answer text for the natural language task.

[0168] In an exemplary embodiment, the device further includes: a continuous training module, used to monitor the update status of the ship engineering rules knowledge base associated with the final ship large model; when the update status meets the preset model training trigger conditions, to acquire each update rule and the knowledge data associated with each update rule, and to construct an incremental training dataset based on each update rule and the knowledge data associated with each update rule; and to train the final ship large model using the incremental training dataset until the preset conditions are met.

[0169] In one exemplary embodiment, this application also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of any of the cross-domain ship large model training methods described above.

[0170] In one exemplary embodiment, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the cross-domain ship large model training methods described above.

[0171] In one exemplary embodiment, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the cross-domain ship large model training methods described in the above embodiments.

[0172] Indicatively, such as Figure 4 As shown, Figure 4 This is a schematic diagram of the internal structure of a computer device 400 provided in an embodiment of this application. The computer device 400 can be provided as a server. (Refer to...) Figure 4 The computer device 400 includes a processing component 402, which further includes one or more processors, and memory resources represented by memory 401 for storing instructions, such as application programs, that can be executed by the processing component 402. The application programs stored in memory 401 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 402 is configured to execute instructions to perform the cross-domain large ship model training method of any of the above embodiments.

[0173] The computer device 400 may also include a power supply component 403 configured to perform power management of the computer device 400, a wired or wireless network interface 404 configured to connect the computer device 400 to a network, and an input / output (I / O) interface 405. The computer device 400 may operate on an operating system stored in memory 401, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.

[0174] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0175] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0176] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0177] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A cross-domain training method for large ship models, characterized in that, The method includes: Obtain domain training datasets for multiple fields involved in the entire ship design process; For any of the aforementioned domains, using the domain training dataset, each sub-specialty adaptation layer of the domain is trained in parallel to obtain each trained specialty adaptation layer of the domain. Specifically, this includes: initializing the first parameter matrix of each sub-specialty adaptation layer of the domain to obtain the initialized first parameter matrix of each sub-specialty adaptation layer; and orthogonalizing the second parameter matrix of each sub-specialty adaptation layer of the domain to obtain the orthogonalized second parameter matrix of each sub-specialty adaptation layer. Each sub-specialty adaptation layer includes a trainable first parameter matrix and a fixed-parameter second parameter matrix. The first parameter matrix is ​​used to learn the professional task features of the domain; the second parameter matrix is ​​used to project the professional task features of the domain learned by the first parameter matrix onto the parameter space of the basic large language model; different orthogonalized second parameter matrices are pairwise orthogonal; using the domain training dataset of the domain, each of the sub-professional adaptation layers under the domain is trained in parallel. During the parallel training process, the orthogonalized second parameter matrix of each sub-professional adaptation layer is fixed, and only the initialized first parameter matrix of each sub-professional adaptation layer is updated until the convergence condition is met, thus obtaining each trained professional adaptation layer of the domain. Based on each trained professional adaptation layer in each of the aforementioned domains, a full-domain adapter is generated, and the full-domain adapter is loaded into the basic large language model to form an initial large ship model; Obtain a cross-domain training dataset covering each of the aforementioned domains, as well as a professional training dataset for stabilizing the performance of each sub-specialty task. Fine-tune the initial large-scale ship model using the cross-domain training dataset and the professional training dataset until a preset condition is met, to obtain the final large-scale ship model. The final large-scale ship model is used to respond to cross-domain tasks in the entire ship design process and output response text for the cross-domain tasks.

2. The method according to claim 1, characterized in that, The generation of a full-domain adapter based on each trained professional adaptation layer in each of the aforementioned domains includes: Determine the domain weight of each domain and the professional weight of each sub-specialty under each domain, and determine the fusion weight of each sub-specialty under each domain based on the domain weight of each domain and the professional weight of each sub-specialty under each domain; By using the fusion weights of each sub-specialty under each domain, the parameter matrices of each trained professional adaptation layer in each domain are weighted and fused to obtain the domain parameter matrix corresponding to each domain. According to the domain weights of each domain, the domain parameter matrices corresponding to each domain are weighted and fused to obtain the full domain parameter matrix, so as to generate the full domain adapter.

3. The method according to claim 1, characterized in that, The cross-domain training dataset includes task samples for multiple cross-domain tasks; each task sample for a cross-domain task includes the cross-domain task and its standard response text; the professional training dataset includes task samples for multiple sub-professional tasks; each task sample for a sub-professional task includes the sub-professional task and its standard response text. The process of fine-tuning the initial large-scale ship model using the cross-domain training dataset and the specialized training dataset until a preset condition is met, to obtain the final large-scale ship model, includes: Each of the cross-domain tasks is input into the initial large ship model to obtain the predicted response text of each of the cross-domain tasks; and each of the sub-specialty tasks is input into the initial large ship model to obtain the predicted response text of each of the sub-specialty tasks. Based on the predicted response text and standard response text of each cross-domain task, cross-domain loss information is determined, and based on the predicted response text and standard response text of each sub-specialty task, sub-specialty loss information is determined. Based on the cross-domain loss information and the subdivided professional loss information, the total loss information is determined; Based on the total loss information, the initial large ship model is fine-tuned until the preset conditions are met, thus obtaining the final large ship model.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Natural language tasks that acquire user input; these natural language tasks may be specialized or cross-domain. Retrieve terms associated with the natural language task and term attributes of the terms from a pre-built knowledge base of ship engineering terminology, and generate contextual prompts adapted to the natural language task based on the terms and term attributes. The contextual prompts are combined with the natural language task and then input into the final large ship model to obtain the response text for the natural language task.

5. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Monitor the update status of the ship engineering rules knowledge base associated with the final ship model; When the update state meets the preset model training triggering conditions, each update rule and the knowledge data associated with each update rule are acquired, and an incremental training dataset is constructed based on each update rule and the knowledge data associated with each update rule. The final large ship model is trained using the incremental training dataset until the preset conditions are met.

6. A cross-disciplinary large-scale ship model training device, characterized in that, The device includes: The data acquisition module is used to acquire domain training datasets from multiple fields involved in the entire ship design process; A parallel training module is used to train, for any given domain, each sub-specialty adaptation layer of that domain in parallel using the domain's training dataset, to obtain each trained specialty adaptation layer of that domain. Specifically, this includes: initializing the first parameter matrix of each sub-specialty adaptation layer of that domain to obtain the initialized first parameter matrix of each sub-specialty adaptation layer; and orthogonalizing the second parameter matrix of each sub-specialty adaptation layer of that domain to obtain the orthogonalized second parameter matrix of each sub-specialty adaptation layer. Each sub-specialty adaptation layer includes a trainable first parameter matrix and a fixed second parameter matrix. The matrix; the first parameter matrix is ​​used to learn the professional task features of the domain; the second parameter matrix is ​​used to project the professional task features of the domain learned by the first parameter matrix onto the parameter space of the basic large language model; different orthogonalized second parameter matrices are pairwise orthogonal; using the domain training dataset of the domain, each of the sub-professional adaptation layers under the domain is trained in parallel. During the parallel training process, the orthogonalized second parameter matrix of each sub-professional adaptation layer is fixed, and only the initialized first parameter matrix of each sub-professional adaptation layer is updated until the convergence condition is met, thus obtaining each trained professional adaptation layer of the domain; The adapter loading module is used to generate a full-domain adapter based on each trained professional adaptation layer of each domain, and load the full-domain adapter into the basic large language model to form an initial ship large model; The model fine-tuning module is used to acquire a cross-domain training dataset covering each of the aforementioned domains and a professional training dataset for stabilizing the performance of each sub-specialty task. The initial large ship model is then fine-tuned using the cross-domain training dataset and the professional training dataset until preset conditions are met, resulting in the final large ship model. The final large ship model is used to respond to cross-domain tasks in the entire ship design process and output response text for the cross-domain tasks.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images