Ship drawing auditing method based on multi-specialty expert large model, computer device, storage medium and computer program product

Abstract

The application provides a ship drawing auditing method based on a multi-specialist large model, a computer device, a storage medium and a computer program product. The method comprises the following steps: obtaining a ship drawing to be audited, and extracting multi-dimensional fusion features of the ship drawing; identifying a violation type of the ship drawing based on the multi-dimensional fusion features, and determining a root cause of the violation type; selecting at least one target specialist network from a pre-trained multi-specialist large model based on the multi-dimensional fusion features, inputting the multi-dimensional fusion features, the violation type and the root cause into each target specialist network, and obtaining an auditing result output by each target specialist network; and generating an auditing report of the ship drawing based on the auditing result output by each target specialist network. The method can accurately and reliably automatically audit complex ship drawings.

Description

Technical Field

[0001] This application relates to the field of large language model technology, and in particular to a method for reviewing ship drawings based on a large model of multiple professional experts, computer equipment, storage medium and computer program product. Background Technology

[0002] As the shipbuilding industry develops towards intelligence and greening, ship design is becoming increasingly complex, placing unprecedented demands on the accuracy and efficiency of design drawing review.

[0003] Currently, general-purpose large models are often used to review ship drawings. However, ship drawing review is essentially a highly specialized and knowledge-intensive decision-making process. A specific design violation often involves cross-judgment and in-depth attribution of multiple professional knowledge. The knowledge structure and reasoning process of existing general-purpose large models are relatively rigid. When the model identifies a potential violation type, its review conclusion often remains at the level of describing the surface violation phenomenon, lacking in-depth analysis of the violation phenomenon. The accuracy and professional credibility of the review results it provides are difficult to meet the requirements of actual engineering drawing review.

[0004] Therefore, existing technologies cannot accurately and reliably automate the review of complex ship drawings. Summary of the Invention

[0005] Based on this, the purpose of this application is to at least solve one of the above-mentioned technical defects, in particular the technical defect that the prior art cannot perform accurate and reliable automated review of complex ship drawings. This application provides a ship drawing review method, computer equipment, computer-readable storage medium and computer program product based on a large model of multi-disciplinary experts that can perform accurate and reliable automated review of complex ship drawings.

[0006] Firstly, this application provides a method for reviewing ship drawings based on a large model by multiple experts, the method including:

[0007] Obtain the ship drawings to be reviewed and extract the multi-dimensional fusion features of the ship drawings;

[0008] Based on multi-dimensional fusion features, the types of violations in ship drawings are identified, and the root causes of the violations are determined.

[0009] Based on multi-dimensional fusion features, at least one target expert network is selected from the pre-trained multi-professional expert large model, and the multi-dimensional fusion features, violation type and root cause are input into each target expert network to obtain the review results output by each target expert network.

[0010] Based on the review results output by the expert networks of each target, a review report of the ship drawings is generated.

[0011] In one exemplary embodiment, multi-dimensional fusion features of ship drawings are extracted, including:

[0012] Extracting multimodal features from ship drawings;

[0013] Based on multimodal features, professional knowledge association features of ship drawings are obtained from a pre-constructed knowledge graph;

[0014] By fusing multimodal features and knowledge-related features, multidimensional fused features are obtained.

[0015] In one exemplary embodiment, extracting multimodal features from ship drawings includes:

[0016] Acquire the 2D drawing data, text data, and 3D ship model data corresponding to the ship drawings;

[0017] Visual features are extracted from 2D drawing data, semantic features are extracted from text data, and spatial features are extracted from 3D ship model data.

[0018] By integrating visual features, semantic features, and spatial features, multimodal features are obtained.

[0019] In one exemplary embodiment, based on multimodal features, professional knowledge association features of ship drawings are obtained from a pre-built knowledge graph, including:

[0020] Based on multimodal features, identify at least one ship component appearing in the ship drawings;

[0021] The embedding features of each ship component are extracted from the pre-built knowledge graph, and the embedding features of each ship component are aggregated to obtain the professional knowledge association features of the ship drawings.

[0022] In one exemplary embodiment, identifying the type of violation in ship drawings based on multi-dimensional fusion features includes:

[0023] Obtain the association matrix between ship components and violation types from the knowledge graph;

[0024] The correlation matrix and multi-dimensional fused features are input into a pre-trained violation type prediction model to obtain the violation type probability distribution results;

[0025] Based on the probability distribution of violation types, the types of violations in the ship drawings are determined.

[0026] In one exemplary embodiment, at least one target expert network is selected from a pre-trained multi-specialty expert large model based on multi-dimensional fusion features, including:

[0027] Based on multi-dimensional fusion features, a multi-dimensional fusion feature query vector is generated; based on the knowledge features and task features of each expert network, a key vector of each expert network is generated; and based on the knowledge graph, a knowledge guidance matrix is ​​constructed. The knowledge guidance matrix represents the adaptation relationship between the features of different professional drawings and different expert networks.

[0028] The multi-dimensional fusion feature query vector, the key vectors of each expert network, and the knowledge guidance matrix are input into the routing probability model to determine the routing probability of each expert network.

[0029] The target expert network is determined from among the expert networks based on the routing probabilities of each expert network.

[0030] In one exemplary embodiment, determining the root cause of the violation type includes:

[0031] Based on the violation causal model, the design influencing factors that lead to the violation type are identified, and the causal effect value of each design influencing factor is determined.

[0032] Based on the causal effect values ​​of each design influencing factor, determine the root cause of the violation type.

[0033] Secondly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0034] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0035] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0036] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0037] The ship drawing review method, computer equipment, storage medium, and computer program product based on a multi-disciplinary expert large model provided in this application acquire the ship drawings to be reviewed and extract multi-dimensional fusion features from the ship drawings; based on the multi-dimensional fusion features, identify the types of violations in the ship drawings and determine the root causes of the violations; based on the multi-dimensional fusion features, select at least one target expert network from the pre-trained multi-disciplinary expert large model, and input the multi-dimensional fusion features, violation types, and root causes into each target expert network to obtain the review results output by each target expert network; based on the review results output by each target expert network, generate a ship drawing review report; thus, a multi-dimensional fusion feature-based review method is constructed. Based on this model, a complete review pipeline is integrated, combining violation type and root cause identification with intelligent invocation of a large multi-disciplinary expert model. This solves the problem of inaccurate and reliable automated review of complex ship drawings. Traditional automated drawing review methods often only perform simple rule checks, failing to understand the deep semantics and professional context of the drawings, let alone conduct cross-disciplinary collaborative judgments. This method integrates multi-source information to form deep features and uses a large multi-disciplinary expert model to simulate the professional perspectives of experts from different fields for reasoning. This enables the review process to not only discover surface violations but also diagnose deep-seated causes, and the review conclusions have the reliability of multi-disciplinary collaboration. Thus, it achieves accurate and reliable automated drawing review that approaches the level of human experts. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is an application environment diagram of a ship drawing review method based on a large multidisciplinary expert model, as shown in one embodiment.

[0040] Figure 2 This is a schematic diagram of a large-scale intelligent drawing interpretation model training method that integrates multiple ship-related professional knowledge in one embodiment;

[0041] Figure 3 This is a flowchart illustrating a ship drawing review method based on a large multidisciplinary expert model, as described in another embodiment.

[0042] Figure 4 This is a structural block diagram of a ship drawing review device based on a large model of multiple professional experts, as shown in one embodiment.

[0043] Figure 5This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] As the shipbuilding industry iterates towards intelligence, greening, and large-scale development, the integration and complexity of ship design have increased significantly. Multidisciplinary collaborative design and drawing review have become core links in ensuring ship design quality, shortening construction cycles, and reducing costs. Currently, ship drawing review still relies primarily on PDF drawings, with the mainstream model combining manual review with traditional computer-aided design tools. While this meets basic drawing review needs, it exposes many insurmountable limitations in areas such as multidisciplinary knowledge integration, massive data processing, and efficient and accurate drawing review.

[0046] Existing ship plan review large model training technology and application scenarios have four major pain points:

[0047] First, multimodal data processing and knowledge fusion are challenging. PDF drawings contain multimodal information such as text, graphics, and formulas, which are difficult to extract accurately and time-consuming. Ship data from multiple disciplines exhibits heterogeneous characteristics, making it difficult to achieve structured fusion of textual knowledge from ship specifications and IMO regulations, spatial knowledge from 3D models, and experiential knowledge from historical drawing review cases. This results in large models lacking a unified knowledge foundation.

[0048] Secondly, the model architecture lacks professional adaptability. General-purpose large models are difficult to adapt to the differentiated needs of multiple professional shipbuilding drawing review. The existing MoE architecture does not divide dedicated expert modules according to the six major shipbuilding professions, lacks professional-specific optimization, and cannot accurately match the drawing review rules and knowledge points of each profession. When a single model processes multi-professional data, it suffers from high computational complexity and poor real-time performance, making it difficult to balance drawing review efficiency and accuracy.

[0049] Third, the training system is incomplete and has weak generalization ability. It lacks a dual optimization strategy of "multi-task basic fine-tuning + professional precise fine-tuning", resulting in low accuracy in identifying small sample violation cases; it has not introduced a reinforcement feedback mechanism from professional drawing reviewers, resulting in discrepancies between the model's drawing review results and human professional judgment, and the accuracy decays significantly when adapting to different ship types, requiring large-scale retraining to adapt to new ship types such as new energy ships.

[0050] Fourth, the interpretability and closed-loop optimization capabilities of the review results are lacking. Traditional deep learning models are "black box" reasoning, unable to connect with specific professional standard clauses and knowledge nodes, making it difficult to trace the review results; lacking a closed-loop iteration mechanism of data-knowledge-model, the model cannot continuously absorb new standards and case knowledge, making it difficult to adapt to the ever-updating regulations and design requirements of the shipbuilding industry.

[0051] The development of MoE architecture, multimodal processing, and reinforcement learning technologies has provided technical solutions to the above problems. MoE architecture allocates tasks to dedicated expert modules through dynamic routing mechanisms, which can achieve a balance between computational efficiency and inference accuracy. However, existing technologies have not been deeply integrated with the multi-disciplinary drawing review scenarios in ships, and have not built expert division of labor and routing logic based on professional knowledge. Although multimodal technology can handle heterogeneous data, it has not been effectively implemented in scenarios where ship specialties cross-reference each other (such as the correlation verification of structural components and pipeline layout).

[0052] Therefore, there is an urgent need to construct a large-scale model training method that integrates multiple ship-related professional knowledge. Through the collaborative design of "professional knowledge structuring - dedicated MoE architecture - dual fine-tuning system - enhanced feedback - closed-loop optimization", a deep integration of multiple professional knowledge and large-scale model training can be achieved, thereby improving the accuracy, real-time performance, interpretability and cross-ship type generalization ability of the model drawing review.

[0053] To address the problems existing in the prior art, this application provides a method for reviewing ship drawings based on a multi-disciplinary expert large model. The multi-disciplinary expert large model used in this method is trained by integrating multiple professional knowledge in the shipbuilding field, and can provide accurate and reliable review reports for review tasks in multi-disciplinary shipbuilding drawing review scenarios. Specific implementation methods will be described in the following embodiments.

[0054] In one exemplary embodiment, such as Figure 1 As shown, a method for reviewing ship drawings based on a large multi-disciplinary expert model is provided. Taking the application of this method to a server as an example, the method includes the following steps S102 to S108. Wherein:

[0055] Step S102: Obtain the ship drawings to be reviewed and extract the multi-dimensional fusion features of the ship drawings.

[0056] Ship drawings refer to engineering drawing files that contain ship design information. These drawings may involve professional knowledge from various disciplines in the shipbuilding field (general, structural, piping, marine, electrical, outfitting).

[0057] Among them, multi-dimensional fusion features refer to a comprehensive digital representation that extracts and integrates multiple types of information from ship drawings, which can be represented as: .

[0058] Optionally, the server obtains the ship's blueprint PDF file, and then performs deep analysis on the blueprint to generate multi-dimensional fusion features. .

[0059] Step S104: Based on multi-dimensional fusion features, identify the types of violations in the ship drawings and determine the root causes of the violations.

[0060] Among them, the type of violation refers to the specific problem category in the drawings that does not comply with relevant specifications, standards or design guidelines, such as structural strength violations, pipeline layout conflicts, etc.

[0061] The root cause refers to the most fundamental influencing factor that leads to the violation, which may include component size, layout spacing, material specifications, equipment model, etc.

[0062] Optionally, the server determines the type of violation and traces its root cause based on multi-dimensional fusion features.

[0063] Step S106: Based on the multi-dimensional fusion features, select at least one target expert network from the pre-trained multi-professional expert large model, and input the multi-dimensional fusion features, violation type and root cause into each target expert network to obtain the review results output by each target expert network.

[0064] Among them, the pre-trained multi-professional expert big model is a machine learning model that integrates multiple professional sub-networks (expert networks). Each professional sub-network has been trained specifically for a certain profession in the shipbuilding field (such as general, structure, piping, marine engineering, electrical, outfitting) and has professional review knowledge of that profession. Each professional sub-network focuses on the core drawing review tasks of the corresponding profession.

[0065] In this application, the multi-disciplinary expert model, based on the differences and knowledge characteristics of the drawing review tasks of the six major shipbuilding specialties, sets up six expert networks. Each expert network focuses on the core drawing review tasks of its corresponding specialty, adopting an architecture of "one specialty, one core expert." A shared parameter layer is also included to carry general shipbuilding drawing review knowledge (such as general regulatory clauses and basic PDF processing capabilities). The parameters of the multi-disciplinary expert model are represented as follows: ,in, The basic parameters shared by all expert modules include multimodal feature extraction of general PDF drawings, general ship specifications, and basic logic for cross-disciplinary collaboration, ensuring that the model has basic cross-disciplinary drawing review capabilities; For the first Private parameters of a professional expert ( These correspond to the general, structural, piping, marine, electrical, and outfitting specialties, respectively. They include knowledge such as PDF drawing feature extraction rules, professional specification matching logic, and exclusive violation type identification algorithms for each specialty, such as strength verification parameters for structural specialties and wiring specification matching parameters for electrical specialties.

[0066] Among them, the target expert network is one or more professional sub-networks that are most relevant to the issue to be reviewed, dynamically selected from a large model of multi-professional experts based on the multi-dimensional fusion characteristics of ship drawings.

[0067] The review result is the professional opinion output by the target expert network after reasoning about the input question. It may include information such as violation judgment, regulatory basis, modification suggestions, and root cause analysis. The review result of the m-th expert can be represented as... .

[0068] Optionally, the server intelligently routes and selects the most relevant target expert network from the multi-professional expert big model based on the multi-dimensional fusion characteristics. Then, the multi-dimensional fusion characteristics, violation type and root cause are input into each selected expert network to obtain their respective independent review results.

[0069] Step S108: Based on the review results output by each target expert network, generate a review report of the ship drawings.

[0070] The audit report is a structured document that summarizes the results of the online audits conducted by various experts.

[0071] Optionally, the server can aggregate the audit results output by all target expert networks, verify them against CCS specification rules, and generate a final, detailed audit report. The aggregation process can be represented as follows: .

[0072] The aforementioned ship drawing review method based on a multi-disciplinary expert large model involves: acquiring the ship drawings to be reviewed and extracting multi-dimensional fusion features; identifying the types of violations based on these features and determining the root causes of those violations; selecting at least one target expert network from a pre-trained multi-disciplinary expert large model based on these features, and inputting the multi-dimensional fusion features, violation types, and root causes into each target expert network to obtain the review results output by each network; and generating a ship drawing review report based on the review results output by each target expert network. Thus, a system based on multi-dimensional fusion features and integrating violation categories is constructed. This complete review pipeline, which integrates type and root cause identification with intelligent multi-disciplinary expert model invocation, solves the problem of inaccurate and reliable automated review of complex ship drawings. Traditional automated drawing review methods often only perform simple rule checks, failing to understand the deep semantics and professional context of the drawings, let alone conduct cross-disciplinary collaborative judgments. This method integrates multi-source information to form deep features and uses a multi-disciplinary expert model to simulate the professional perspectives of experts from different fields for reasoning. This enables the review process to not only discover surface violations but also diagnose deep-seated causes, and the review conclusions possess the reliability of multi-disciplinary collaboration. Thus, it achieves accurate and reliable automated drawing review that approaches the level of human experts.

[0073] In an exemplary embodiment, the extraction of multi-dimensional fusion features of ship drawings includes: extracting multimodal features of ship drawings; obtaining professional knowledge association features of ship drawings from a pre-constructed knowledge graph based on the multimodal features; and fusing the multimodal features and the professional knowledge association features to obtain multi-dimensional fusion features.

[0074] Among them, multimodal features refer to the fusion of features from different modes of ship drawings.

[0075] The pre-built knowledge graph is a structured professional knowledge base consisting of ship engineering entities (such as components, systems, and code clauses) as nodes and the relationships between them (such as inclusion, compliance, and action) as edges.

[0076] Among them, professional knowledge association features refer to the structured embedding of the knowledge graph, which can be represented as: It is a structured feature obtained by querying a knowledge graph and associating and extracting the entities and their relationship network information related to the current ship drawing content. It places the drawing content in a vast professional knowledge system.

[0077] Optionally, the server first extracts the features of ship drawings in each modality through a parallel processing flow. Then, it uses the features in each modality as queries to perform retrieval and reasoning in a pre-built knowledge graph, associates these information to obtain professional knowledge-related features, and finally, through multimodal joint embedding, integrates the multimodal features and professional knowledge-related features into a unified, more information-rich, multi-dimensional fusion feature representation.

[0078] In this embodiment, by introducing a knowledge graph in the feature extraction stage to obtain professional knowledge-related features and fusing them with the original multimodal features, a leap from understanding the visual and textual information of drawings to understanding the context of engineering knowledge is achieved. This solves the problem that traditional methods only analyze the data of the drawings themselves, lacking the support of professional domain knowledge background, which leads to a superficial and error-prone interpretation of the content of the drawings. The fused multidimensional features not only know what the drawings depict, but also understand the meaning, constraints and relationships of the content depicted in the shipbuilding engineering system. This provides a solid and semantically deep information foundation for subsequent violation identification and expert review, which is the key to improving the accuracy and reliability of the review.

[0079] In an exemplary embodiment, extracting multimodal features from ship drawings includes: acquiring two-dimensional drawing data, text data, and three-dimensional ship model data corresponding to the ship drawings; extracting visual features based on the two-dimensional drawing data, extracting semantic features based on the text data, and extracting spatial features based on the three-dimensional ship model data; and fusing the visual features, semantic features, and spatial features to obtain multimodal features.

[0080] Among them, two-dimensional drawing data can refer to the data of CAD drawings (PDF format) of ships.

[0081] Text data refers to all textual information extracted from drawings, including dimensioning, technical specifications, bill of materials, legends, etc.

[0082] Among them, three-dimensional ship model data refers to the data of the three-dimensional model file associated with ship drawings, which includes the three-dimensional shape, position and assembly relationship of ship components.

[0083] Among them, visual features are deep features extracted by CNN (Convolutional Neural Network), such as graphic contours, component layouts, and labeled text positions, which can be represented as: .

[0084] Among them, semantic features are obtained by word embedding and sequence modeling of text data using natural language processing techniques. They represent features such as constraint logic or technical terms in the text and can be represented as follows: .

[0085] Among them, spatial features are obtained by extracting component spatial coordinates, topological relationships, and device spacing through the PointNet++ model, and can be represented as follows: .

[0086] Optionally, the server receives the original drawing file package and automatically parses out the 2D drawing data, text data, and associated 3D ship model data. For the 2D data, it may first be vectorized or rasterized, and then input into a visual feature extraction network to obtain visual feature embeddings. For text data, after cleaning and word segmentation, the data is input into a text feature extraction model to obtain semantic feature embeddings. For 3D data, after format standardization and preprocessing, the data is input into a spatial feature extraction network to obtain spatial feature embeddings. The three feature extraction processes are executed in parallel, each outputting a high-dimensional feature vector. Finally, the features from these three sources are fused through weighted processing to obtain multimodal features. , The modality weight matrix is ​​adaptively adjusted through training to highlight the contribution of each modality to the image review task.

[0087] In this embodiment, by extracting and fusing visual features, semantic features, and spatial features in parallel, a multimodal feature set that comprehensively describes the information in ship drawings is constructed. This solves the problem of incomplete information and ambiguity caused by a single data source. For example, visual features alone may not be able to understand the specific meaning of a symbol, but the text annotation next to it can make it clear. Two-dimensional projection alone may not be able to determine whether a pipe interferes with a structural beam in space, but three-dimensional spatial features can make a precise judgment. This multimodal fusion ensures that subsequent steps can be analyzed based on the most complete and accurate drawing information.

[0088] In an exemplary embodiment, based on multimodal features, the professional knowledge association features of ship drawings are obtained from a pre-built knowledge graph, including: based on multimodal features, identifying at least one ship component appearing in the ship drawings; extracting the embedding features of each ship component from the pre-built knowledge graph, and aggregating the embedding features of each ship component to obtain the professional knowledge association features of the ship drawings.

[0089] Ship components refer to the physical or functional units that make up a ship, such as ribs, bulkheads, main engines, fire pumps, cable trays, etc.

[0090] Embedded features refer to low-dimensional dense vectors that represent each entity (node) in a knowledge graph, learned through graph embedding techniques (such as TransE). These vectors contain the structural and attribute information of the entity in the graph.

[0091] The professional knowledge association characteristics of the ship drawings in this application can be expressed as follows: This corresponds to the structured embedding of knowledge graphs. It is obtained by using the TransE algorithm to transform entities and relationships into vector representations and then incorporating professional knowledge-related information.

[0092] Optionally, the server utilizes features from multimodal features The information in the diagram is used to identify a list of key ship components involved in the current ship drawing. Then, using these component names as query keys, the corresponding entity nodes are searched in a pre-built knowledge graph, and the pre-calculated embedding features of these nodes are directly read. Since a ship drawing usually involves multiple components, the embedding features of these multiple components need to be aggregated to finally generate a professional knowledge association feature vector representing the knowledge background of the entire drawing. .

[0093] In this embodiment, by extracting and aggregating the embedding features of specific ship components from the knowledge graph, the abstract content of the drawings is anchored and associated with specific, structured engineering knowledge. This solves the problem that it is difficult to systematically introduce massive amounts of external professional standards, design conventions, and association rules relying solely on the data of the drawings themselves. The embedding features of a component are essentially a condensed representation of all its associated knowledge (such as functions, constraints, and related systems). The features formed after aggregation enable the model to know the position of the component in the current drawing within the networked knowledge system and the rules it should follow, thereby greatly enhancing the breadth of knowledge and the depth of reasoning of the review system.

[0094] This application constructs a multimodal joint embedding model by integrating implicit knowledge from text, images, and 3D models with explicit knowledge from knowledge graphs. This transforms data from different modalities into feature vectors of a unified dimension, facilitating processing by large models. The multimodal joint embedding model can be represented as follows: , The modality weight matrix is ​​adaptively adjusted through training to highlight the contribution of each modality to the image review task. The knowledge embedding fusion coefficient is set to a value of 0.4 to 0.8, with 0.6 being the preferred value, to balance the fusion ratio of multimodal data features and structured knowledge. , , , These correspond to visual feature embedding, semantic feature embedding, spatial feature embedding, and structured embedding of knowledge graphs, respectively.

[0095] In an exemplary embodiment, identifying the violation type of ship drawings based on multi-dimensional fusion features includes: obtaining the association matrix between ship components and violation types from a knowledge graph; inputting the association matrix and multi-dimensional fusion features into a pre-trained violation type prediction model to obtain the violation type probability distribution result; and determining the violation type of ship drawings based on the violation type probability distribution result.

[0096] The association matrix between ship components and violation types is a matrix derived from a knowledge graph, reflecting which types of violations different ship components may cause, represented as follows: This can clearly identify the violation scenarios and judgment standards corresponding to ship components of different specialties, and guide the model to focus on the core points of drawing review.

[0097] The pre-trained violation type prediction model is an inference model that takes multi-dimensional fused features and an association matrix as input and outputs the probability of various violations. This model can be represented as follows: .in, It is a multilayer perceptron used to learn multi-dimensional fused features. The mapping relationship between the drawing review results and the actual drawing results enables data-driven reasoning. The association matrix derived from the knowledge graph represents the weight of a specific violation type corresponding to a certain professional component, which explicitly constrains the reasoning direction and ensures that the reasoning result conforms to professional knowledge. This enables element-wise multiplication, merging data-driven features with professional knowledge constraints. The probability distribution of component violation types is output, along with the probability of occurrence of each violation type and the corresponding standard basis.

[0098] The probability distribution of violation types is a probability vector output by the model. Each dimension corresponds to a violation type, and its value represents the confidence level of the model in judging that the violation has occurred.

[0099] Optionally, the server first calculates a "professional component-violation type" association matrix from the knowledge graph. Then, it inputs the multi-dimensional fusion features and the "professional component-violation type" association matrix into a pre-trained violation type prediction model. The model integrates specific features and prior knowledge to output the violation type probability distribution. Based on the probability threshold or by taking the Top-K probability value, the final violation type is determined.

[0100] In this embodiment, by inputting multi-dimensional fusion features reflecting the specific content of drawings and an association matrix reflecting general rules of the domain into the violation type prediction model, the specific analysis and prior guidance for violation identification are combined. Based on the entity association relationship of the knowledge graph, explicit constraints are imposed on the deep learning reasoning process, avoiding "black box" reasoning and professional logic deviations, thereby improving the accuracy and interpretability of identification.

[0101] In an exemplary embodiment, at least one target expert network is selected from a pre-trained multi-disciplinary expert large model based on multi-dimensional fusion features. This includes: generating a multi-dimensional fusion feature query vector based on the multi-dimensional fusion features; generating a key vector for each expert network based on the knowledge features and task features of each expert network; and constructing a knowledge guidance matrix based on a knowledge graph. The knowledge guidance matrix represents the adaptation relationship between different professional drawing features and different expert networks. The multi-dimensional fusion feature query vector, the key vectors of each expert network, and the knowledge guidance matrix are input into a routing probability model to determine the routing probability of each expert network. Based on the routing probability of each expert network, a target expert network is determined among the expert networks.

[0102] The query vector is a vector obtained by further transforming the multi-dimensional fusion features, and is used to query the most suitable expert in the expert network.

[0103] Among them, the key vector is the business card of each expert network, which is encoded by the knowledge domain (knowledge feature) and task type (task feature) that it is good at handling.

[0104] The knowledge guidance matrix is ​​a matrix learned or defined from a knowledge graph. Its element values ​​reflect the weight relationship of which expert network should be given priority to process the content of the drawing when it has certain professional characteristics.

[0105] Among them, the routing probability model is a model that calculates the probability of each expert network receiving a task assignment based on the query vector, key vector, and knowledge guidance matrix (such as based on an attention mechanism).

[0106] Optionally, the server first maps the multi-dimensional fused features into query vectors. Meanwhile, each expert network has its predefined key vector and derives a knowledge guidance matrix from the knowledge graph. Then, these three types of information are input into the routing probability model, which may calculate the similarity between the query vector and each key vector and use the knowledge guidance matrix for weighting and adjustment. Finally, it outputs a set of probability values ​​(routing probabilities), representing the probability that the current ship drawing should be processed by each expert network. Based on these probabilities, one or more networks with the highest probabilities are selected as the target expert network.

[0107] This embodiment corresponds to a knowledge-guided dynamic routing mechanism that combines multi-dimensional fusion features. Based on the domain knowledge graph, a gating network was designed to achieve accurate matching between input PDF drawings and professional expert networks, ensuring that the appropriate expert network handles the corresponding professional drawing review task. The routing probability model is represented as follows: .in, The multi-dimensional fusion feature query vector is generated from the multi-dimensional fusion features of the input PDF drawing. For the first Each professional expert's key vector carries the knowledge characteristics and task features of the corresponding profession; The vector dimension is used for normalization to avoid the curse of dimensionality. As a knowledge-guided matrix, it is constructed based on the matching degree of "professional-expert-PDF drawing features" of knowledge graph to quantify the adaptation relationship between different professional drawing features and expert modules; Assign the input PDF drawing data to the first Based on the probability of each expert, the expert module with the top-2 to top-3 probabilities is selected to handle the current drawing review task, thereby realizing multi-expert collaborative calculation.

[0108] In this embodiment, an intelligent routing mechanism based on query vector-key vector matching and knowledge-guided matrix control is designed to dynamically and accurately assign experts to the most suitable professional expert network according to the specific content of the drawings. This solves the problems of low review efficiency or mismatch between professional expertise caused by using all experts or randomly selecting experts. This mechanism ensures that structural violations are mainly handled by structural experts, and marine engineering issues are handled by marine engineering experts. At the same time, the knowledge-guided matrix can handle cross-disciplinary issues (such as the installation of marine equipment involving structural vibration), intelligently allocating or coordinating multiple experts. This precise routing is the core scheduling strategy for leveraging the collaborative advantages of a multi-expert large model and ensuring the professionalism and accuracy of review results.

[0109] In one exemplary embodiment, determining the root cause of a violation type includes: identifying each design influencing factor that causes the violation type based on a violation causal model, and determining the causal effect value of each design influencing factor; and determining the root cause of the violation type based on the causal effect value of each design influencing factor.

[0110] Among them, the violation causal model is a model designed to reveal how design decisions (causes) lead to violations (effects).

[0111] Among them, design influencing factors refer to factors such as component dimensions, spacing, material specifications, and equipment models that may lead to violations, and can be expressed as follows: .

[0112] Among them, the causal effect value is an indicator that quantifies the contribution of each design influencing factor to the occurrence of violations, and can be expressed as: It is used to quantify the degree of influence of a certain design factor on the occurrence of violations. The larger the absolute value, the higher the probability that the factor is the root cause of the violation.

[0113] Optionally, after identifying the violation type, the server inputs the multi-dimensional fusion features and violation type into the violation causal model to analyze which specific design influencing factors have deviated and how, and calculates the causal effect value of each factor. Based on the size of the causal effect value, the server sorts the factors and determines the one or several factors with the largest effect value as the root cause.

[0114] In practical applications, considering the multi-disciplinary nature of ship design review, this paper models the causal relationships between violations of components from multiple disciplines, constructs a violation causal model, accurately identifies the root causes of violations, and improves the interpretability of the design review results. The model can be represented as follows: .in, This is the causal effect value. As design influencing factors, To avoid obfuscating variables such as ship type, deadweight tonnage, navigation area, and specification version, and to prevent non-causal factors from interfering with root cause determination, To intervene, the value of a certain influencing factor is fixed, and the change in the probability of violation is observed. The influence of confounding variables is removed, and the root cause of the violation is accurately located (e.g., the root cause of structural strength violation is excessive rib spacing rather than ambient temperature).

[0115] In this embodiment, by introducing a violation causal model for attribution analysis, a leap from discovering violations to diagnosing their root causes is achieved. This overcomes the limitations of traditional automated audits, which can only report errors but cannot pinpoint why they occurred or how to correct them. By calculating causal effect values ​​and locating the root cause, the generated audit report not only identifies the problem but also provides precise directions for improvement, enhancing the practicality and guidance value of the audit results. This makes the automated audit system a truly powerful tool for design optimization, thereby improving the reliability and depth of the audit.

[0116] In another embodiment, a method for training a large-scale model that integrates multiple ship-related professional knowledge is provided. This method achieves deep integration of multi-professional knowledge with large-scale model training through a collaborative design of "professional knowledge structuring - dedicated MoE architecture - dual fine-tuning system - reinforcement feedback - closed-loop optimization". The following explanation uses a multi-professional expert large-scale model of a container ship as an example; please refer to [reference needed]. Figure 2 The training method specifically includes the following steps:

[0117] Step 1: Construct a knowledge base for ship review that integrates multidisciplinary expertise. Using PDF drawings as the core, establish unified standards for multidisciplinary ship data: 3D models use STEP format, text data uses JSON format, and graphic data uses standardized pixel and resolution parameters. Integrate data from the entire ship design and construction lifecycle, including 2D drawings from six major disciplines, 3D ship models, classification society specifications, regulatory documents, historical review cases, expert experience summaries, and other multimodal data.

[0118] (1) Based on PDF drawings (including converted CAD vector graphics), STEP format 3D models, CCS specification texts, IMO SOLAS convention clauses, and historical drawing review cases (including more than 5,000 compliant drawings, more than 1,200 non-compliant drawings, and manual correction opinions) of six major specialties such as container ships (general structure, structure, piping, marine engineering, electrical, and outfitting), a unified data standard was formulated and implemented. This function achieves precise alignment between component numbers in PDF drawings and components in 3D models, as well as CCS specification clauses; it also addresses small sample data issues such as structural weld violations and electrical wiring conflicts through... The function performs case synthesis (generating over 200 synthetic violation cases) and image rotation and cropping enhancement to construct a fused dataset. , Represented as: In the formula, This is 2D CAD drawing data (including converted PDF vector graphics and bitmaps). This is for 3D ship model data (extracting the spatial location and dimensional features of the structure and equipment). This includes extracting textual data from classification society rules and IMO standards (structured extraction of clause numbers, constraints, violation judgment criteria, etc.). This is a multi-source data alignment function that uses semantic matching, coordinate calibration, and feature mapping to unify heterogeneous data formats and associate them with professional knowledge. For example, it can accurately match component numbers in PDF drawings with components in 3D models and specification clauses. This is low-quality / small sample drawing review data (including rare violation cases and blurry drawings). For data augmentation functions, methods such as drawing rotation, cropping, noise addition, violation case synthesis, and text semantic augmentation are used to expand the dataset and solve the problem of small sample training. This is a feature splicing operation that enables deep fusion of multi-dimensional features.

[0119] (2) A combination of manual annotation and machine extraction was adopted to model explicit knowledge in the ship design review domain using entity-relationship-attribute triples, constructing a domain knowledge graph covering six major specialties. The core entities of the graph include "specialty type, component name, standard clause, violation type, equipment model, and ship type parameters," while the relationships include "component-standard constraint, violation type-responsible specialty, and equipment-layout requirements." Attributes include component size limits, standard effective time, and violation severity. A dynamic update model for the knowledge graph was designed to achieve real-time knowledge iteration. The model is represented as follows: In the formula, The knowledge graph state at time t contains entities, relationships, and attribute sets of six major professions, forming a complete knowledge chain of "profession-component-standard-violation type"; This includes knowledge update actions such as adding new entities (e.g., battery compartment components for new energy ships), revising relationships (e.g., updating the correspondence between violation types and standard clauses), and supplementing attributes (e.g., adding new limit parameters after the standard revision). The knowledge consistency verification reward function is based on the logical constraints of ship drawing review specifications and professional knowledge. It verifies the rationality of the updated content (such as avoiding conflicts between component size limits and specifications). If it conforms to the logic, a positive reward is given; otherwise, the update is inhibited. To update the step size, a value of 0.02 to 0.06 is used. To balance the speed and stability of knowledge updates, a value of 0.04 is preferred.

[0120] (3) By integrating the implicit knowledge of text, images, and 3D models with the explicit knowledge of knowledge graphs, a multimodal joint embedding model is constructed. This model transforms data from different modalities into feature vectors of a unified dimension, facilitating processing by large models. The model is represented as follows: In the formula, Visual features are embedded in CAD drawings (PDF format), and features such as graphic contours, component layouts, and annotation text positions are extracted using CNN (convolutional neural network). To standardize the semantic feature embedding of text and image review records, a BERT pre-trained model is used to generate the embeddings, capturing the constraint logic and professional terminology in the text. To embed spatial features of the 3D model, features such as component spatial coordinates, topological relationships, and device spacing are extracted using the PointNet++ model. For structured embedding of knowledge graphs, the TransE algorithm is used to transform entities and relations into vector representations, incorporating professional knowledge-related information; The modality weight matrix is ​​adaptively adjusted through training to highlight the contribution of each modality to the image review task. The knowledge embedding fusion coefficient is set to a value of 0.4 to 0.8, with 0.6 being the preferred value, to balance the fusion ratio of multimodal data features and structured knowledge.

[0121] Step 2: Construct a multimodal reasoning model that integrates multiple professional knowledge. Integrate rule-based reasoning (based on professional knowledge) and data-driven reasoning (based on training data) to build an interpretable and highly accurate multi-professional drawing review reasoning system, which serves as the "drawing review decision core" of the MoE architecture, ensuring that the reasoning process conforms to the logic of ship professional drawing review.

[0122] (1) Based on the entity association relationship of the knowledge graph, explicit constraints are imposed on the deep learning reasoning process to avoid "black box" reasoning and deviations from professional logic. This is achieved through the "professional component-violation type" association matrix in the knowledge graph. This clarifies the violation scenarios and judgment criteria corresponding to different professional components, guiding the model to focus on core drawing review points. The inference model formula is expressed as follows: In the formula, It is a multilayer perceptron used to learn multimodal fusion features. The mapping relationship between the drawing review results and the actual drawing results enables data-driven reasoning. The association matrix derived from the knowledge graph represents the weight of a specific violation type corresponding to a certain professional component, which explicitly constrains the reasoning direction and ensures that the reasoning result conforms to professional knowledge. This enables element-wise multiplication, merging data-driven features with professional knowledge constraints. The probability distribution of component violation types is output, along with the probability of occurrence of each violation type and the corresponding standard basis.

[0123] (2) Combining the characteristics of multi-disciplinary ship design review, model the causal relationships of violations in multi-disciplinary components, construct a structural causal model, accurately locate the root causes of violations, and improve the interpretability of the design review results. The model is represented as follows: In the formula, This is the causal effect value, which quantifies the degree of influence of a design factor on the occurrence of violations. The larger the absolute value, the higher the probability that the factor is the root cause of the violation; Design influencing factors include component dimensions, spacing, material specifications, and equipment models. To avoid obfuscating variables such as ship type, deadweight tonnage, navigation area, and specification version, and to prevent non-causal factors from interfering with root cause determination; To intervene, the value of a certain influencing factor is fixed, and the change in the probability of violation is observed. The influence of confounding variables is removed, and the root cause of the violation is accurately located (e.g., the root cause of structural strength violation is excessive rib spacing rather than ambient temperature).

[0124] (3) To address the real-time requirements of ship plan review scenarios, and considering the computational characteristics of the MoE architecture, an inference efficiency optimization model is constructed to control plan review latency. The model is represented as follows: In the formula, The number of activated expert networks is controlled by a dynamic routing mechanism, with an optimal value of 2 to 3 to balance accuracy and efficiency. For the first The input feature dimension of each expert network is adaptively adjusted according to the professional complexity (e.g., the feature dimension of marine engineering is higher than that of the overall professional). Calculate weights for the expert network and allocate them based on the priority of professional drawing review; Optimized for GPU parallel computing to improve hardware computing speed; To reduce the time consumption of dynamic routing, a knowledge-guided routing strategy is used to decrease matching time; the model training objective is to delay the review process for a single set of PDF drawings. This meets the actual needs for drawing review efficiency.

[0125] Step 3: Build a large-scale professional expert model and implement dual fine-tuning training. Divide the model into dedicated expert modules according to the six major specialties of shipbuilding, build a professionally adapted MoE architecture, design a dual strategy of "multi-task fine-tuning + expert hybrid fine-tuning", combine human reinforcement feedback learning in each specialty to achieve professional-level fine-tuning of the model, and construct the "collaborative computing and accuracy optimization framework" of the MoE architecture.

[0126] (1) Based on the differences in drawing review tasks and knowledge characteristics among the six major shipbuilding specialties, six professional expert networks are established. Each expert network focuses on the core drawing review tasks of its corresponding specialty, forming a "one specialty, one core expert" architecture. A shared parameter layer is also set up to carry general shipbuilding drawing review knowledge (such as general regulatory clauses and basic PDF processing capabilities). The large model parameters are represented as follows: In the formula, The basic parameters shared by all experts in the network include multimodal feature extraction of general PDF drawings, general ship specifications, and basic logic for cross-disciplinary collaboration, ensuring that the model has basic cross-disciplinary drawing review capabilities; For the first Private parameters of an expert network ( These correspond to the general, structural, piping, marine, electrical, and outfitting specialties, respectively. They include knowledge such as PDF drawing feature extraction rules, professional specification matching logic, and exclusive violation type identification algorithms for each specialty, such as strength verification parameters for structural specialties and wiring specification matching parameters for electrical specialties.

[0127] (2) Training is carried out by combining “multi-task basic fine-tuning + expert mixed fine-tuning” to achieve synergistic improvement of general ability and professional ability.

[0128] Multi-task basic fine-tuning uses common drawing review tasks from six major disciplines as training targets, constructing a unified fine-tuning task set, including PDF drawing text / graphic information extraction, general specification compliance verification, and cross-disciplinary basic conflict detection. This is based on a multi-disciplinary fusion dataset. Training models share parameters By optimizing parameters through gradient descent algorithm, the model is ensured to have cross-disciplinary basic drawing review capabilities and be able to handle common drawing review scenarios of various disciplines.

[0129] Expert hybrid fine-tuning involves configuring a dedicated fine-tuning dataset for each professional review task, including PDF drawings, specific standards and regulations, historical review cases, and expert experience data for each expert network. For example, for piping experts, dedicated data on pipe layout conflicts, pipe diameter matching, and fluid mechanics standards are configured to train expert-specific parameters. Simultaneously, cross-disciplinary samples are introduced for mixed training (such as cross-conflict cases between structural components and pipeline layout) to avoid overfitting of expert networks, strengthen the collaborative review capabilities among disciplines, and ensure that the model can handle cross-disciplinary review tasks.

[0130] (3) Professional-level reinforcement feedback fine-tuning. Based on the reinforcement feedback learning behavior shaping module for various professions, feedback information from professional reviewers on the model's review results is collected, including result correction opinions, violation priority marking, supplementary reasons for violations, and corrections to regulatory basis. A reinforcement feedback reward function is constructed to guide iterative optimization of the model, aligning it with professional human judgment. The reward function formula is expressed as follows: In the formula, To reward the accuracy of image review, a reward is calculated based on the results of manual correction. A positive reward is given if the model's review result matches the human judgment; a negative penalty is given for missed or incorrect reviews. The quantification formula is as follows: ; The reward is for interpretability of results. It is based on the degree of human approval of the model's reasoning logic. If the model can accurately associate violations with professional normative clauses and clearly locate the root cause, a high reward will be given. To reward the reasonableness of cross-disciplinary references, the accuracy of the model's cross-disciplinary correlation judgment is evaluated (such as the judgment of cross-conflicts between structural components and electrical wiring). This is a weighting coefficient, adjusted according to professional drawing review requirements, with the optimal value selected. By iteratively optimizing expert parameters through the reinforcement learning PPO algorithm, professional-level model fine-tuning is achieved. Five senior ship drawing review experts were invited to provide feedback annotations on the model output, and a reinforcement feedback reward function was constructed. The expert parameters are iteratively optimized using the PPO algorithm.

[0131] (4) Design a knowledge-guided dynamic routing mechanism. This should be combined with multimodal fusion features. By integrating domain knowledge graphs, a gating network is designed to construct a knowledge-guided dynamic routing mechanism. This mechanism enables precise matching of input PDF drawing data with professional expert modules, ensuring that the appropriate expert module handles the corresponding professional drawing review task. The routing probability model is represented as follows: In the formula, This is a multimodal fusion feature query vector, generated from the multimodal features of the input PDF drawing; For the first Each professional expert's key vector carries the knowledge characteristics and task features of the corresponding profession; The vector dimension is used for normalization to avoid the curse of dimensionality. As a knowledge-guided matrix, it is constructed based on the matching degree of "professional-expert-PDF drawing features" of knowledge graph to quantify the adaptation relationship between different professional drawing features and expert modules; Assign the input PDF drawing data to the first The probability of each expert network is considered, and the expert networks with the top-2 to top-3 probabilities are selected to handle the current image review task, thereby achieving multi-expert collaborative computing.

[0132] (5) Collaborative Fusion of Multi-Expert Review Results. A combination of weighted fusion and rule-based validation is used to aggregate the output results of the expert network, generating the final review results and ensuring their accuracy and professionalism. The fusion method is represented as follows: In the formula, The number of expert networks activated for Top-K, ranging from 2 to 3; For the first The review results output by the expert network include information such as violation judgment, standard basis, modification suggestions, and root cause analysis; The routing probabilities of each expert are used as weights for result fusion; To standardize rule verification functions, based on the professional logic of IMO standards, classification society regulations, and knowledge graphs, cross-validation and correction of multi-expert results are performed (such as resolving conflicts in expert results and supplementing missing regulatory basis). To enhance the results, we integrate the strengths of various experts to generate complete and accurate drawing review results.

[0133] Step 4: Implement multimodal cross-referencing and closed-loop optimization. Based on multimodal technology, cross-referencing of knowledge and data across different disciplines is achieved. Combined with review results and human reinforcement feedback, a closed-loop mechanism of "data iteration - knowledge update - model optimization" is constructed to continuously improve the performance of large models.

[0134] (1) Multimodal cross-disciplinary reference implementation. Based on multimodal fusion features and knowledge graphs, a cross-disciplinary association model is constructed for PDF drawings, ship specifications, IMO regulations, and historical experience data to achieve accurate cross-disciplinary knowledge reference and solve the problem of cross-disciplinary drawing review conflict detection. For example, the component dimensions in structural PDF drawings can be automatically associated with the layout spacing specifications of the piping discipline, and the wiring design of the electrical discipline can be associated with the equipment location and heat dissipation requirements of the marine engineering discipline. The model is represented as follows: In the formula, For the first Professional and First Professional cross-citation probability; the higher the probability, the stronger the correlation between the data / knowledge of the two disciplines. The function for calculating feature similarity is cosine similarity algorithm, which is used to calculate the similarity between the multimodal feature embeddings of the two disciplines. They are respectively Professional multimodal feature embedding, including drawings, specifications, and knowledge graph features corresponding to the relevant profession; As a cross-disciplinary association matrix, it is constructed based on historical cross-disciplinary drawing review cases and knowledge graphs, and quantifies the association weights between various disciplines (e.g., the association weight of structural and piping disciplines is higher than that of general and electrical disciplines).

[0135] (2) Knowledge graph update driven by drawing review results. New violation associations and new professional knowledge (such as violation cases of battery compartment layout of new energy ships and newly added IMO regulations) discovered during the model reasoning process are combined with the results of manual review and confirmation, and updated through the knowledge graph update formula. Update the knowledge graph. This includes reinforcing the feedback reward function. Using step S33 This ensures that the updated knowledge aligns with professional logic and drawing review requirements, enriches the knowledge base for drawing review, and provides more comprehensive knowledge support for subsequent model training.

[0136] (3) Closed-loop optimization iteration. Based on multimodal cross-referencing results, image review results, and human reinforcement feedback information, closed-loop optimization data is formed. On the one hand, the knowledge graph is iteratively updated through the knowledge graph update formula to optimize... (Professional Components - Violation Type Correlation Matrix) and (Knowledge-guided matrix) provides more precise professional knowledge constraints for the inference model and dynamic routing mechanism; on the other hand, based on the optimized dataset and knowledge graph, double fine-tuning training is carried out again to iteratively optimize the MoE expert private parameters. With shared parameters The task weights and training iterations of the dual fine-tuning strategy were adjusted, and the parameters of the multimodal cross-reference model were optimized. Each round of image review and model optimization forms a closed-loop iteration, allowing the model to continuously absorb new cases and new standard knowledge in practical applications, and its performance gradually improves.

[0137] This invention integrates multiple professional knowledge of ships to build a large model training system, which has the following significant advantages over the existing technology: (1) It improves the integration of multiple professional knowledge and PDF processing capabilities. Focusing on the core carrier of PDF drawings, it effectively solves the problems of heterogeneous data of multiple professional ships, difficulty in extracting PDF information and weak cross-professional correlation through multimodal data fusion, domain knowledge graph construction and cross-professional reference technology, and realizes the structured integration and efficient reuse of multiple professional knowledge; (2) It optimizes the model accuracy and real-time performance in a coordinated manner. The professional-specific MoE architecture is combined with the "double fine-tuning + enhanced feedback" strategy, which greatly improves the model's accuracy and professional adaptability to the review of multiple professional PDF drawings, and effectively replaces manual completion of most repetitive tasks. (3) Enhanced cross-ship type generalization capability. Through the differentiated design of "shared parameters + professional private parameters", the model can quickly adapt to various ship types such as new energy ships, large container ships, and bulk carriers. The cross-ship type review accuracy decreases, and there is no need to carry out large-scale retraining for new ship types, reducing the cost of adapting to the review of new ship types and adapting to the diversified design needs of the shipbuilding industry. (4) Achieved continuous performance iteration and improvement. The closed-loop optimization mechanism of "data-knowledge-model-feedback" enables the model to continuously absorb new specifications and new case knowledge in actual review applications. After running for a total of 3 months, the review accuracy is further improved. It can dynamically adapt to the updates of shipbuilding industry regulations and the upgrade of design technology, and support the intelligent review of ships in the long term.

[0138] In another embodiment, such as Figure 3 As shown, a method for reviewing ship drawings based on a large multi-disciplinary expert model is provided. Taking the application of this method to a server as an example, the method includes the following steps:

[0139] Step S302: Obtain the ship drawings to be reviewed and extract the multi-dimensional fusion features of the ship drawings.

[0140] Step S304: Obtain the association matrix between ship components and violation types from the knowledge graph.

[0141] Step S306: Input the association matrix and multi-dimensional fusion features into the pre-trained violation type prediction model to obtain the violation type probability distribution results.

[0142] Step S308: Based on the probability distribution of violation types, determine the violation type of the ship drawings and the root cause of the violation type.

[0143] Step S310: Based on the multi-dimensional fusion features, select at least one target expert network from the pre-trained multi-professional expert large model, and input the multi-dimensional fusion features, violation type and root cause into each target expert network to obtain the review results output by each target expert network.

[0144] Step S312: Based on the review results output by each target expert network, generate a review report of the ship drawings.

[0145] It should be noted that the specific limitations of the above steps can be found in the above description of the specific limitations of a ship drawing review method based on a large model by multiple professional experts.

[0146] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0147] The following describes the ship drawing review device based on a multi-disciplinary expert large model provided in the embodiments of this application. The ship drawing review device based on a multi-disciplinary expert large model has the same inventive concept as the ship drawing review method based on a multi-disciplinary expert large model described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the ship drawing review device based on a multi-disciplinary expert large model provided below can be found in the limitations of the ship drawing review method based on a multi-disciplinary expert large model described above. The ship drawing review device based on a multi-disciplinary expert large model described below and the ship drawing review method based on a multi-disciplinary expert large model described above can be referred to each other, and will not be repeated here.

[0148] In one exemplary embodiment, Figure 4 A structural schematic diagram of a ship drawing review device based on a large multidisciplinary expert model provided in this application embodiment is shown below. Figure 4 As shown, the ship drawing review device based on a multi-disciplinary expert large model includes: an acquisition module 402, a recognition module 404, an input module 406, and an output module 408, wherein:

[0149] The acquisition module 402 is used to acquire the ship drawings to be reviewed and extract the multi-dimensional fusion features of the ship drawings;

[0150] The identification module 404 is used to identify the types of violations in ship drawings based on multi-dimensional fusion features and determine the root causes of the violations.

[0151] Input module 406 is used to select at least one target expert network from the pre-trained multi-professional expert large model based on multi-dimensional fusion features, and input the multi-dimensional fusion features, violation type and root cause into each target expert network to obtain the review results output by each target expert network;

[0152] Output module 408 is used to generate an audit report of ship drawings based on the audit results output by each target expert network.

[0153] In an exemplary embodiment, the acquisition module 402 is specifically used to extract multimodal features of ship drawings; based on the multimodal features, to obtain professional knowledge association features of ship drawings from a pre-constructed knowledge graph; and to fuse the multimodal features and professional knowledge association features to obtain multidimensional fused features.

[0154] In an exemplary embodiment, the acquisition module 402 is specifically used to acquire two-dimensional drawing data, text data, and three-dimensional ship model data corresponding to the ship drawings; extract visual features based on the two-dimensional drawing data, extract semantic features based on the text data, and extract spatial features based on the three-dimensional ship model data; and fuse the visual features, semantic features, and spatial features to obtain multimodal features.

[0155] In an exemplary embodiment, the acquisition module 402 is specifically used to identify at least one ship component appearing in the ship drawings based on multimodal features; extract the embedding features of each ship component from a pre-built knowledge graph, and aggregate the embedding features of each ship component to obtain the professional knowledge association features of the ship drawings.

[0156] In an exemplary embodiment, the identification module 404 is specifically used to obtain the association matrix between ship components and violation types from the knowledge graph; input the association matrix and multi-dimensional fusion features into a pre-trained violation type prediction model to obtain the violation type probability distribution result; and determine the violation type of the ship drawings based on the violation type probability distribution result.

[0157] In an exemplary embodiment, the input module 406 is specifically configured to generate a multi-dimensional fusion feature query vector based on multi-dimensional fusion features, generate key vectors for each expert network based on the knowledge features and task features of each expert network, and construct a knowledge guidance matrix based on a knowledge graph; the knowledge guidance matrix represents the adaptation relationship between different professional drawing features and different expert networks; input the multi-dimensional fusion feature query vector, the key vectors of each expert network, and the knowledge guidance matrix into the routing probability model to determine the routing probability of each expert network; and determine the target expert network among the expert networks based on the routing probability of each expert network.

[0158] In an exemplary embodiment, the identification module 404 is configured to determine, based on the violation causal model, each design influencing factor that leads to the occurrence of a violation type, and determine the causal effect value of each design influencing factor; and determine the root cause of the violation type based on the causal effect value of each design influencing factor.

[0159] In one exemplary embodiment, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the ship drawing review methods based on a multi-disciplinary expert large model described above.

[0160] In one exemplary embodiment, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the ship drawing review methods based on multi-disciplinary expert large models described above.

[0161] In one exemplary embodiment, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the ship drawing review methods based on a multi-disciplinary expert large model as described in the above embodiments.

[0162] Indicatively, such as Figure 5 As shown, Figure 5 This is a schematic diagram of the internal structure of a computer device 500 provided in an embodiment of this application. The computer device 500 can be provided as a server. (Refer to...) Figure 5 The computer device 500 includes a processing component 502, which further includes one or more processors, and memory resources represented by memory 501 for storing instructions, such as application programs, that can be executed by the processing component 502. The application programs stored in memory 501 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 502 is configured to execute instructions to perform the ship drawing review method based on a multi-disciplinary expert large model, as described in any of the above embodiments.

[0163] The computer device 500 may also include a power supply component 503 configured to perform power management of the computer device 500, a wired or wireless network interface 504 configured to connect the computer device 500 to a network, and an input / output (I / O) interface 505. The computer device 500 may operate on an operating system stored in memory 501, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.

[0164] Those skilled in the art will understand that Figure 5The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0165] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0166] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0167] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for reviewing ship drawings based on a large model by multiple experts, characterized in that, The method includes: Obtain the ship drawings to be reviewed and extract the multi-dimensional fusion features of the ship drawings; Based on the multi-dimensional fusion features, the types of violations in the ship drawings are identified, and the root causes of the violations are determined. Based on the multi-dimensional fusion features, at least one target expert network is selected from the pre-trained multi-professional expert large model, and the multi-dimensional fusion features, the violation type and the root cause are input into each target expert network to obtain the audit results output by each target expert network. Based on the review results output by each of the target expert networks, a review report for the ship drawings is generated.

2. The method according to claim 1, characterized in that, The extraction of multi-dimensional fusion features from the ship drawings includes: Extract the multimodal features of the ship drawings; Based on the multimodal features, the professional knowledge association features of the ship drawings are obtained from the pre-constructed knowledge graph; The multimodal features and the professional knowledge-related features are fused to obtain the multidimensional fused features.

3. The method according to claim 2, characterized in that, The extraction of multimodal features from the ship drawings includes: Obtain the two-dimensional drawing data, text data, and three-dimensional ship model data corresponding to the ship drawings; Based on the two-dimensional drawing data, visual features are extracted; based on the text data, semantic features are extracted; and based on the three-dimensional ship model data, spatial features are extracted. The multimodal features are obtained by fusing the visual features, the semantic features, and the spatial features.

4. The method according to claim 2, characterized in that, The step of obtaining the professional knowledge association features of the ship drawings from a pre-constructed knowledge graph based on the multimodal features includes: Based on the multimodal features, at least one ship component appearing in the ship drawings is identified; The embedding features of each ship component are extracted from the pre-constructed knowledge graph, and the embedding features of each ship component are aggregated to obtain the professional knowledge association features of the ship drawings.

5. The method according to claim 4, characterized in that, The identification of violation types in the ship drawings based on the multi-dimensional fusion features includes: Obtain the association matrix between the ship components and the types of violations from the knowledge graph; The correlation matrix and the multi-dimensional fused features are input into a pre-trained violation type prediction model to obtain the violation type probability distribution results; Based on the probability distribution of the violation types, the violation types of the ship drawings are determined.

6. The method according to claim 2, characterized in that, The step of selecting at least one target expert network from a pre-trained multi-professional expert large model based on the multi-dimensional fusion features includes: Based on the multi-dimensional fusion features, a multi-dimensional fusion feature query vector is generated; based on the knowledge features and task features of each expert network, a key vector of each expert network is generated; and based on the knowledge graph, a knowledge guidance matrix is ​​constructed; the knowledge guidance matrix represents the adaptation relationship between different professional drawing features and different expert networks. The multi-dimensional fusion feature query vector, the key vector of each expert network, and the knowledge guidance matrix are input into the routing probability model to determine the routing probability of each expert network. The target expert network is determined among the expert networks based on the routing probabilities of each expert network.

7. The method according to any one of claims 1 to 6, characterized in that, The determination of the root cause of the aforementioned violation type includes: Based on the violation causal model, the design influencing factors that lead to the occurrence of the violation type are identified, and the causal effect value of each design influencing factor is determined. Based on the causal effect values ​​of each of the aforementioned design influencing factors, determine the root cause of the aforementioned violation type.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images