Nuclear power engineering drawing identification method and system based on visual language multi-modal large model

Abstract

The invention belongs to the field of nuclear power engineering, and particularly relates to a nuclear power engineering drawing recognition method and system based on a visual language multi-modal large model, and the method comprises the following steps: obtaining a to-be-recognized nuclear power engineering drawing and a text input by a user, comparing the drawing information with a data set based on the multi-modal large model, and obtaining a recognition result; and verifying the logic consistency of the information on the drawing by using a multi-modal data fusion technology, and outputting a verification result and analyzing the information. Compared with the prior art, the challenges of complexity, accuracy and the like encountered by a traditional drawing recognition method in the nuclear power engineering field are solved; according to the scheme, an advanced multi-modal large model (visual language large model) is fused, nuclear power engineering drawings and related document data are combined, a hidden association mode is mined through organic combination of multi-modal data, the recognition and analysis effect is improved, and the limitation of traditional single-modal analysis is broken through.

Description

Technical Field

[0001] The present invention belongs to the field of nuclear power engineering, and in particular relates to a method and system for recognizing nuclear power engineering drawings based on a visual language multimodal large model. Background Art

[0002] Traditional engineering drawing recognition generally uses deep learning combined with traditional computer vision algorithms, which can automatically extract text, symbols and graphics from drawings and realize vectorization conversion. Optical character recognition technology is mature and can accurately extract text information from drawings, even if the font is irregular or noisy. Despite this, in the face of complex layouts, non-standard symbols and degradation of old drawings in nuclear power engineering drawings, the recognition rate still needs to be improved, and support for Chinese characters is insufficient. In addition, traditional drawing recognition technology cannot be used for more complex situational understanding, such as establishing associations between drawings and related specifications or manuals. At present, engineering drawing recognition technology based on multimodal large models is still in a blank stage.

[0003] With the rapid development of artificial intelligence technology, large model technology has made significant progress and has been widely used in language processing, image recognition and other fields, improving the decision-making efficiency and quality of various industries. Although large models in vertical fields have advantages in professionalism and accuracy, they also face multiple obstacles such as data limitations, high costs, overfitting risks, limited generalization capabilities, difficulties in updating and maintenance, model homogeneity, technical integration challenges, ethical bias issues, real-time response limitations, and high dependence on professional knowledge. These factors together constitute the complexity and limitations of its application process, which need to be fully considered and resolved in actual deployment.

[0004] As for the field of nuclear power engineering, there is currently no multimodal large model specifically for this field, which provides broad opportunities for technological innovation. Summary of the invention

[0005] The purpose of the present invention is to provide a nuclear power engineering drawing recognition method and system based on a visual language multimodal large model in order to solve at least one of the above problems, so as to solve the challenges of complexity, accuracy and the like encountered by traditional drawing recognition methods in the field of nuclear power engineering. This solution integrates an advanced multimodal large model (visual language large model), combines nuclear power engineering drawings with related document data, and through the organic combination of multimodal data, excavates hidden association patterns, improves the recognition and analysis effects, and breaks the limitations of traditional single modal analysis.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] The first aspect of the present invention discloses a method for recognizing nuclear power engineering drawings based on a visual language multimodal large model, comprising the following steps:

[0008] Obtain the nuclear power engineering drawings to be identified and the text entered by the user, compare the drawing information with the data set based on the multimodal large model, use the multimodal data fusion technology to verify the logical consistency of the information on the drawings, and output the verification results and information analysis;

[0009] The training strategy of the multimodal large model includes low-rank fine-tuning, quantized low-rank adaptation and full-parameter fine-tuning, and the multimodal large model is optimized through key-value caching;

[0010] The data set is constructed by the steps of data collection, data processing and data cleaning; the data sources of the data collection include nuclear power engineering drawings and public data; the data processing includes data construction and core data semantic expansion, the data construction includes generating question-answer pairs, and the core data semantic expansion includes similar semantic expansion and form expansion.

[0011] The public data include professional knowledge of the nuclear power industry such as standard specifications, technical manuals, operating guidelines, procedural documents, and standard drawings; the nuclear power engineering drawings include internal engineering drawings, past cases, and historical versions.

[0012] Preferably, the data construction includes calling a large language model to generate a question-answer data set for public data and manually marking question-answer pairs on nuclear power engineering drawings to generate a question-answer data set.

[0013] Preferably, the data cleaning includes regularized matching, removal of duplicate images, keyword filtering and entity recognition.

[0014] Preferably, the similar semantic expansion includes similarly expanding the keywords of the question; the form expansion includes converting the form of the question and answer pair into other forms, and the other forms include cloze, polynomial selection and continuation.

[0015] The second aspect of the present invention discloses a nuclear power engineering drawing recognition system based on a visual language multimodal large model, using any of the above methods;

[0016] The identification system includes a basic layer, an application layer and a display layer;

[0017] The basic layer includes a data module, a model training module and an evaluation and optimization module; the data module is used for data storage and data set construction; the model training module is used to train and optimize the multimodal large model using the data set; the evaluation and optimization module is used to evaluate and improve the multimodal large model;

[0018] The application layer includes a nuclear power engineering drawing review module and a nuclear power knowledge question and answer module; the nuclear power engineering drawing review module is used to review the information of the nuclear power engineering drawings to be identified; the nuclear power knowledge question and answer module is used to answer questions from users;

[0019] The display layer includes a user interface and an interaction module; the user interface and the interaction module are used to interact with the user.

[0020] Preferably, the evaluation and optimization module includes a recognition accuracy test submodule, and the recognition accuracy test submodule quantifies the recognition accuracy of the multimodal large model by calculating key performance indicators, and the key performance indicators include accuracy, recall rate and F1 score.

[0021] Preferably, the evaluation and optimization module includes a non-ideal condition test submodule, and the non-ideal condition test submodule checks the performance of the multimodal large model under non-ideal conditions through noise injection and variant testing.

[0022] Preferably, the evaluation and optimization module includes a decision mechanism analysis submodule, and the decision mechanism analysis submodule analyzes the decision mechanism of the multimodal large model by generating a heat map and tracing the decision path.

[0023] Preferably, the base layer also includes a model deployment module;

[0024] The model deployment module includes a model segmentation submodule and a model compression submodule;

[0025] The model segmentation submodule is used to segment the multimodal large model into a bottom model and a top model; the bottom model includes an embedding layer and is deployed in a trusted execution environment of a computing center server; the top model is deployed in a GPU of the computing center server;

[0026] The model compression submodule is used to implement model compression on a large multimodal model; the model compression includes weight pruning, quantization and knowledge distillation.

[0027] Preferably, the base layer also includes a security and compliance module; the security and compliance module protects sensitive information by implementing encryption technology, access control and audit tracking.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] (1) The first of its kind in the industry and of great application value:

[0030] This method and system are tailor-made for the field of nuclear power engineering. It uses intelligent means to accurately recognize and analyze engineering drawings in the nuclear power field and automatically extract key information. The system not only has powerful drawing review and verification functions to ensure the accuracy of the design, but also has a built-in knowledge question-and-answer system for the nuclear power industry to answer professional questions in real time. Through conditional logical reasoning, the system can assist in decision-making, optimize emergency response strategies, and further improve the operating safety and efficiency of nuclear power facilities. In addition, the system is also pioneeringly applied to the training and education of nuclear industry knowledge, using advanced scientific and technological means to cultivate industry talents. This invention marks the first successful practice of multimodal large model technology in the field of nuclear power engineering.

[0031] (2) Multimodal datasets in the nuclear power field:

[0032] Different from other multimodal datasets, this system includes a brand new multimodal dataset in the field of nuclear power. Focusing on the field of nuclear power, the system combines manual annotation by engineering experts and calls large models to automatically generate similar semantic content, conduct semantic amplification and form expansion in the private domain of core data, expand the amount and diversity of data, and greatly increase data diversity and generalization effect.

[0033] (3) Ability to understand complex scenes and symbols:

[0034] Traditional drawing recognition systems currently have difficulty processing highly complex and dense information in nuclear power engineering drawings, such as complex conditional judgments and logical reasoning, multi-level structural details, and a large number of professional symbols. By training a large-scale visual language model, this system can deeply understand these complex scenes, accurately identify and analyze various elements and their relationships. Nuclear power engineering drawings contain a large number of unique and complex symbols, and accurate identification of these symbols is crucial for the correct interpretation of the drawings. Combined with the professional knowledge base of nuclear power engineering, this system realizes the recognition and understanding of the meaning of these professional symbols, significantly improving the accuracy of drawing analysis.

[0035] (4) Multimodal model training at multiple levels and different model scales:

[0036] After semantic expansion and diversified augmentation of private and non-private data, fine-tuning and optimizing multiple multimodal large models with different parameter scales. The model scale ranges from the smallest 2B to the highest 26B parameter level; training strategies covering low-rank fine-tuning, quantized low-rank adaptation, and full-parameter fine-tuning methods are carried out to meet many practical needs in various scenarios and different video memory conditions; key-value cache optimization is also carried out, which can more conveniently and efficiently deploy the private domain nuclear industry drawing system.

[0037] (5) Security and compliance assurance:

[0038] Taking into account the particularity of the nuclear power field, special requirements for data and business confidentiality, and a unique nuclear safety culture, this system has particularly strengthened the security and compliance inspection mechanism to ensure that the system complies with relevant laws and regulations and the strict standards and specifications of the nuclear power industry, providing safe and reliable support for subsequent engineering design, construction, and operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the architecture of the nuclear power engineering drawing recognition system of the present invention;

[0040] Figure 2 A schematic diagram of a Web interactive interface of a nuclear power engineering drawing recognition system of the present invention;

[0041] Figure 3 A schematic diagram of a process flow for constructing a data set for a nuclear power engineering drawing recognition system of the present invention;

[0042] In the figure: 100-basic layer; 200-application layer; 300-display layer; 110-data module; 120-model training module; 130-evaluation and optimization module; 140-model deployment module; 150-safety and compliance module; 210-nuclear power engineering drawing review module; 220-nuclear power knowledge question and answer module; 310-user interface and interaction module. DETAILED DESCRIPTION

[0043] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Example 1

[0045] like Figure 1-3 shown.

[0046] This embodiment discloses a method for recognizing nuclear power engineering drawings based on a visual language multimodal large model, comprising the following steps:

[0047] Obtain the nuclear power engineering drawings to be identified and the text entered by the user, compare the drawing information with the data set based on the multimodal large model, use the multimodal data fusion technology to verify the logical consistency of the information on the drawings, and output the verification results and information analysis;

[0048] The training strategy of the multimodal large model includes low-rank fine-tuning, quantized low-rank adaptation and full-parameter fine-tuning, and the multimodal large model is optimized through key-value caching;

[0049] The data set is constructed by the steps of data collection, data processing and data cleaning; the data sources of the data collection include nuclear power engineering drawings and public data; the data processing includes data construction and core data semantic expansion, the data construction includes generating question-answer pairs, and the core data semantic expansion includes similar semantic expansion and form expansion.

[0050] This embodiment also discloses a nuclear power engineering drawing recognition system based on a visual language multimodal large model, using the method described above;

[0051] The identification system comprises a base layer 100, an application layer 200 and a presentation layer 300;

[0052] The base layer 100 includes a data module 110, a model training module 120 and an evaluation and optimization module 130; the data module 110 is used for data storage and data set construction; the model training module 120 is used to train and optimize the multimodal large model using the data set; the evaluation and optimization module 130 is used to evaluate and improve the multimodal large model;

[0053] The application layer 200 includes a nuclear power engineering drawing review module 210 and a nuclear power knowledge question and answer module 220; the nuclear power engineering drawing review module 210 is used to review the information of the nuclear power engineering drawings to be identified; the nuclear power knowledge question and answer module 220 is used to answer questions from users;

[0054] The display layer 300 includes a user interface and an interaction module 310 ; the user interface and the interaction module 310 are used to interact with the user.

[0055] More specifically, in this embodiment:

[0056] This nuclear power engineering drawing recognition system contains multiple layers and modules, such as Figure 1 As shown, the basic layer 100 includes a data module 110, a model training module 120, an evaluation and optimization module 130, a model deployment module 140, and a safety and compliance module 150; the application layer 200 includes a nuclear power engineering drawing review module 210 and a nuclear power knowledge question and answer module 220; and the display layer 300 includes a user interface and interaction module 310.

[0057] The data module 110 is the core of data lifecycle management. It consists of a series of submodules, including the data storage and management submodule, the data set construction submodule, the data update submodule, and the data maintenance submodule. In the data set construction submodule, the construction of the data set covers data collection, processing, and cleaning to ensure high-quality data supply; in the data update submodule, the data update mechanism is responsible for seamlessly integrating new data into the existing data set to maintain the timeliness of the data; and in the data maintenance submodule, data maintenance ensures data traceability and version comparison capabilities through version control and change records. Through these finely divided submodules, the entire data module 110 realizes full-process automation and intelligent management from data collection to maintenance, providing data support for other modules.

[0058] The model training module 120 is one of the core components of the entire system. Its design and implementation must achieve a careful balance between model accuracy, computing performance and environmental adaptability. Relying on the detailed resources provided by the data module 110, the model training module 120 has the ability to train multimodal large models ranging from 2B parameter scale to 26B parameter scale. The training strategy covers methods such as low-rank fine-tuning, quantized low-rank adaptation, and full-parameter fine-tuning to cope with training under different application backgrounds and GPU memory constraints, ensuring that even in a resource-constrained environment, model training can be efficiently promoted, fully tapping and utilizing existing hardware resources, and accurately meeting the complex and diverse recognition needs in the field of nuclear power engineering. In addition, the introduction of key-value cache optimization measures has significantly improved the efficiency and fluency of model deployment and operation, providing a more efficient and convenient solution for private domain deployment scenarios, and greatly improving the system's real-time response capability and deployment flexibility.

[0059] The evaluation and optimization module 130 is a core component of the nuclear power engineering drawing recognition system, which aims to comprehensively evaluate and optimize the performance of the multimodal large model. The evaluation and optimization module 130 is connected to the data module 110, which provides a variety of test data sets that can cover drawings of various complexities to ensure that the model can perform stably in different scenarios. The recognition accuracy test submodule in the evaluation and optimization module 130 quantifies the recognition accuracy of the model by calculating key performance indicators such as accuracy, recall rate and F1 score, and measures the running speed of each module to ensure its efficient operation. In order to enhance the robustness of the system, the non-ideal condition test submodule in the evaluation and optimization module 130 also performs noise injection and variant testing to check the performance of the model under non-ideal conditions. In addition, the decision mechanism analysis submodule in the evaluation and optimization module 130 deeply analyzes the decision mechanism of the model by generating heat maps and tracing decision paths to improve its interpretability. The evaluation and optimization module 130 also collects feedback from domain experts, combines the test result feedback of the user interface, and forms a detailed evaluation report to provide a basis for the continuous optimization of the model. The entire evaluation process is repeated over and over again, and the joint training module continuously drives the improvement of model capabilities, making the recognition system more accurate, reliable, and fast.

[0060] The model deployment module 140 mainly includes a model segmentation submodule and a model compression submodule. Specifically, the model segmentation submodule divides the large model into two parts: a bottom model and a top model. The bottom model mainly includes a small number of embedding layers and is deployed in the trusted execution environment of the computing center server to reduce the computing load; at the same time, the top model is placed in the graphics card of the computing center server to utilize the high processing power of the GPU. The model compression submodule compresses the model through a variety of technical means, including but not limited to the use or combined use of one or more of weight pruning, quantization, knowledge distillation, etc. By applying these technologies, the number of parameters and computing requirements of the model are significantly reduced, thereby reducing the computing burden in the trusted execution environment and speeding up the execution of the model on the GPU. The model deployment module 140 encapsulates the trained model and converts it into an executable service, which is called by each module of the application layer 200.

[0061] Security and Compliance Module 150 focuses on ensuring data security, model stability, and operational compliance. It protects sensitive information by implementing encryption technology, access control, and audit trails. Security and Compliance Module 150 also ensures that the design and operation of this system comply with relevant laws and regulations in the nuclear field and standards of the nuclear energy industry, provides continuous system monitoring and emergency response mechanisms, and can generate compliance reports regularly. Together with employee training and risk assessment, a comprehensive security protection system is built to provide a solid guarantee for system security and compliance.

[0062] The user interface and interaction module 310 is a web-based interactive interface that is user-centric and provides an intuitive and responsive operating experience. As a display window for each module of the application layer 200, the user interface and interaction module 310 integrates advanced real-time data processing technology, greatly improving interactivity and efficiency. The adaptive design of this module ensures consistent high-quality visual effects regardless of the device. At the same time, it also has a powerful session management function, allowing users to save progress at any time and continue in the future, which facilitates the processing of complex tasks. In addition, the development of the module takes into account a high degree of customization requirements. Through a simple programming interface, it allows developers to fine-tune every detail according to specific application scenarios and expand functions.

[0063] The nuclear power engineering drawing review module 210 can automatically compare the drawings with the procedure files, standard drawings and historical versions stored in the existing database based on the multimodal large model generated by the model training module 120 according to the nuclear power engineering drawing images input by the user in the user interface and interaction module 310, and use the multimodal data fusion technology to verify the logical consistency between the text descriptions, graphic symbols, digital annotations and other information on the drawings, and detect differences and potential errors. Finally, the review results are output in the user interface and interaction module 310. The nuclear power engineering drawing review module 210 can automatically generate a review report based on the review results, listing the problems found, modification suggestions and review conclusions in detail, so that project managers and designers can quickly understand the status of the drawings and take corresponding measures.

[0064] The nuclear power knowledge question and answer module 220 is an important part of this system, which aims to provide instant, accurate and in-depth knowledge query and answer functions in the field of nuclear power. This module makes full use of the powerful capabilities of the multimodal large model, combined with the professional knowledge of the nuclear power industry in the data module 110, such as standards and specifications, technical manuals, engineering drawings, operating guidelines and past cases, etc., through multimodal information fusion, to provide comprehensive and accurate answers, and can process and understand complex nuclear power engineering drawings and related technical documents, and realize efficient knowledge retrieval and intelligent question and answer. The nuclear power knowledge question and answer module 220 includes a user feedback mechanism to collect feedback from designers, engineers and end users, continuously optimize the question and answer effect, and improve the system accuracy and user satisfaction.

[0065] like Figure 2 The figure shows a web-based interactive interface provided in this embodiment.

[0066] Step 101, first upload the nuclear power engineering drawing file in the upload box, usually set the acceptable format to picture or PDF, and support up to five pictures to be uploaded at the same time, and then execute step 102.

[0067] Step 102, adjust the output parameters in the parameter adjustment column, such as the maximum number of outputs, the cluster size, and the repetition penalty, and then execute step 103.

[0068] Step 103, ask questions in the dialog box (if drawings are uploaded, ask questions in conjunction with the drawings), the system will answer the questions based on the questions asked, and then execute step 104.

[0069] Step 104, there are multiple functional cases at the bottom of the dialog box, and you can choose to regenerate, clear history, like, or dislike to provide feedback.

[0070] like Figure 3 Shown is a flow chart of constructing a data set according to the present invention.

[0071] Step 201, collect data, which are divided into non-confidential data (public data: manuals, standards, guidelines, etc.) and confidential data (non-public data: nuclear power engineering drawings, past cases, etc.) according to their types. Non-confidential data mainly comes from the Internet, and program logic diagrams on the Internet are crawled through a web crawler program. Confidential data mainly comes from internal nuclear power logic diagram screenshots, and then step 202 is executed.

[0072] Step 202, data processing, is further divided into data construction and core data semantic expansion. Non-confidential data crawled from the Internet needs to call a large language model to generate a question-and-answer data set. Confidential data such as nuclear power engineering logic drawings use industry experts to manually annotate question-and-answer pairs to generate data sets. The core data semantic expansion is: 1) Perform similar semantic expansion (keyword expansion) on the questions to increase the number of pictures and question-and-answer pairs, thereby improving the recognition effect; 2) Perform form expansion on the questions, converting question-and-answer pairs into cloze, polynomial selection, continuation, etc., thereby improving the diversity of data set formats and generalization effects. By calling a large language model, the prompt words are optimized, and the semantics are automatically expanded to generate new question-and-answer pairs and new forms, and then execute step 203.

[0073] In step 202, non-confidential data can call large language models such as gpt4-o to generate a qa data set; similar semantic expansion can call Internllm-20b to optimize the prompt, automatically expand the semantics, and generate new qa; formal expansion can call Internllm-20b to optimize the prompt, automatically expand the semantics, and generate new qa.

[0074] Step 203, clean the data. Non-confidential data needs to go through a series of cleansing. Through regularized matching, advertisements, illegal links, etc. are removed. Duplicate images are removed to reduce storage space requirements and avoid redundancy in model training. Unify the format for easy processing and storage. Standardize the size to speed up model training and reduce memory consumption. Remove sensitive sentences through keyword filtering, entity recognition, etc. to comply with relevant laws and regulations and public order and good customs. Review the questions and verify the correctness of the results generated by the big prediction model, and delete scientific fantasy and erroneous content. Improve the quality of question and answer pairs, continuously optimize and improve, and correct the naming, question and answer difficulty, question and answer angle, and logic.

[0075] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.

Claims

1. A method for recognizing nuclear power engineering drawings based on a multimodal large model of visual language, characterized in that: The steps include: Obtain the nuclear power engineering drawings to be identified and the text entered by the user, compare the drawing information with the data set based on the multimodal large model, use the multimodal data fusion technology to verify the logical consistency of the information on the drawings, and output the verification results and information analysis; The training strategy of the multimodal large model includes low-rank fine-tuning, quantized low-rank adaptation and full-parameter fine-tuning, and the multimodal large model is optimized through key-value caching; The data set is constructed by the steps of data collection, data processing and data cleaning; the data sources of the data collection include nuclear power engineering drawings and public data; the data processing includes data construction and core data semantic expansion, the data construction includes generating question-answer pairs, and the core data semantic expansion includes similar semantic expansion and form expansion.

2. According to claim 1, a method for recognizing nuclear power engineering drawings based on a visual language multimodal large model is characterized in that: The data construction includes calling a large language model to generate a question-answer data set for public data and manually marking question-answer pairs on nuclear power engineering drawings to generate a question-answer data set.

3. The method for recognizing nuclear power engineering drawings based on a visual language multimodal large model according to claim 1 is characterized in that: The data cleaning includes regularized matching, removal of duplicate images, keyword filtering and entity recognition.

4. The method for recognizing nuclear power engineering drawings based on a visual language multimodal large model according to claim 1 is characterized in that: The similar semantic expansion includes similar expansion of keywords in the question; the form expansion includes converting the form of the question and answer pair into other forms, and the other forms include cloze, polynomial selection and continuation.

5. A nuclear power engineering drawing recognition system based on a visual language multimodal large model, characterized in that: Adopt the method as claimed in any one of claims 1 to 4; The identification system comprises a basic layer (100), an application layer (200) and a presentation layer (300); The basic layer (100) includes a data module (110), a model training module (120) and an evaluation and optimization module (130); the data module (110) is used for data storage and data set construction; the model training module (120) is used for training and optimizing a multimodal large model using a data set; the evaluation and optimization module (130) is used for evaluating and improving the multimodal large model; The application layer (200) comprises a nuclear power engineering drawing review module (210) and a nuclear power knowledge question and answer module (220); the nuclear power engineering drawing review module (210) is used to review information on nuclear power engineering drawings to be identified; the nuclear power knowledge question and answer module (220) is used to answer questions from users; The display layer (300) comprises a user interface and an interaction module (310); the user interface and the interaction module (310) are used to interact with a user.

6. A nuclear power engineering drawing recognition system based on a visual language multimodal large model according to claim 5, characterized in that: The evaluation and optimization module (130) includes a recognition accuracy test submodule, and the recognition accuracy test submodule quantifies the recognition accuracy of the multimodal large model by calculating key performance indicators, and the key performance indicators include accuracy, recall rate and F1 score.

7. The nuclear power engineering drawing recognition system based on visual language multimodal large model according to claim 5 is characterized in that: The evaluation and optimization module (130) includes a non-ideal condition test submodule, and the non-ideal condition test submodule checks the performance of the multimodal large model under non-ideal conditions through noise injection and variant testing.

8. The nuclear power engineering drawing recognition system based on visual language multimodal large model according to claim 5 is characterized in that: The evaluation and optimization module (130) includes a decision mechanism analysis submodule, and the decision mechanism analysis submodule analyzes the decision mechanism of the multimodal large model by generating a heat map and tracing the decision path.

9. The nuclear power engineering drawing recognition system based on visual language multimodal large model according to claim 5 is characterized in that: The base layer (100) further includes a model deployment module (140); The model deployment module (140) includes a model segmentation submodule and a model compression submodule; The model segmentation submodule is used to segment the multimodal large model into a bottom model and a top model; the bottom model includes an embedding layer and is deployed in a trusted execution environment of a computing center server; the top model is deployed in a GPU of the computing center server; The model compression submodule is used to implement model compression on a large multimodal model; the model compression includes weight pruning, quantization and knowledge distillation.

10. The nuclear power engineering drawing recognition system based on visual language multimodal large model according to claim 5 is characterized in that: The base layer (100) also includes a security and compliance module (150); the security and compliance module (150) protects sensitive information by implementing encryption technology, access control and audit tracking.

Images