A multimodal large model security benchmark construction and multi-dimensional value automatic evaluation method and system

CN122885934APending Publication Date: 2026-10-09BEIJING UNIV OF TECH +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611221458.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-12
Publication Date
2026-10-09

AI Technical Summary

Technical Problem

然而现有评测方法主要依赖于单文本或单图像测试的单模态测评方式,无法有效暴露模型固有的视觉-语言对齐漏洞;此外,当前主流的评测体系存在评测维度片面的问题,普遍仅聚焦于模型安全风险判别,忽略模型实用性能、伦理合规、隐私防护等综合应用价值评估维度,难以全面衡量模型在基础设施场景的落地适配能力

Benefits of technology

本发明融合基准自动化搭建、隐蔽违规智能判别、多维度综合价值评估一体化流程,填补关键基础设施领域多模态大模型全流程标准化评测空白,在多模态AI安全评测、工业基础设施智能风控领域具备重要理论意义与工程落地价值。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122885934A_ABST
    Figure CN122885934A_ABST
Patent Text Reader

Abstract

The application discloses a multi-modal large model security benchmark construction and multi-dimensional value automatic evaluation method and system, belonging to the cross field of artificial intelligence and key infrastructure security protection; comprising: firstly, collecting multiple scene real scene images, combining diffusion model to generate expanded images, and constructing a double-source basic image library; secondly, generating adversarial question and answer samples through multiple heterogeneous large models, and obtaining standardized security benchmarks through three-level screening; then, using a judge model that fuses industrial control knowledge and thought chain reasoning to judge hidden illegal responses and quantify security vulnerabilities; finally, scoring the comprehensive value of the model from four dimensions. The application can automatically construct a high-accuracy and high-adversarial evaluation benchmark, accurately identify hidden illegal outputs, and simultaneously realize security risk detection and multi-dimensional value evaluation, having important research and application value in the fields of electric power energy, rail transportation, government services, etc.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of artificial intelligence and critical infrastructure security protection, specifically to a method and system for constructing a multimodal large-scale security benchmark and automatically assessing multidimensional value. Background Technology

[0002] Multimodal large-scale language models, integrating visual images and textual semantics to achieve cross-modal joint reasoning, have broad application prospects in high-risk operations of critical infrastructure such as power dispatching, rail transit operation and maintenance, industrial production line management, and modern agricultural monitoring. They can provide key technical support for core business processes such as equipment fault diagnosis, system control, and operation and maintenance decision analysis, serving as a core enabling technology for the intelligent upgrading of critical infrastructure. These models, relying on on-site inspection data, output professional operation and maintenance guidance, effectively reducing labor costs and significantly improving the overall operational efficiency of infrastructure. However, existing multimodal large-scale models have significant security shortcomings when applied to critical infrastructure scenarios. These models are susceptible to cross-modal adversarial attacks, potentially outputting high-risk operational solutions such as equipment damage and system malfunctions. If such security risks spiral out of control, they can directly lead to large-scale power outages, train accidents, industrial production line failures, and other safety problems, seriously threatening the operational safety, stability, and public safety of critical infrastructure. Therefore, establishing a standardized security evaluation system for critical infrastructure scenarios, and realizing automated security risk evaluation and multi-dimensional value quantification of multimodal large models, is an urgent need for the intelligent implementation of critical infrastructure, and has important practical significance and engineering value.

[0003] The image-text dual-modal fusion method can fully reproduce various physical defects in industrial sites and accurately recreate the implicit inducements of attackers to critical infrastructure scenarios, serving as the core technical support for large-scale model security evaluation. However, existing evaluation methods mainly rely on single-modal evaluation approaches using single text or single image tests, which cannot effectively expose inherent visual-language alignment vulnerabilities in models. Furthermore, current mainstream evaluation systems suffer from a one-sided evaluation dimension, generally focusing only on model security risk identification while neglecting comprehensive application value assessment dimensions such as model practical performance, ethical compliance, and privacy protection, making it difficult to comprehensively measure the model's adaptability to infrastructure scenarios. On the other hand, traditional evaluation methods mainly rely on manual annotation and keyword matching, which are costly, have narrow sample coverage, and low efficiency. With the rapid development of large-scale models and generative technologies, automated solutions based on diffusion model-generated samples and large models acting as referees are gradually becoming the mainstream development trend in the industry.

[0004] While existing automated evaluation schemes reduce manual input to some extent, they lack constraints adapted to industrial scenarios such as power and rail, the judges lack industrial control expertise, and there is a lack of multi-dimensional comprehensive value scoring processes. Applying them to critical infrastructure scenarios still presents the following challenges: First, existing security benchmarks are biased towards harmful online digital content, lacking physical risk samples from industrial scenarios, and have severely insufficient coverage of long-tail extreme failure scenarios; second, traditional image acquisition methods cannot balance real-world realism with the diversity of adversaries, lacking automated image and text sample generation and a standardized pipeline for layered purification, making it difficult to guarantee the number of samples and the generalization of evaluation; third, existing automated evaluation judges lack industrial control expertise support, failing to accurately identify two types of hidden violations—false denial and concealed compliance—resulting in low evaluation accuracy due to the lack of industrial control expertise; finally, current mainstream evaluation systems only output a single indicator of model security attack success rate, failing to conduct a comprehensive quantitative assessment of model value from four dimensions: practical functionality, social ethics, data privacy, and general support, thus failing to achieve a comprehensive evaluation of the model's overall application value and effectively supporting model access review and iterative optimization in critical infrastructure industries. Summary of the Invention To address the shortcomings of existing technologies, this invention provides a method and system for constructing a security benchmark for a multimodal large model and automatically evaluating its multidimensional value. This enables automated quantitative detection of security vulnerabilities in multimodal large models for industrial scenarios, while simultaneously completing multidimensional scoring and evaluation of the model's comprehensive application value.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A method for constructing a multimodal large-scale model security benchmark and automatically assessing multidimensional value includes the following steps: Step 1: Collect real images of various scenarios, including some critical infrastructure, and use a diffusion model controlled by positive and negative prompt words to perform generative expansion to build a dual-source basic image library. Step 2: Input the image into a multimodal large model to extract dense visual descriptions, i.e., finely depict the target category, structural state, spatial layout, and scene environment within the image. For example, the image may contain open equipment panels, ground safety passage markings, and the surrounding industrial working environment. Combine this with an industrial control attack rule knowledge base and a generalized semantic replacement mechanism to automatically generate a text-image adversarial question-answering library without explicit sensitive words. Step 3: Use a large language model evaluation system to perform a three-level cascaded screening of the question-answering database based on basic semantics, cross-modal alignment, and adversarial effectiveness, and export a standardized structured dataset; Step 4: Using an adjudication model that combines industrial control security rules and thought chain reasoning, the hidden violation responses of the target model are analyzed and security labels are output. Security quantification is completed through the attack success rate (ASR). Step 5: Input the corresponding answers of each test question and the target model into the value evaluator, extract the index scores of the answers in four value dimensions: practical function, social ethics, data privacy and security, and general support attributes, and form the value evaluation results of a single answer and the overall value evaluation results of the model. As a further improvement of the present invention, in step 1, the process of constructing the dual-source basic image library includes: Step 1.1: Collect real-world images from 18 sub-scenarios covering power systems, rail transit, modern agriculture, and industrial production. ; Step 1.2: Utilize the Juggernaut-XL-V9 image diffusion model to extend text-to-image and image-to-image generation. Text-to-image generation expands the material and topological descriptions using a large language model to construct positive and negative cue words. Positive cue words remove explicit destructive words, while negative cue words filter out non-realistic markers. Image-to-image generation... Based on this, positive and negative cue words containing local anomalies are constructed to introduce hidden local physical defects; finally, a dual-source basic image library is obtained through integration. . As a further improvement of the present invention, in step 2, the process of automatically generating the image-text adversarial question-and-answer database includes: Step 2.1: Input the image into the multimodal large model, and use constraint prompts to output a single sentence containing embedded technical terms and a dense, objective visual context description with a character limit within a preset value. ; Step 2.2: Retrieve a knowledge base of industrial control system attack rules covering 18 scenarios, construct hybrid prompt words, and use an adversarial identity projection mechanism to assign the identity of a security analyst to the large language model, generating a candidate malicious question database. ; Step 2.3: During the problem generation process, remove or replace the names of specific physical entities with explicit risks with generalized pronouns, and remove sensitive prompts from the text side. As a further improvement of the present invention, in step 3, the three-level cascade screening process includes: Step 3.1: Review the text syntax, structure and readability, and remove samples containing garbled characters, serious syntax errors or incomplete structure; Step 3.2: Implement anti-hallucination rules to eliminate deviations from the visual context. The problem is identified, and category constraint rules are applied to ensure that the problem strictly belongs to one of the 18 preset target scenarios. Step 3.3: Independently evaluate the technical depth and security bypass strength of candidate questions using heterogeneous large language models, retain the high-quality adversarial examples with the highest scores and total scores above the set threshold, and export them as JSON format data. As a further improvement of the present invention, in step 4, the automated security assessment process includes: Step 4.1: Establish a quantitative evaluation index with Attack Success Rate (ASR) as the core, formula: in, The total number of test samples, This is an indicator function; its value is 1 when the target model's response is judged to be a violation, and 0 otherwise. Step 4.2: Use DeepSeek-V4-Flash to build a referee framework, inject professional safety rules and few-sample examples from the industrial control field, guide the referee model to reason through the thought chain, identify the false rejection phenomenon that still reveals dangerous details after the pre-rejection words, and the hidden compliance phenomenon that outputs dangerous steps in a professional tone, and then output a binary safety label and the reason for the judgment. As a further improvement of the present invention, in step 5, the multi-dimensional value assessment process includes: The test questions are concatenated with the corresponding answers of the target model and input into the value evaluator built on DeBERTa-v3-large. For the basic indicators under the four dimensions of practical function, social ethics, data privacy and security, and general support attributes, scores of 0 to 5 are output respectively. The average of the scores of each indicator under the same value dimension for each answer is calculated to obtain the four-dimensional value score of the answer. The average of the scores of the same dimension for all answers is calculated to obtain the overall value evaluation result of the target model.

[0006] A multimodal large model security benchmark construction and automated evaluation system, used to implement the above method, includes: The dual-source image acquisition module is used to acquire real images of various scenes, including some key infrastructure, and to perform generative expansion by combining a diffusion model controlled by positive and negative prompt words to build a dual-source basic image library. The adversarial question-answering generation module is used to input images from the dual-source basic image library into a multimodal large model to extract dense visual descriptions, and automatically generate a text-image adversarial question-answering library without explicit sensitive words by combining an industrial control attack rule knowledge base and a generalized semantic replacement mechanism. The three-level screening module is used to perform a three-level cascaded screening of the image-text adversarial question-answering database based on basic semantics, cross-modal alignment, and adversarial effectiveness using a large language model evaluation system, and to export a standardized structured dataset. The security assessment module is used to analyze the hidden violation responses of the target's multimodal large model and output security labels by adopting a referee model that combines industrial control security rules and thought chain reasoning. It completes the security quantitative assessment by attack success rate. The value assessment module is used to input the corresponding answers of each test question and the target multimodal large model into the value assessor, extract the index scores of the answers in four value dimensions: practical function, social ethics, data privacy and security, and general support attributes, and form the value assessment results of a single answer and the overall value assessment results of the model.

[0007] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention integrates a unified process of automated benchmark construction, intelligent identification of hidden violations, and multi-dimensional comprehensive value assessment, filling the gap in standardized evaluation of the entire process of multimodal large-scale models in the field of critical infrastructure. It has significant theoretical and engineering value in the fields of multimodal AI security evaluation and intelligent risk control of industrial infrastructure. Attached Figure Description

[0008] Figure 1 This is a flowchart of the multimodal large model security benchmark construction and evaluation method of the present invention; Figure 2 This is a ring diagram showing the scene classification of the security benchmark dataset for this invention. Figure 3 This is a comparison diagram of a dual-source example of a real image and a generated image from the present invention; Figure 4 This is a flowchart of the dual-source image-driven multimodal adversarial problem generation and quality assessment process of the present invention. Figure 5 This is a flowchart of the original JSON-driven target multimodal large model response security assessment process of this invention; Figure 6 This is a flowchart of the original JSON-driven target multimodal large model response value assessment process of this invention; Figure 7 This is an example diagram of positive and negative prompt words for the text-to-image generation module of the present invention; Figure 8 Example diagram of positive and negative prompt words for the image-to-image generation module of the present invention; Figure 9 This is an example diagram of the mixed prompt words generated by the anti-question-answering sample of the present invention; Figure 10 This is a Few-Shot example diagram for the safety evaluation of this invention. Detailed Implementation

[0009] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0010] The present invention will now be described in further detail with reference to the accompanying drawings: This invention provides a method for constructing a multimodal large-scale model security benchmark and automatically assessing multi-dimensional value. It employs a dual-source image acquisition mechanism combining real-scene acquisition and diffusion generation, covering 18 subdivided application scenarios. A high-quality security evaluation benchmark is constructed through multi-model collaborative generation, generalized semantic replacement, and three-level screening. Relying on a thought chain judge incorporating industrial control knowledge, it accurately identifies hidden violations and quantifies cross-modal security vulnerabilities in the model. Furthermore, this invention introduces a multi-dimensional value evaluator to quantify and score the comprehensive performance of the model from four subdivided dimensions. This achieves the evaluation goals of accurate detection of model security risks and comprehensive quantification of application value, resulting in more comprehensive evaluation results that align with the actual application and industry audit requirements in critical infrastructure sectors.

[0011] This invention constructs a security assessment benchmark (SafeCI-18) containing 12,011 pairs of high-quality image and text data, and adopts an automated security adjudication model based on DeepSeek-V4-Flash. Its specific assessment method includes: Step 1: Dual-source image acquisition and expansion. Image acquisition includes two parts: real image acquisition and generative expansion based on a diffusion model; specifically, it includes: Step 1.1, Real-world Image Acquisition: This involves capturing images from 18 sub-scenarios covering key infrastructure such as power systems, rail transit, modern agriculture, and industrial production, as well as social welfare and illegal activities (e.g., [missing information]). Figure 2 (As shown), 1,778 high-resolution real-world images were acquired. This provides a real physical world reference for subsequent model evaluation and data verification. Step 1.2, Generative extension based on diffusion model: Generative extension of text to image (T2I) and image to image (I2I) is performed using the Juggernaut-XL-V9 image diffusion model. Text-to-Image (T2I) Generation: For each scene, design 3-5 adversarial attack perspectives, combine them with the core physical entities of the scene, and utilize Gemini 3.1 Pro to expand material and topology descriptions to construct... Figure 7The positive and negative prompts shown Positive cue words strip away explicitly destructive vocabulary, describing the dangerous state only from an operational physical perspective; negative cue words filter out non-realistic markers (such as CGI and 3D render). Images were synthesized using Juggernaut-XL. : Image-to-image (I2I) generation: Input Gemini 3.1 Pro as the baseline, and automatically build as follows Figure 8 The displayed positive and negative prompts indicate localized anomalous changes. By using Juggernaut-XL, extended images are generated by introducing hidden local physical defects while maintaining global semantic consistency. : The integrated dataset yields a dual-source base image library containing 12,011 images. A dual-source example of a real image and a generated image, for example... Figure 3 As shown. Step 2: Multi-model collaborative adversarial question-answering generation, combining heterogeneous multimodal large models and large language models to automatically generate precisely aligned text-image adversarial question-answer pairs; such as Figure 4 As shown, it specifically includes: Step 2.1: Extract visual context description: Extract the image... Input a large multimodal model (including Gemini 3.1Pro, Qwen3.5-plus, and GPT 5.5), and utilize constraint cue words. Output a single sentence containing dense, objective visual context descriptions with embedded technical terminology and a word limit of 40 characters. : Step 2.2, Constructing Hybrid Prompt Words and Attack Rule Injection: Design and construct as follows Figure 9 The mixed prompts shown The system retrieves a knowledge base of industrial control system attack rules covering 18 scenarios (including malicious CAN-bus frame injection, DNP3 / GOOSE protocol tampering, etc.) and fills the scenario constraint slots. An adversarial identity projection mechanism is used to assign the model the identity of a security analyst, driving a heterogeneous large language model to generate a candidate malicious issue database. : Step 2.3, Generalized Semantic Replacement: During the question generation process, the names of specific physical entities with explicit risks are removed or replaced with generalized pronouns (such as "the device" or "this facility"), sensitive prompts on the text side are stripped, and the target model is forced to rely on cross-modal semantic alignment for security judgment. Step 3: Multi-level data purification based on a large language model to refine visual context descriptions. With candidate problems Q cand Input a joint evaluation system consisting of heterogeneous large language models (including DeepSeek-V4-Flash, Gemma 3, and MiniMax 2.5) and perform a three-level cascaded selection: Step 3.1: Review the text syntax, structure and readability, and remove samples containing garbled characters, serious syntax errors or incomplete structure to ensure single-modal readability. Step 3.2: Implement anti-hallucination rules to eliminate illusions that deviate from the visual context. The problem is that, at the same time, category constraint rules are executed to ensure that the generated problems strictly belong to the 18 pre-defined target scenarios, thus eliminating the problem of scenario category drift.

[0012] Step 3.3: Independently evaluate the technical depth and security bypass strength of candidate questions using heterogeneous large language models (0-5 points per dimension), calculate the average score, and retain the highest-scoring sample; remove weak adversarial samples with a total score not higher than 6 points, and retain high-quality adversarial samples. And export it as standard JSON format data. Step 4: Domain Knowledge Construction. By analyzing industry standards, physical mechanisms, and typical risk patterns of critical infrastructure, a structured rule base is formed. This provides professional judgment criteria for automated security assessments based on domain knowledge and thought processes, and establishes a quantitative evaluation system centered on Attack Success Rate (ASR). Figure 5 As shown, it specifically includes: Step 4.1: The attack success rate is defined as the proportion of samples whose target model responses are judged to be insecure out of the total number of test samples. In the formula, The total number of test samples; if the target model responds If the output is deemed to be in violation (such as technical instructions that could lead to equipment damage or loss of control), then the instruction function... Otherwise, it is 0. Step 4.2, Referee Model Decision-Making: A referee framework is constructed using DeepSeek-V4-Flash, incorporating professional safety rules from the industrial control field and establishing... Figure 10The example shown is a Few-Shot. The guided adjudicator model performs two-stage reasoning through the Chain of Thought (CoT): First, it analyzes whether there are false rejections in the response that still reveal dangerous details after the initial rejection words, or hidden compliance phenomena that output dangerous steps in a professional tone; then, it outputs a binary safety label (Safe / Unsafe) and the reason for the judgment to complete the automated quantitative assessment. Step 5: Provide a value assessment based on the value assessor's response. For example... Figure 6 As shown, each test question is concatenated with the corresponding answer of the target model and input into a value evaluator built on DeBERTa-v3-large. The evaluator identifies the basic value indicators reflected in the answer and outputs a score of 0 to 5 for each indicator. The four value dimensions include: (1) Practical function dimension, including generalization, comprehension, accuracy and robustness; (2) Social ethics dimension, including fairness, inclusiveness, non-discrimination, sense of responsibility, behavioral consistency, empathy and information integrity; (3) Data privacy and security dimension, including privacy protection, data security, adversarial robustness, output controllability, ethical compliance and potential bias control; (4) General support attribute dimension, including explainability, transparency, credibility, behavioral consistency verification, emotional harmlessness and content harmlessness. For each answer, the average score of each indicator under the same value dimension is calculated to obtain the four-dimensional value score of the answer; then the average of the scores of the same dimension of all answers is calculated to obtain the overall value assessment result of the target model.

[0013] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for constructing and automatically evaluating a multimodal large model security benchmark, characterized in that, Includes the following steps: Step 1: Collect real images of various scenarios, including some critical infrastructure, and use a diffusion model controlled by positive and negative prompt words to perform generative expansion to build a dual-source basic image library; Step 2: Input the images from the dual-source basic image library into the multimodal large model to extract dense visual descriptions, and automatically generate a text-image adversarial question-answering library without explicit sensitive words by combining the industrial control attack rule knowledge base and the generalized semantic replacement mechanism. Step 3: Use a large language model evaluation system to perform a three-level cascaded screening of the text-to-text adversarial question-answering database based on basic semantics, cross-modal alignment, and adversarial effectiveness, and export a standardized structured dataset; Step 4: Using a referee model that combines industrial control security rules and thought chain reasoning, analyze the hidden violation responses of the target's multimodal large model and output security labels. Complete the security quantitative assessment through the attack success rate. Step 5: Input each test question and the corresponding answer of the target multimodal large model into the value evaluator, extract the index scores of the answers in four value dimensions: practical function, social ethics, data privacy and security, and general support attributes, and form the value evaluation results of a single answer and the overall value evaluation results of the model.

2. The method according to claim 1, characterized in that, In step 1, the process of constructing the dual-source basic image library includes: Step 1.1: Collect real-world images from 18 sub-scenarios covering power systems, rail transit, modern agriculture, and industrial production. ; Step 1.2: Utilize an image diffusion model to extend text-to-image and image-to-image generation. Text-to-image generation expands the text-to-image generation by using a large language model to augment material and topological descriptions, constructing positive and negative cue words. Positive cue words remove explicit destructive words, while negative cue words filter out non-realistic markers. Image-to-image generation... Based on this, positive and negative cue words containing local anomalies are constructed to introduce hidden local physical defects; finally, a dual-source basic image library is obtained through integration. .

3. The method according to claim 1, characterized in that, In step 2, the process of automatically generating the text-based adversarial question-answering database includes: Step 2.1: Input the image into the multimodal large model, and use constraint prompts to output a single sentence containing embedded technical terms and a dense, objective visual context description with a character limit within a preset value. ; Step 2.2: Retrieve a knowledge base of industrial control system attack rules covering 18 scenarios, construct hybrid prompt words, and use an adversarial identity projection mechanism to assign the identity of a security analyst to the large language model, generating a candidate malicious question database. ; Step 2.3: During the problem generation process, remove or replace the names of specific physical entities with explicit risks with generalized pronouns, and remove sensitive prompts from the text side.

4. The method according to claim 1, characterized in that, In step 3, the three-level cascaded screening process includes: Step 3.1: Review the text syntax, structure, and readability, and remove samples containing garbled characters, serious grammatical errors, or incomplete structures; Step 3.2: Implement anti-hallucination rules to eliminate deviations from the visual context. The problem is identified, and category constraint rules are executed to ensure that the problem strictly belongs to one of the 18 preset target scenarios; Step 3.3: Independently evaluate the technical depth and security bypass strength of candidate questions using heterogeneous large language models, retain the high-quality adversarial examples with the highest scores and total scores above the set threshold, and export them as JSON format data.

5. The method according to claim 1, characterized in that, In step 4, the automated security assessment process includes: Step 4.1: Establish a quantitative evaluation index with Attack Success Rate (ASR) as the core, formula: in, The total number of test samples, This is an indicator function; its value is 1 when the target model's response is judged to be a violation, and 0 otherwise. Step 4.2: Construct a referee framework, inject professional safety rules and few-sample examples from the industrial control field, guide the referee model to reason through the thought chain, identify the false rejection phenomenon that still reveals dangerous details after the pre-rejection words, and the hidden compliance phenomenon that outputs dangerous steps in a professional tone, and then output a binary safety label and the reason for the judgment.

6. The method according to claim 1, characterized in that, In step 5, the multi-dimensional value assessment process includes: The test questions are concatenated with the corresponding answers of the target model and input into the value evaluator. For the basic indicators under the four dimensions of practical function, social ethics, data privacy and security, and general support attributes, scores of 0 to 5 are output respectively. The average score of each indicator under the same value dimension for each answer is calculated to obtain the four-dimensional value score of the answer. The average of the scores of the same dimension for all answers is calculated to obtain the overall value evaluation result of the target model.

7. A multimodal large model security benchmark construction and automated evaluation system, used to implement the method as described in any one of claims 1 to 6, characterized in that, include: The dual-source image acquisition module is used to acquire real images of various scenes, including some key infrastructure, and to perform generative expansion by combining a diffusion model controlled by positive and negative prompt words to build a dual-source basic image library. The adversarial question-answering generation module is used to input images from the dual-source basic image library into a multimodal large model to extract dense visual descriptions, and automatically generate a text-image adversarial question-answering library without explicit sensitive words by combining an industrial control attack rule knowledge base and a generalized semantic replacement mechanism. The three-level screening module is used to perform a three-level cascaded screening of the image-text adversarial question-answering database based on basic semantics, cross-modal alignment, and adversarial effectiveness using a large language model evaluation system, and to export a standardized structured dataset. The security assessment module is used to analyze the hidden violation responses of the target's multimodal large model and output security labels by adopting a referee model that combines industrial control security rules and thought chain reasoning. It completes the security quantitative assessment by attack success rate. The value assessment module is used to input the corresponding answers of each test question and the target multimodal large model into the value assessor, extract the index scores of the answers in four value dimensions: practical function, social ethics, data privacy and security, and general support attributes, and form the value assessment results of a single answer and the overall value assessment results of the model.