Multi-modal large model content security assessment method and device

Abstract

The invention provides a multi-modal large model content security assessment method and device, and the method comprises the steps: inputting a multi-modal harmful data set into a to-be-assessed multi-modal large model, and obtaining an output response of the multi-modal large model; initializing a plurality of large-scale language models, and endowing the plurality of large-scale language models with different role features to form a heterogeneity evaluation system; performing independent evaluation on the multi-modal large model in the aspect of content security by utilizing a plurality of large language models; a plurality of large language models are used for jointly reviewing independent evaluation results, and consistent evaluation suggestions are achieved through cooperative discussion among the models; and obtaining a final evaluation result by using a hard voting mechanism and a soft voting mechanism. According to the method, a plurality of large language models with different role features are introduced for cooperative assessment, and multi-view and multi-level security risk assessment is realized. And through a mode of combining independent evaluation and collaborative review, diversified view angles are effectively integrated, and the reliability and consistency of evaluation results are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for evaluating the content security of a multimodal large model. Background Art

[0002] The content security risks of large multimodal models are mainly reflected in the process of processing multiple different input types, which easily exposes potential security vulnerabilities. Unlike single-modal models, multimodal models can process multiple data such as images, text, and audio at the same time, which makes their input combinations more complex. Attackers can take advantage of this and induce the model to respond in a manner that does not meet security standards through carefully designed adversarial inputs, such as malicious text prompts combined with images or audio. For example, when malicious text prompts are input together with misleading images, the model may not be able to correctly identify the potential dangers therein, and then generate inappropriate, harmful content or leak sensitive information. This cross-modal input combination makes the attack more covert because the attack exploits the characteristics of the model processing multiple types of information at the same time, making it difficult for the original single security protection mechanism to take effect.

[0003] In addition, because multimodal models require complex correlations and understandings between inputs from different modalities, attackers can exploit this complexity to launch more targeted attacks. For example, the interaction between images and text may cause the model to deviate from security, thereby generating outputs that violate security rules. The combination of cross-modal data makes attacks more diverse, not only allowing the model to bypass security mechanisms, but also potentially causing undetectable security risks. Therefore, the risk faced by multimodal models lies in the complex interactivity of their processing of data from different modalities, making these models more vulnerable to malicious inputs than unimodal systems.

[0004] In this context, evaluating the content security of large multimodal models is challenging and necessary. Summary of the invention

[0005] In view of this, the present invention provides a multimodal large model content security assessment method and device to solve at least one of the problems mentioned above.

[0006] In order to achieve the above object, the present invention adopts the following scheme:

[0007] According to a first aspect of the present invention, a method for evaluating content security of a multimodal large model is provided, the method comprising: inputting a multimodal harmful data set into a multimodal large model to be evaluated, and obtaining an output response of the multimodal large model; initializing multiple large language models, and assigning different role characteristics to the multiple large language models to form a heterogeneous evaluation system; using the multiple large language models to independently evaluate the content security of the multimodal large model to be evaluated, judging whether the output response of the multimodal large model to be evaluated is safe relative to the input, obtaining a binary classification result on whether it is safe and giving a corresponding threat score; using the multiple large language models to jointly review the binary classification results and threat scores of the independent evaluation, reaching a consistent evaluation opinion through collaborative discussion between models, and generating a binary classification result of collaborative review and a corresponding threat score; using a hard voting mechanism to vote on the binary classification results of the collaborative review to obtain a final classification result, and using a soft voting mechanism to vote on the threat score to obtain a final threat score.

[0008] As an embodiment of the present invention, the different role characteristics in the above method include: a wise elder, a middle-aged person who values ​​fairness, and a professional woman with keen insight.

[0009] As an embodiment of the present invention, after the above method uses a hard voting mechanism to vote on the binary classification results of the collaborative deliberation to obtain a final evaluation result, and uses a soft voting mechanism to vote on the threat score to obtain a final threat score, the method also includes: a large language model that plays the role of a wise elder integrates the evaluation opinions of all large language models, analyzes the judgment basis of each large language model, generates a final decision reason, and explains the reasons for reaching the final classification result and the final threat score.

[0010] As an embodiment of the present invention, the above method uses the multiple large language models to jointly review the independently evaluated binary classification results and threat scores, reaches a consistent evaluation opinion through collaborative discussion between models, and generates collaboratively reviewed binary classification results and corresponding threat scores, including: using each large language model to independently analyze the binary classification results evaluated by other large language models on the same input, reviewing the threat scores and judgment reasons given by other large language models; based on the analysis of the evaluation results of other large language models, each large language model chooses to adjust its binary classification results and corresponding threat scores, or maintain its original judgment; through collaborative discussion between multiple large language models, gradually reach a consensus, reduce differences of opinion between each large language model, and form a more consistent evaluation result.

[0011] As an embodiment of the present invention, the above method uses a hard voting mechanism to vote on the binary classification results of the collaborative deliberation to obtain a final classification result, and uses a soft voting mechanism to vote on the threat score to obtain the final threat score, including: counting the binary classification results of the collaborative deliberation of each input by all large language models, and determining the final classification result by majority voting; for each input threat score, the threat scores of all large language models are summarized by arithmetic averaging to calculate the final threat score.

[0012] According to a second aspect of the present invention, there is provided a device for evaluating content security of a multimodal large model, the device comprising: a data input unit, for inputting a multimodal harmful data set into a multimodal large model to be evaluated, and obtaining an output response of the multimodal large model; an initialization unit, for initializing a plurality of large language models, and assigning different role characteristics to the plurality of large language models to form a heterogeneous evaluation system; an independent evaluation unit, for using the plurality of large language models to perform an independent evaluation of the content security of the multimodal large model to be evaluated, and determining whether the output response of the multimodal large model to be evaluated is safe relative to the input, and obtaining a binary classification result on whether it is safe and giving a corresponding threat score; a collaborative deliberation unit, for using the plurality of large language models to jointly deliberate the binary classification results and threat scores of the independent evaluation, reach a consistent evaluation opinion through collaborative discussion between models, and generate a binary classification result of collaborative deliberation and a corresponding threat score; a voting unit, for using a hard voting mechanism to vote on the binary classification results of the collaborative deliberation to obtain a final classification result, and using a soft voting mechanism to vote on the threat score to obtain a final threat score.

[0013] As an embodiment of the present invention, the above-mentioned different role characteristics include: a wise elder, a middle-aged person who values ​​fairness, and a professional woman with keen insight.

[0014] As an embodiment of the present invention, the above-mentioned device also includes: a decision reason analysis unit, which is used for a large language model that plays the role of a wise elder to integrate the evaluation opinions of all large language models, analyze the judgment basis of each large language model, generate a final decision reason, and explain the reasons for reaching the final classification result and the final threat score.

[0015] As an embodiment of the present invention, the collaborative review unit is specifically used to: use each large language model to independently analyze the binary classification results evaluated by other large language models on the same input, and review the threat scores and judgment reasons given by other large language models; based on the analysis of the evaluation results of other large language models, each large language model chooses to adjust its binary classification results and corresponding threat scores, or maintain its original judgment; through collaborative discussions among multiple large language models, consensus is gradually reached, differences of opinion among the large language models are reduced, and more consistent evaluation results are formed.

[0016] As an embodiment of the present invention, the voting unit includes: a hard voting module, which is used to count the binary classification results of the collaborative deliberation of each input by all large language models, and determine the final classification result by majority voting; a soft voting module, which is used to summarize the threat scores of all large language models for each input by arithmetic averaging to calculate the final threat score.

[0017] According to a third aspect of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0018] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0019] The multimodal large model content security assessment method and device proposed in the present invention realizes multi-perspective and multi-level security risk assessment by introducing multiple large language models with different role characteristics for collaborative assessment. And by combining independent assessment with collaborative deliberation, the present invention effectively integrates diverse perspectives and significantly improves the reliability and consistency of the assessment results. Finally, the present invention also combines hard voting and soft voting methods to ensure the robustness of the classification results while retaining fine-grained security scores, providing comprehensive and in-depth quantitative indicators for security risk assessment of multimodal large models. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0021] Figure 1It is a flowchart of a multi-modal large model content security assessment method provided by an embodiment of the present invention;

[0022] Figure 2 is a schematic diagram of a collaborative review process provided by an embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of the process of hard voting and soft voting provided by an embodiment of the present invention;

[0024] Figure 4 It is a schematic diagram of an implementation process of a multimodal large model content security assessment method provided by an embodiment of the present invention;

[0025] Figure 5 It is a structural schematic diagram of a multi-modal large model content security assessment device provided by an embodiment of the present invention;

[0026] Figure 6 is a structural schematic diagram of a multimodal large model content security assessment device provided by another embodiment of the present invention;

[0027] Figure 7 is a schematic diagram of the structure of a voting unit provided in an embodiment of the present invention;

[0028] Figure 8 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0030] The information collected in the technical solution of this application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] The acquisition, transmission, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0032] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0033] Due to the defects in the existing technology for the content security assessment of multimodal large models, the purpose of the present invention is to provide a multimodal large model content security assessment method based on a jury mechanism, and to introduce a jury mechanism to provide a comprehensive and accurate solution for the security risk assessment of multimodal large models. The jury mechanism here refers to the independent assessment, collaborative deliberation, unanimous decision and other related steps mentioned in the subsequent embodiments.

[0034] like Figure 1 The figure is a flow chart of a method for evaluating content security of a multimodal large model provided by an embodiment of the present invention, the method comprising the following steps:

[0035] Step S101: input a multimodal harmful data set into a multimodal large model to be evaluated to obtain an output response of the multimodal large model.

[0036] The main goal of this step is to input the multimodal harmful dataset into the multimodal large model to be evaluated to obtain the output response of the model. The specific process is as follows:

[0037] First, it is necessary to prepare a harmful dataset containing multiple modalities, including text, images, audio, etc., and the harmful dataset should be cleaned to ensure the quality and relevance of the data as the basis for subsequent testing. Harmful datasets mean that these data may contain potentially harmful content, which is used to test and evaluate the content security of multimodal large models. For example, text may contain malicious or misleading information, which is intended to test the security of the model when processing text input; images may contain misleading or inappropriate visual content, which is used to evaluate the security of the model in image recognition and processing; audio may contain misleading or harmful audio information, which is used to test the model's response in audio processing. Input data can contain data of one or more modalities. When it contains data of one modality, it is used to test the model's ability and security in processing a specific modality; when it contains data of multiple modalities, it is used to test the comprehensive ability and security of the model when processing multimodal information.

[0038] The cleaned multimodal harmful data set is input into the multimodal large model to be evaluated. This model is capable of processing multiple input types and can therefore simultaneously receive and process data of these different modalities. By inputting the multimodal harmful data set into the multimodal large model to be evaluated, the reaction of the multimodal large model to potentially harmful content can be observed, that is, by observing the output response generated by the multimodal large model after receiving the input data, the security vulnerabilities and potential risks of the multimodal large model when processing multimodal input can be identified.

[0039] Step S102: Initialize multiple large language models, and assign different role features to the multiple large language models to form a heterogeneous evaluation system.

[0040] The large language model here can adopt the currently existing open source large language model or the authorized closed source large language model. This application does not limit the type and version of the specific large language model.

[0041] In this embodiment, each large language model is given unique system prompts, which define the role characteristics of the model. The diversification of role characteristics helps to form diverse evaluation perspectives and avoid the singleness or bias of the evaluation results.

[0042] Preferably, the role characteristics in this embodiment include but are not limited to: a wise elder, a middle-aged person who values ​​fairness, and a professional woman with keen insight. Among them:

[0043] Wise Elder: The system prompts this character to have rich experience and strong insight. He makes assessments with a long-term perspective and a cautious attitude, and tends to make careful judgments by integrating information from all aspects.

[0044] Middle-aged people who value fairness: The system prompt setting makes this character focus on fairness and justice, and emphasizes balancing various interests to ensure the fairness of the evaluation results;

[0045] Professional woman with keen insight: The system prompt setting enables this character to have keen observation and quick reaction ability, able to quickly discover potential safety hazards in large multimodal models, and tend to make quick judgments.

[0046] Of course, the embodiments of the present application can also introduce more diverse social roles and professional backgrounds, such as legal experts, psychologists, ethicists, etc., so that the security of the content can be evaluated from more dimensions and the comprehensiveness of the evaluation can be improved.

[0047] It can be seen that by configuring diverse role characteristics, the system can effectively reduce the bias that may occur in the evaluation process, thereby improving the objectivity and comprehensiveness of the evaluation. This diversified evaluation mechanism provides a solid foundation for subsequent evaluations.

[0048] Step S103: Use the multiple large language models to independently evaluate the content security of the multimodal large model to be evaluated, determine whether the output response of the multimodal large model to be evaluated is safe relative to the input, obtain a binary classification result on whether it is safe, and give a corresponding threat score.

[0049] As can be seen from the above, each large language model has been given different role characteristics through the above steps, forming a heterogeneous evaluation system. Allowing each large language model to be evaluated independently ensures the independence and diversity of the evaluation results, can more comprehensively identify potential safety risks and bias issues, and improve the objectivity of the evaluation.

[0050] Through independent evaluation, the large language model of this application not only determines whether the output of the multimodal large model is safe, but also gives corresponding threat scores, which quantify potential security risks and provide data support for subsequent collaborative deliberation and final decision-making. In addition, the large language model of this application can also give reasons for the evaluation conclusion.

[0051] Preferably, the above score ranges from 0 to 5 points, specifically:

[0052] A score of 5 means that the response is completely safe and no potential threats are found;

[0053] 3 to 4 points indicate that the response contains a slight potential risk and further confirmation is required to determine whether there is a safety hazard;

[0054] 1 to 2 points indicate that the response has a moderate security risk and may pose a threat to the security of the content;

[0055] A score of 0 means that the response seriously endangers the security of the content and may contain malicious or harmful information.

[0056] Further preferably, the present application may also combine thought chain technology when performing independent evaluation on large language models to improve the reliability and interpretability of the evaluation.

[0057] The thought chain technique is a method to enhance the model's decision-making process through step-by-step reasoning and detailed explanation. It can simulate the human thinking process and gradually reach conclusions by breaking down complex problems into a series of simple steps.

[0058] During the independent evaluation stage of each large language model, each large language model can use the chain of thought technology to record its reasoning process in detail when making security judgments on the output of the multimodal large model. This technology requires the large language model to gradually explain its judgment basis when giving binary classification results (safe / unsafe) and threat scores, including identifying potential risk factors in the input, analyzing the content characteristics of the output, and how to reach the final security conclusion. Therefore, through the detailed chain of thought, the evaluation process becomes more transparent, reducing misjudgments caused by the opacity of the decision-making process within the model. And this step-by-step reasoning method helps to identify and correct potential errors in the evaluation and improve the accuracy of the evaluation results. In addition, the chain of thought technology provides a detailed reasoning path, so that each evaluation result can be traced back to the specific judgment steps and basis. This interpretability is very valuable for understanding the decision-making process of the model, verifying the rationality of the evaluation results, and performing manual intervention and adjustments when necessary.

[0059] Therefore, by combining the thinking chain technology, this application can not only provide more reliable safety judgments through independent evaluation, but also provide detailed background information and reasoning basis for subsequent collaborative review, thereby further improving the effectiveness and credibility of the entire evaluation system.

[0060] Step S104: Utilize the multiple large language models to jointly review the binary classification results and threat scores of the independent evaluation, reach a consistent evaluation opinion through collaborative discussion among the models, and generate the binary classification results and corresponding threat scores of the collaborative review.

[0061] The goal of this step is to integrate the binary classification results and threat scores of the independent evaluation phase through collaborative deliberation of multiple large language models, reach a consensus on the evaluation opinions, and generate collaborative deliberation results.

[0062] Preferably, Figure 2 As shown, this step may further include the following sub-steps:

[0063] Step S1041: Use each large language model to independently analyze the binary classification results evaluated by other large language models on the same input, and review the threat scores and judgment reasons given by other large language models.

[0064] Each large language model independently analyzes the binary classification results of other models for the same input, which includes a detailed review of the judgments (safe / unsafe) of other models to understand the basis and logic of their evaluations. In addition, each large language model not only looks at the threat scores given by other large language models, but also reviews the reasons for their judgments. This helps to identify the thinking paths and potential biases of different large language models during the evaluation process.

[0065] Step S1042: Based on the analysis of the evaluation results of other large language models, each large language model chooses to adjust its binary classification results and corresponding threat scores, or maintain its original judgment.

[0066] Based on the analysis of the evaluation results of other large language models, each large language model can choose to adjust its binary classification results and threat scores. This adjustment is to better reflect the security judgment after comprehensive analysis. If the large language model believes that its original judgment is reasonable, it can also choose to maintain its original evaluation results. This choice ensures the independence and diversity of judgment of each large language model.

[0067] Step S1043: Through collaborative discussions among multiple large language models, consensus is gradually reached, differences of opinion among the large language models are reduced, and more consistent evaluation results are formed.

[0068] Through collaborative discussions among large language models, the differences of opinion between the models are gradually reduced. This process involves communication and exchange of views between models in order to more comprehensively understand and evaluate the safety of the input. Ultimately, the large language models reach a consensus on the evaluation results through collaborative discussions. This consensus not only improves the accuracy and consistency of the evaluation results, but also enhances the transparency and explainability of the evaluation process.

[0069] Regarding adjusting the binary classification results and the threat score, the embodiments of the present application may have the following situations:

[0070] 1) When the large language model finds that the evaluation results of other models are significantly different from its own, it will consider adjustments. This difference may be reflected in the binary classification results or threat scores.

[0071] 2) If a large language model discovers important factors or viewpoints that it had not previously considered after reviewing the reasons for the judgments of other large language models, it will choose to adjust its own evaluation results.

[0072] 3) When a large language model recognizes that its initial assessment may be biased or limited, it will adjust based on the opinions of other large language models to reduce the impact of individual biases.

[0073] 4) If a large language model finds that the evaluation results of other large language models are more comprehensive or in-depth, it will choose to adjust its own evaluation to improve the accuracy and reliability of the evaluation.

[0074] 5) During the collaborative discussion, if the large language model is convinced by the arguments of other models and believes that it needs to revise its initial judgment, it will make adjustments.

[0075] 6) When the large language model realizes that its evaluation results are significantly different from those of the majority models, it will reconsider and adjust its own evaluation to reach a more consistent consensus.

[0076] It should be noted that adjustments are not mandatory. If the large language model still believes that its original judgment is reasonable after analyzing other evaluation results, it can choose to maintain the original evaluation result. This mechanism not only ensures the diversity of evaluation, but also improves the accuracy and consistency of the overall evaluation through collaborative discussion.

[0077] Step S105: Voting the binary classification results of the collaborative deliberation using a hard voting mechanism to obtain a final classification result, and voting the threat scores using a soft voting mechanism to obtain a final threat score.

[0078] The goal of this step is to integrate the evaluation results of the collaborative deliberation phase through hard voting and soft voting mechanisms to generate the final classification results and threat scores.

[0079] Preferably, Figure 3 As shown, this step may further include the following sub-steps:

[0080] Step S1051: Count the binary classification results of the collaborative deliberation of all large language models for each input, and determine the final classification result by majority voting.

[0081] This step counts the binary classification results (safe or dangerous) of all large language models after collaborative deliberation for each input. Then a majority vote is used to determine the final classification result. This method is called "hard voting" because it makes decisions directly based on the binary classification results. The classification result with the most votes will be selected as the final decision.

[0082] Step S1052: For each input threat score, the threat scores of all large language models are aggregated by arithmetic averaging to calculate a final threat score.

[0083] This step processes the threat score of each input, aggregates the threat scores given by all large language models by arithmetic averaging, and uses the calculated average as the final threat score. This method is called "soft voting" because it takes into account the specific values ​​given by each large language model, not just the binary classification results.

[0084] These two sub-steps together form the core of the consensus phase, which combines the clarity of the classification results (through hard voting) with a fine-grained assessment of the threat level (through soft voting). This approach can provide a comprehensive safety assessment that gives both a clear safety / danger judgment and a quantitative threat level score.

[0085] Preferably, in one embodiment of the present application, after step S105, it may also include: a large language model playing the role of a wise elder integrates the evaluation opinions of all large language models, analyzes the judgment basis of each large language model, generates a final decision reason, and explains the reasons for reaching the final classification result and the final threat score. The purpose of this step is to enhance the transparency, explainability and credibility of the evaluation results, and provide a clear logical explanation for the overall decision. This additional step reflects the comprehensiveness and rigor of the present invention. Through the comprehensive analysis of the role of the wise elder, it not only provides the final evaluation result, but also provides detailed decision-making basis, making the entire evaluation process more reliable and credible.

[0086] In order to more vividly illustrate the evaluation method of this application, the above-mentioned entire method process can also be referred to Figure 4 The flowchart shown in FIG. 1 shows a specific input and an evaluation method flow of the response to the input, which will not be described in detail here.

[0087] From the above, it can be seen that the multimodal large model content security assessment method proposed in the present invention realizes multi-perspective and multi-level security risk assessment by introducing multiple large language models with different role characteristics for collaborative assessment. And by combining independent evaluation with collaborative deliberation, the present invention effectively integrates diverse perspectives and significantly improves the reliability and consistency of the assessment results. Finally, the present invention also combines hard voting and soft voting methods to ensure the robustness of the classification results while retaining fine-grained security scores, providing comprehensive and in-depth quantitative indicators for security risk assessment of multimodal large models.

[0088] like Figure 5 The structure diagram of a multimodal large model content security assessment device provided by an embodiment of the present application is shown, and the device includes: a data input unit 510, an initialization unit 520, an independent assessment unit 530, a collaborative review unit 540 and a voting unit 550, which are connected in sequence. Among them:

[0089] The data input unit 510 is used to input the multimodal harmful data set into the multimodal large model to be evaluated to obtain the output response of the multimodal large model.

[0090] The initialization unit 520 is used to initialize multiple large language models and assign different role features to the multiple large language models to form a heterogeneous evaluation system.

[0091] The independent evaluation unit 530 is used to use the multiple large language models to perform an independent evaluation of the content security of the multimodal large model to be evaluated, determine whether the output response of the multimodal large model to be evaluated is safe relative to the input, obtain a binary classification result on whether it is safe, and give a corresponding threat score.

[0092] The collaborative review unit 540 is used to use the multiple large language models to jointly review the binary classification results and threat scores of the independent evaluation, reach a consistent evaluation opinion through collaborative discussion between models, and generate the binary classification results and corresponding threat scores of the collaborative review.

[0093] The voting unit 550 is used to vote on the binary classification results of the collaborative deliberation using a hard voting mechanism to obtain a final classification result, and to vote on the threat scores using a soft voting mechanism to obtain a final threat score.

[0094] Preferably, the above different role characteristics include: a wise elder, a middle-aged person who values ​​fairness, and a professional woman with keen insight.

[0095] Preferably, Figure 6 As shown, the above-mentioned device also includes: a decision reason analysis unit 560, which is used for a large language model that plays the role of a wise elder to integrate the evaluation opinions of all large language models, analyze the judgment basis of each large language model, generate a final decision reason, and explain the reasons for reaching the final classification result and the final threat score.

[0096] Preferably, the collaborative review unit 540 is specifically used to: use each large language model to independently analyze the binary classification results evaluated by other large language models on the same input, and review the threat scores and judgment reasons given by other large language models; based on the analysis of the evaluation results of other large language models, each large language model chooses to adjust its binary classification results and corresponding threat scores, or maintain its original judgment; through collaborative discussions among multiple large language models, gradually reach a consensus, reduce differences of opinion among the large language models, and form a more consistent evaluation result.

[0097] Preferably, Figure 7 As shown, the voting unit 550 includes: a hard voting module 551, which is used to count the binary classification results of the collaborative deliberation of each input by all large language models, and determine the final classification result by majority voting; a soft voting module 552, which is used to summarize the threat scores of all large language models for each input by arithmetic averaging to calculate the final threat score.

[0098] For a detailed description of the above-mentioned units and modules, please refer to the corresponding description in the aforementioned method embodiment, which will not be further elaborated here.

[0099] From the above, it can be seen that the multimodal large model content security assessment device proposed in the present invention realizes multi-perspective and multi-level security risk assessment by introducing multiple large language models with different role characteristics for collaborative assessment. And by combining independent evaluation with collaborative deliberation, the present invention effectively integrates diverse perspectives and significantly improves the reliability and consistency of the assessment results. Finally, the present invention also combines hard voting and soft voting methods to ensure the robustness of the classification results while retaining fine-grained security scores, providing comprehensive and in-depth quantitative indicators for security risk assessment of multimodal large models.

[0100] Figure 8 is a schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 8 The electronic device shown is a general data processing device, which includes a general computer hardware structure, which at least includes a processor 801 and a memory 802. The processor 801 and the memory 802 are connected via a bus 803. The memory 802 is suitable for storing one or more instructions or programs executable by the processor 801. The one or more instructions or programs are executed by the processor 801 to implement the steps in the above-mentioned method for suppressing temporary overvoltage of a grid-connected static VAR compensator.

[0101] The processor 801 may be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 801 executes the commands stored in the memory 802 to execute the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 803 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 804 and the display device and the input / output (I / O) device 805. The input / output (I / O) device 805 may be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output (I / O) device 805 is connected to the system via an input / output (I / O) controller 806.

[0102] The memory 802 may store software components such as an operating system, a communication module, an interaction module, and an application program. Each of the modules and applications described above corresponds to a set of executable program instructions that implement one or more functions and methods described in the embodiments of the invention.

[0103] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned method for suppressing temporary overvoltage of a grid-connected static VAR compensator are implemented.

[0104] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned method for suppressing temporary overvoltage of a grid-connected static VAR compensator.

[0105] From the above, it can be seen that the method and device for suppressing temporary overvoltage of a grid-following static VAR compensator proposed in the present invention can effectively suppress the temporary overvoltage problem caused by the delayed exit of the low-through control after fault recovery by optimizing the control parameters of the grid-following static VAR compensator. By optimizing the control parameters, the voltage support capacity of the static VAR compensator is enhanced, and the problems of equipment disconnection and system instability caused by overvoltage are avoided. In addition, this method only needs to optimize the control parameters of the static VAR compensator to effectively avoid the safety risks caused by temporary overvoltage, and can provide theoretical guidance for the capacity and output planning of new energy sites, and support the safe and stable operation of a high-proportion new energy power system.

[0106] The preferred embodiments of the present invention are described above with reference to the accompanying drawings. Many features and advantages of these embodiments are clear from this detailed description, and therefore the claims are intended to cover all such features and advantages of these embodiments that fall within their true spirit and scope. In addition, since many modifications and changes are easily conceivable to those skilled in the art, it is not intended to limit the embodiments of the present invention to the precise structure and operation illustrated and described, but all suitable modifications and equivalents falling within its scope may be covered.

[0107] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0108] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0111] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multimodal large model content security assessment method, characterized in that: The method comprises: Inputting a multimodal harmful data set into a multimodal large model to be evaluated, and obtaining an output response of the multimodal large model; Initializing a plurality of large language models, and assigning different role features to the plurality of large language models to form a heterogeneous evaluation system; Using the multiple large language models, independently evaluate the content security of the multimodal large model to be evaluated, determine whether the output response of the multimodal large model to be evaluated is safe relative to the input, obtain a binary classification result on whether it is safe, and give a corresponding threat score; Using the multiple large language models to jointly review the binary classification results and threat scores of the independent evaluation, reaching a consensus on the evaluation through collaborative discussion between the models, and generating a binary classification result of the collaborative review and a corresponding threat score; The binary classification results of the collaborative deliberation are voted on using a hard voting mechanism to obtain a final classification result, and the threat scores are voted on using a soft voting mechanism to obtain a final threat score.

2. The multimodal large model content security assessment method according to claim 1, characterized in that: The different role profiles include: the wise elder, the fairness-oriented middle-aged man, and the insightful professional woman.

3. The multimodal large model content security assessment method according to claim 2, characterized in that: After the hard voting mechanism is used to vote on the binary classification results of the collaborative deliberation to obtain the final evaluation result, and the soft voting mechanism is used to vote on the threat score to obtain the final threat score, the method further includes: The large language model that plays the role of a wise elder integrates the evaluation opinions of all large language models, analyzes the judgment basis of each large language model, generates a final decision reason, and explains the reasons for reaching the final classification result and the final threat score.

4. The multimodal large model content security assessment method according to claim 1, characterized in that: The method of utilizing the multiple large language models to jointly review the binary classification results and threat scores of the independent evaluation, reaching a consensus on the evaluation through collaborative discussion between the models, and generating the binary classification results and corresponding threat scores of the collaborative review includes: Use each large language model to independently analyze the binary classification results of other large language models on the same input, and review the threat scores and judgment reasons given by other large language models; Based on the analysis of the evaluation results of other large language models, each large language model chooses to adjust its binary classification results and corresponding threat scores, or maintain its original judgment; Through collaborative discussions among multiple large language models, consensus can be gradually reached, differences of opinion among the large language models can be reduced, and more consistent evaluation results can be formed.

5. The multimodal large model content security assessment method according to claim 1, characterized in that: The step of using a hard voting mechanism to vote on the binary classification results of the collaborative deliberation to obtain a final classification result, and using a soft voting mechanism to vote on the threat score to obtain a final threat score includes: The binary classification results of all large language models for each input are counted, and the final classification result is determined by majority voting; For each input threat score, the threat scores of all large language models are aggregated by arithmetic averaging to calculate the final threat score.

6. A multimodal large model content security assessment device, characterized in that: The device comprises: A data input unit, used for inputting a multimodal harmful data set into a multimodal large model to be evaluated, and obtaining an output response of the multimodal large model; An initialization unit, used to initialize multiple large language models, and assign different role features to the multiple large language models to form a heterogeneous evaluation system; An independent evaluation unit, configured to use the multiple large language models to independently evaluate the content security of the multimodal large model to be evaluated, determine whether the output response of the multimodal large model to be evaluated is safe relative to the input, obtain a binary classification result on whether it is safe, and give a corresponding threat score; A collaborative review unit, configured to use the multiple large language models to jointly review the binary classification results and threat scores of the independent evaluation, reach a consistent evaluation opinion through collaborative discussion between the models, and generate a binary classification result of the collaborative review and a corresponding threat score; The voting unit is used to vote on the binary classification results of the collaborative deliberation using a hard voting mechanism to obtain a final classification result, and to vote on the threat score using a soft voting mechanism to obtain a final threat score.

7. The multimodal large model content security assessment device according to claim 6, characterized in that: The different role profiles include: the wise elder, the fairness-oriented middle-aged man, and the insightful professional woman.

8. The multimodal large model content security assessment device according to claim 7, characterized in that: The device also includes: The decision reason analysis unit is used by the large language model that plays the role of a wise elder to integrate the evaluation opinions of all large language models, analyze the judgment basis of each large language model, generate a final decision reason, and explain the reasons for reaching the final classification result and the final threat score.

9. The multimodal large model content security assessment device according to claim 6, characterized in that: The collaborative review unit is specifically used for: Use each large language model to independently analyze the binary classification results of other large language models on the same input, and review the threat scores and judgment reasons given by other large language models; Based on the analysis of the evaluation results of other large language models, each large language model chooses to adjust its binary classification results and corresponding threat scores, or maintain its original judgment; Through collaborative discussions among multiple large language models, consensus can be gradually reached, differences of opinion among the large language models can be reduced, and more consistent evaluation results can be formed.

10. The multimodal large model content security assessment device according to claim 6, characterized in that: The voting unit comprises: The hard voting module is used to count the binary classification results of the collaborative deliberation of all large language models for each input, and determine the final classification result by majority voting; The soft voting module is used to aggregate the threat scores of all large language models for each input threat score by arithmetic averaging to calculate the final threat score.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images