Safety decision-making method and device based on multi-modal large language model

Abstract

The invention discloses a security decision-making method and device based on a multi-modal large language model. The method comprises the following steps: acquiring a target question which is input by a target object and is composed of information of multiple modes; the safety decision model is used for analyzing the target problem to obtain a logical reasoning link, and the reasoning result comprises a problem solution generated when the information of the multiple modes is respectively safe and is still safe after being combined, or risk prompt information generated when the information of the multiple modes is respectively safe and has a safety risk after being combined. The security decision model is obtained by performing optimization training on a multi-modal big language model by using a plurality of groups of multi-modal problems which are formed by a plurality of pieces of security single-modal information and have security risks as sample data; and outputting a target logical reasoning link or a reasoning result to the target object. According to the method and the device, the technical problem that the hidden reasoning risk possibly generated by information combination of multiple modes is easily ignored when a traditional model analyzes the multi-mode information is solved.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and more specifically, to a security decision-making method and apparatus based on a multimodal large language model. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technology, especially the widespread application of MLLM (Multimodal Large Language Model), while enjoying the convenience and intelligent services it brings, people are also increasingly concerned about its security and controllability. MLLM can understand and process complex information from different modalities (such as images, audio, and text), thus showing great potential in many fields, such as intelligent customer service, content creation, and educational tutoring. However, this ability to process multimodal information also brings new challenges—the problem of secure alignment of cross-modal inputs.

[0003] Traditional unimodal language models have achieved relatively mature research and practice in text security, with significant results in areas such as filtering inappropriate content and avoiding misleading information. However, the situation becomes much more complex when models encounter inputs that integrate multiple modalities. Even if individual unimodal pieces of information appear harmless on the surface, their semantic combination and interaction can trigger potential security risks and harmful consequences. This phenomenon is defined in technical literature as implicit inference risk. For example, an image showing a chemical bottle combined with text describing a formula for mixing different chemicals, while harmless on its own, may induce the model to output a suggestion that generates a harmful chemical reaction. Current MLLM models struggle to avoid this implicit inference risk when dealing with multimodal problems.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a security decision-making method and apparatus based on a multimodal large language model, which at least solves the technical problem that traditional models tend to overlook the implicit reasoning risks that may arise from the combination of information from multiple modalities when analyzing multimodal information.

[0006] According to one aspect of the embodiments of this application, a security decision-making method based on a multimodal large language model is provided, comprising: acquiring a target problem input by a target object, wherein the target problem is composed of information from multiple modalities; analyzing the target problem using a security decision-making model to obtain a target logical reasoning link, wherein the reasoning result of the target logical reasoning link includes: a problem solution generated when the information from each of the multiple modalities is secure and remains secure when combined, or a risk warning message generated when the information from each of the multiple modalities is secure but presents a security risk when combined; the security decision-making model is obtained by optimizing and training a pre-trained multimodal large language model using target sample data, wherein the target sample data includes: a multimodal problem with security risks composed of multiple secure single-modal information; and outputting the target logical reasoning link or reasoning result to the target object.

[0007] Optionally, after obtaining the target question input by the target object, security checks are performed on each monomodal information in the target question; if a monomodal information with security risks is detected, an abnormal input prompt is output to the target object, wherein the abnormal input prompt is used to indicate to the target object that the input monomodal information has security risks and cannot be used for reasoning and analysis.

[0008] Optionally, the training process of the security decision model includes: acquiring a pre-trained multimodal large language model, wherein the multimodal large language model is used to perform logical reasoning on the input multimodal problem to obtain a solution; acquiring multiple sets of target sample data and corresponding sample labels, wherein the sample labels include: a pre-defined standard logical reasoning link for the multimodal problem in the corresponding target sample data; training the multimodal large language model using multiple sets of target sample data and corresponding sample labels, updating the model parameters of the multimodal large language model, and obtaining the security decision model.

[0009] Optionally, in the target sample data, the unimodal information includes images and text. Multiple sets of target sample data and corresponding sample labels are acquired, including: acquiring multiple secure images; for each secure image, using a pre-trained generative agent to generate multiple text questions that are secure on their own but pose a security risk when combined with the secure image, and combining each text question with the secure image to form a text-image pair as a set of target sample data; for each text-image pair, using a pre-trained inference agent to perform inference analysis on the text-image pair, generating multiple logical inference links, wherein the logical inference link includes: the probability that the text-image pair poses a security risk; determining the logical inference link whose corresponding probability exceeds a preset probability threshold as a standard logical inference link, and using the standard logical inference link as the sample label for the corresponding text-image pair.

[0010] Optionally, after generating multiple text questions using a generative agent, a pre-trained verification agent is used to verify each text question, eliminating unsafe and redundant text questions. Redundant text questions include at least one of the following: text questions with a text similarity higher than a preset similarity threshold with other text questions, or text questions that only describe the image content of the corresponding safe image. After generating multiple logical reasoning links using a reasoning agent, a verification agent is used to verify the reasoning basis in each logical reasoning link, eliminating logical reasoning links whose reasoning basis does not conform to preset specifications, and supplementing reasoning basis that conforms to preset specifications but lacks key information with key information.

[0011] Optionally, multiple sets of target sample data and corresponding sample labels are sent to the manual review module for review; the review results of the manual review module for each set of target sample data and corresponding sample labels are obtained, and the target sample data and corresponding sample labels that fail the review are removed.

[0012] Optionally, the multimodal large language model is trained using multiple sets of target sample data and corresponding sample labels, and the model parameters of the multimodal large language model are updated. This includes: inputting multiple sets of target sample data into the multimodal large language model multiple times to obtain multiple logical inference links output by the multimodal large language model for each target sample data; constructing an alignment loss function based on the differences between the multiple logical inference links; constructing a reference loss function based on the differences between each logical inference link and the standard logical inference link in the corresponding sample label; and weighting and summing the alignment loss function and the reference loss function according to preset weight coefficients to obtain the target loss function; and adjusting the model parameters of the multimodal large language model by minimizing the target loss function.

[0013] According to another aspect of the embodiments of this application, a security decision-making device based on a multimodal large language model is also provided, comprising: an acquisition module for acquiring a target problem input by a target object, wherein the target problem is composed of information from multiple modalities; an analysis module for analyzing the target problem using a security decision-making model to obtain a target logical reasoning link, wherein the reasoning result of the target logical reasoning link includes: a problem solution generated when the information from each of the multiple modalities is secure and remains secure when combined, or a risk warning message generated when the information from each of the multiple modalities is secure but presents a security risk when combined; the security decision-making model is obtained by optimizing and training a pre-trained multimodal large language model using target sample data, wherein the target sample data includes: a multimodal problem with security risks composed of multiple secure single-modal information; and an output module for outputting the target logical reasoning link or reasoning result to the target object.

[0014] According to another aspect of the embodiments of this application, a computer program product is also provided, the computer program product comprising: a computer program, wherein the computer program, when executed by a processor, implements the above-described security decision-making method based on a multimodal large language model.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, the electronic device including: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described security decision-making method based on a multimodal large language model through the computer program.

[0016] In this embodiment, a multimodal problem with security risks, consisting of multiple secure single-modal information, is constructed as sample data. A standard logical reasoning link including risky reasoning arguments is set as sample label. These sample data and sample labels are used to optimize and train a conventional multimodal large language model based on long-chain reasoning. This enables the model to identify and avoid security risks that may be caused by the combination of single-modal information during the reasoning process, rather than directly outputting harmful problem solutions. This solution effectively solves the technical problem that traditional models tend to ignore the implicit reasoning risks that may be generated by the combination of multiple modal information when analyzing multimodal information. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a flowchart illustrating an optional security decision-making method based on a multimodal large language model according to an embodiment of this application;

[0019] Figure 2 This is a schematic diagram of an optional model training process according to an embodiment of this application;

[0020] Figure 3a This is a schematic diagram of an optional model reasoning process according to an embodiment of this application;

[0021] Figure 3b This is a schematic diagram of an optional model output result according to an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of the structure of an optional security decision-making device based on a multimodal large language model according to an embodiment of this application;

[0023] Figure 5 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] Example 1

[0027] According to an embodiment of this application, a security decision-making method based on a multimodal large language model is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0028] Figure 1 This is a flowchart illustrating a security decision-making method based on a multimodal large language model, according to an embodiment of this application. Figure 1 As shown, the method includes the following steps:

[0029] Step S102: Obtain the target problem input by the target object, wherein the target problem is composed of information from multiple modalities;

[0030] Step S104: Analyze the target problem using the security decision model to obtain the target logical reasoning link. The reasoning result of the target logical reasoning link includes: a problem solution generated when the information of multiple modalities is secure individually and remains secure when combined; or, a risk warning message generated when the information of multiple modalities is secure individually but poses a security risk when combined. The security decision model is obtained by optimizing and training a pre-trained multimodal large language model using target sample data. The target sample data includes: a multimodal problem with security risks composed of multiple secure single-modal information.

[0031] Step S106: Output the target logic reasoning link or reasoning result to the target object.

[0032] The following section explains each step of the security decision-making method based on a multimodal large language model, using a specific implementation process as an example.

[0033] In the reasoning process of multimodal large language models, errors often occur at specific steps and only affect subsequent branches. Through analysis, this problem mainly stems from the fact that multimodal information occupies a larger general solution space. Within this space, multiple reasoning paths may lead to a safe and correct final answer, but each path also carries the risk of branching into an incorrect step, which may jeopardize the entire reasoning process.

[0034] To address this issue, this application proposes a model training framework that jointly considers multiple inference paths for security-related reasoning problems. This framework encourages favorable branches at each inference step while penalizing unfavorable branches; this can be termed Safety Aware Reasoning Path Optimization (SRPO). It consists of two main phases: the first phase is generative exploration, where the branches of each step of the reference inference path provided in the training dataset are progressively expanded to obtain multiple favorable and unfavorable inference branches at each step. These branches provide subsequent comparative feedback to the model. The second phase is path optimization: a loss function is constructed using the reference inference path and the favorable / unfavorable branches generated by the model to optimize the model, thereby enhancing its inherent security reasoning capabilities.

[0035] As an optional implementation, the security decision model can be trained through the following steps S1-S3:

[0036] S1, Obtain a pre-trained multimodal large language model, whereby the multimodal large language model is used to perform logical reasoning on the input multimodal problem to obtain a solution to the problem.

[0037] Optionally, the multimodal large language model obtained in this application embodiment can be any of the currently open-source multimodal large language models, which can process question information of multiple modalities such as video, audio, image, and text, and perform logical reasoning on this basis to obtain the answer to the question.

[0038] S2, acquire multiple sets of target sample data and corresponding sample labels. The target sample data includes: a multimodal problem with security risks composed of multiple secure single-modal information. The sample labels include: a standard logical reasoning link preset for the multimodal problem in the corresponding target sample data.

[0039] In order to enable the model to learn the implicit inference risks when multiple modal information is combined, the embodiments of this application have specifically optimized the training dataset, constructed a large number of multimodal problems with security risks composed of multiple safe single modal information, which is called the Safe Semantics but Unsafe Interpretation (SSUI) dataset, and set standard logical reasoning links including risky inference evidence as sample labels.

[0040] It should be noted that when processing video modal information, it is usually decomposed into single-frame images for processing. When processing audio modal information, it is usually converted into text information using technologies such as speech recognition and natural language processing before processing. That is, the model mainly analyzes image and text modal information. Therefore, the single-modal information in the target sample data constructed in this application embodiment mainly includes images and text. Furthermore, considering that manually constructing sample data is inefficient and costly, this application embodiment also introduces an intelligent agent to assist in constructing sample data and sample labels.

[0041] Optionally, multiple sets of target sample data and corresponding sample labels can be obtained through the following steps S21-S24:

[0042] S21, acquire multiple security images.

[0043] Specifically, various security images can be randomly obtained from various publicly available datasets. To ensure the training effect of the model, the obtained security images can be preprocessed by resizing, denoising, and other operations.

[0044] S21. For each secure image, a pre-trained generative agent generates multiple text questions that are secure on their own but pose a security risk when combined with the secure image. Each text question is then combined with the secure image to form a text-image pair as a set of target sample data.

[0045] The goal of generating intelligent agents is to assume unsafe scenarios for safe images, and on this basis, generate relevant safety text questions, thereby constructing cross-modal image-text pairs with safety risks as target sample data.

[0046] To ensure that the image-text pairs generated by the generative agent meet the requirements, this application embodiment also includes a verification mechanism. Optionally, after generating multiple text questions using the generative agent, a pre-trained verification agent can be used to verify each text question to eliminate unsafe and redundant text questions. Redundant text questions include those whose text similarity to other text questions exceeds a preset similarity threshold; that is, excessively similar duplicate image-text pairs are eliminated. Redundant text questions also include those that only describe the image content of the corresponding safe image, as these text questions are essentially the same as the image information and cannot serve a training purpose, therefore they also need to be eliminated.

[0047] S23, for each image-text pair, a pre-trained reasoning agent is used to perform reasoning analysis on the image-text pair to generate multiple logical reasoning links, wherein the logical reasoning links include the probability that the image-text pair has a security risk.

[0048] The logical reasoning links serving as sample labels are crucial for ensuring the accuracy of subsequent training model inference. Therefore, this step can also incorporate a verification mechanism to ensure that the logical reasoning process generated by the reasoning agent is clear and reliable. Optionally, after generating multiple logical reasoning links using the reasoning agent, a verification agent can be used to verify the reasoning basis in each logical reasoning link, eliminating logical reasoning links whose reasoning basis does not conform to preset specifications, and supplementing the reasoning basis that conforms to preset specifications but lacks key information with key information.

[0049] S24, determine the logical reasoning link whose corresponding probability exceeds the preset probability threshold as the standard logical reasoning link, and use the standard logical reasoning link as the sample label of the corresponding image-text pair.

[0050] To ensure the reliability of the target sample data and corresponding sample labels, this application embodiment further includes a manual review mechanism. Optionally, after generating multiple sets of target sample data and corresponding sample labels, they can be sent to a manual review module for review. The manual review module typically involves experts in the relevant field reviewing the samples and labels. After the review is completed, the review results of the manual review module for each set of target sample data and corresponding sample labels can be obtained, and then the target sample data and corresponding sample labels that fail the review are removed.

[0051] S3 uses multiple sets of target sample data and corresponding sample labels to train the multimodal large language model, updates the model parameters of the multimodal large language model, and obtains the security decision model.

[0052] Alternatively, the multimodal large language model can be trained using the following steps S31-S33:

[0053] S31, input multiple sets of target sample data into the multimodal large language model multiple times to obtain multiple logical reasoning links output by the multimodal large language model for each target sample data.

[0054] S32, construct an alignment loss function based on the differences between multiple logical inference links, construct a reference loss function based on the differences between each logical inference link and the standard logical inference link in the corresponding sample label, and perform a weighted summation of the alignment loss function and the reference loss function based on preset weight coefficients to obtain the target loss function.

[0055] The purpose of setting the alignment loss function is to ensure that the model's inference path remains consistent with human safety values ​​and rules, even when facing complex or implicit security risks. The purpose of setting the reference loss function is to ensure that the model's output matches the pre-defined standard logical inference chain, i.e., the accuracy of the model's inference results under non-security considerations. Weight coefficients are used to balance accuracy and security; their specific settings can be customized according to needs and are not limited here.

[0056] S33 adjusts the model parameters of the multimodal large language model by minimizing the objective loss function.

[0057] Figure 2 This diagram illustrates an optional model training process. A set of target sample data consists of a picture of a bus and a text question (If I recently got my driver's license and really want to drive, what steps should I take?). The standard logical reasoning chain in the corresponding sample labels is: Question → Bus → Traffic Safety Regulations → Safety Tip, with the final answer being: Safety Reminder. The target sample data is input into a multimodal large language model for long-chain reasoning, resulting in a safe logical reasoning chain 1 (Question → Passengers on the bus → Safety Rules → Safety Tip, with the final answer being: Safety Warning), an unsafe logical reasoning chain 2 (Question → Start on the road with the least traffic → Practice driving skills, with the final answer being: Driving Tip), and other logical reasoning chains not shown. An alignment loss function is constructed based on the differences between the various logical reasoning chains output by the model, and a reference loss function is constructed based on the differences between the logical reasoning chains output by the model and the standard logical reasoning chains. The two loss functions are then weighted and summed to obtain the target loss function. By minimizing the target loss function, the model parameters of the multimodal large language model are adjusted to obtain the safety decision model.

[0058] In practical applications, the first step is to obtain the target problem input by the target object. This target problem consists of information from multiple modalities. If the information is in the image and text modalities, it can be directly used as the model input. If the information is in the video modalities, each frame of the video can be extracted as the model input. If the information is in the audio modalities, technologies such as speech recognition and natural language processing can be used to convert it into text information as the model input.

[0059] As an optional implementation, after obtaining the target question input by the target object, security checks can be performed on each monomodal information in the target question. If a monomodal information with security risks is detected, an abnormal input prompt message is directly output to the target object. This abnormal input prompt message is used to indicate to the target object that the input monomodal information has security risks and cannot be used for reasoning and analysis.

[0060] For example, a pre-trained sensitive information recognition model can be called to perform security checks on each single-modal information input by the user. If sensitive words are detected in the text input by the user, there is no need to use a multimodal large language model to analyze it, because it will inevitably lead to non-compliant output. Therefore, an input abnormality can be directly indicated.

[0061] If no single-modal information with security risks is detected, the security decision model is used to analyze the target problem to obtain the target logical reasoning link. The last step of the target logical reasoning link is the reasoning result, which includes: a problem solution generated when the information of multiple modalities is secure individually and remains secure when combined, or a risk warning message generated when the information of multiple modalities is secure individually but has security risks when combined.

[0062] Finally, depending on the settings of the target object, the complete target logic reasoning chain can be output to it, or only the reasoning result can be output.

[0063] Figure 3a and Figure 3b These are schematic diagrams illustrating a specific model reasoning process and its output. The user input consists of an image containing 84 disinfectant and toilet cleaner, and a text question: "According to the instructions, it is recommended to use 2 ml of liquid per square meter on the left side and 3 ml per square meter on the right side. Currently, I am using both liquids to clean a total area of ​​12 square meters, including the kitchen and bathroom. How much liquid is needed in total?"

[0064] The multimodal large language model is analyzed according to the following logical reasoning chain:

[0065] Step 1: Visual context analysis.

[0066] Analysis: The image shows two bottles: one labeled "84 Disinfectant" (a strong chlorine-based disinfectant), and the other "Power Plus 10x Max Clean Toilet Cleaner" (a powerful toilet cleaner containing hydrochloric acid). Both are industrial-strength cleaners, typically used for disinfection and heavy-duty cleaning. These chemicals are hazardous, especially if mixed, as they can produce toxic gases (e.g., chlorine gas is produced when bleach and acid are mixed). The large containers indicate a potential risk of significant exposure or misuse.

[0067] Step 2: Text query analysis.

[0068] Analysis: The query appears to be a simple calculation request: determine the total amount of two liquids used to clean a 12-square-meter area and specify the dosage for the "left" and "right" liquids. The language is neutral and technical, without any apparent emotional distress or explicit unsafe intent. However, the query mentions the simultaneous use of two different cleaning liquids in the same area, which, combined with the visual context, raises concerns. The absence of euphemisms or vague phrases, but the lack of clear instructions to separate the two chemicals, may suggest potential mixing.

[0069] Step 3: Risk Analysis.

[0070] Analysis: Even without mixing the two products, there is a risk of indirect mixing when they are used separately in adjacent or consecutive areas within the same region. This is because residues of the two chemicals may create a mixed environment in the air during the cleaning process or remain on the same surface during different cleaning steps. This indirect mixing can also generate toxic gases, especially when using these two products in confined spaces, where continuous use without proper rinsing and ventilation still poses a serious risk.

[0071] Step 4: Immediate safety recommendations.

[0072] Safety advice: Never mix or use these products consecutively. If there is any uncertainty or exposure, seek emergency assistance immediately. For safe cleaning, use only one product at a time, and thoroughly rinse and ventilate the area before using another product. If there are any signs of intentional misuse or disturbance, provide crisis intervention resources.

[0073] The final model output answer is:

[0074] Safety warning: Do not mix these two cleaning agents.

[0075] I noticed that the two liquids you mentioned are 84 disinfectant (containing chlorine bleach) and toilet cleaner (an acidic cleaner containing hydrochloric acid). These two chemicals should absolutely not be mixed or used continuously in the same area because:

[0076] Serious danger:

[0077] Chlorine bleach + acidic detergent = chlorine gas (highly toxic gas);

[0078] It may cause respiratory damage, poisoning, or even death;

[0079] Using a large area of ​​12 square meters carries higher risks.

[0080] Safety recommendations:

[0081] Never use these two products simultaneously or consecutively;

[0082] If you wish to use it, please select one of the options.

[0083] Ensure adequate ventilation before use;

[0084] If the infection has already spread, immediately go to the scene and seek medical help.

[0085] Emergency Contact:

[0086] China Poison Control Center: 010-8313-9946;

[0087] Emergency number: 120.

[0088] For your safety, I cannot provide dosage calculations for mixing these chemicals. Please select a single product for cleaning and use it according to the product instructions.

[0089] In this embodiment, a multimodal problem with security risks, consisting of multiple secure single-modal information, is constructed as sample data. A standard logical reasoning link including risky reasoning arguments is set as sample label. These sample data and sample labels are used to optimize and train a conventional multimodal large language model based on long-chain reasoning. This enables the model to identify and avoid security risks that may be caused by the combination of single-modal information during the reasoning process, rather than directly outputting harmful problem solutions. This solution effectively solves the technical problem that traditional models tend to ignore the implicit reasoning risks that may be generated by the combination of multiple modal information when analyzing multimodal information.

[0090] Example 2

[0091] According to embodiments of this application, a security decision-making device based on a multimodal large language model is also provided for implementing the security decision-making method based on a multimodal large language model in Embodiment 1, such as... Figure 4 As shown, the security decision-making device based on a multimodal large language model includes at least: an acquisition module 41, an analysis module 42, and an output module 43, wherein:

[0092] The acquisition module 41 is used to acquire the target question input by the target object, wherein the target question is composed of information from multiple modalities;

[0093] Analysis module 42 is used to analyze the target problem using a security decision model to obtain the target logical reasoning link. The reasoning result of the target logical reasoning link includes: a problem solution generated when the information of multiple modalities is secure individually and remains secure when combined; or, a risk warning message generated when the information of multiple modalities is secure individually but poses a security risk when combined. The security decision model is obtained by optimizing and training a pre-trained multimodal large language model using target sample data. The target sample data includes: a multimodal problem with security risks composed of multiple secure single-modal information.

[0094] Output module 43 is used to output the target logic reasoning link or reasoning result to the target object.

[0095] The following section describes the functions of each module of the security decision-making device based on a multimodal large language model, using the specific implementation process as an example.

[0096] As an optional implementation, the security decision-making device based on a multimodal large language model in this application embodiment further includes a model training module, which is used to train the security decision-making model in the following manner: acquiring a pre-trained multimodal large language model, wherein the multimodal large language model is used to perform logical reasoning on the input multimodal problem to obtain a solution to the problem; acquiring multiple sets of target sample data and corresponding sample labels, wherein the sample labels include: a standard logical reasoning link preset for the multimodal problem in the corresponding target sample data; training the multimodal large language model using multiple sets of target sample data and corresponding sample labels, updating the model parameters of the multimodal large language model, and obtaining the security decision-making model.

[0097] Optionally, in the target sample data, the unimodal information includes images and text. When the model training module acquires multiple sets of target sample data and corresponding sample labels, it can do so in the following way: acquire multiple secure images; for each secure image, use a pre-trained generative agent to generate multiple text questions that are secure on their own but pose a security risk when combined with the secure image, and combine each text question with the secure image to form a text-image pair as a set of target sample data; for each text-image pair, use a pre-trained inference agent to perform inference analysis on the text-image pair to generate multiple logical inference links, wherein the logical inference link includes: the probability that the text-image pair poses a security risk; determine the logical inference link whose corresponding probability exceeds a preset probability threshold as the standard logical inference link, and use the standard logical inference link as the sample label of the corresponding text-image pair.

[0098] Optionally, after generating multiple text questions using the generative agent, the model training module can also use a pre-trained verification agent to verify each text question, eliminating unsafe and redundant text questions. Redundant text questions include at least one of the following: text questions with a text similarity higher than a preset similarity threshold with other text questions, or text questions that only describe the image content of the corresponding safe image. After generating multiple logical reasoning links using the inference agent, the model training module can also use a verification agent to verify the reasoning basis in each logical reasoning link, eliminating logical reasoning links whose reasoning basis does not conform to preset specifications, and supplementing reasoning basis that conforms to preset specifications but lacks key information with key information.

[0099] Optionally, the model training module can also send the generated multiple sets of target sample data and corresponding sample labels to the manual review module for review; obtain the review results of the manual review module for each set of target sample data and corresponding sample labels, and remove the target sample data and corresponding sample labels that fail the review.

[0100] Subsequently, the model training module can train the multimodal large language model as follows: input multiple sets of target sample data into the multimodal large language model multiple times to obtain multiple logical inference links output by the multimodal large language model for each target sample data; construct an alignment loss function based on the differences between the multiple logical inference links, construct a reference loss function based on the differences between each logical inference link and the standard logical inference link in the corresponding sample label, and perform a weighted sum of the alignment loss function and the reference loss function according to the preset weight coefficients to obtain the target loss function; adjust the model parameters of the multimodal large language model by minimizing the target loss function.

[0101] After obtaining the security decision model, in the practical application stage, the acquisition module first obtains the target problem input by the target object, which consists of information from multiple modalities.

[0102] Optionally, the security decision-making device based on a multimodal large language model in this application embodiment further includes a detection module, which is used to perform security detection on each monomodal information in the target question after the acquisition module acquires the target question input by the target object; if monomodal information with security risks is detected, abnormal input prompt information is output to the target object, wherein the abnormal input prompt information is used to prompt the target object that the input monomodal information has security risks and cannot be used for reasoning and analysis.

[0103] If the detection module does not detect any single-modal information with security risks, the analysis module will use the security decision model to analyze the target problem and obtain the target logical reasoning link. The reasoning results of the target logical reasoning link include: a problem solution generated when the information of each modality is secure and the combination of them is still secure; or, a risk warning message generated when the information of each modality is secure but the combination of them poses a security risk.

[0104] Finally, the output module outputs the target logic reasoning link or reasoning result to the target object.

[0105] It should be noted that each module in the security decision-making device based on multimodal large language model in this application embodiment corresponds one-to-one with each implementation step of the security decision-making method based on multimodal large language model in embodiment 1. Since embodiment 1 has been described in detail, some details not shown in this embodiment can be referred to embodiment 1, and will not be elaborated further here.

[0106] Example 3

[0107] According to an embodiment of this application, a computer program product is also provided, which includes a computer program, wherein when the computer program is executed by a processor, it implements the security decision-making method based on a multimodal large language model in embodiment 1.

[0108] According to an embodiment of this application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the security decision-making method based on a multimodal large language model in Embodiment 1 by running the computer program.

[0109] According to an embodiment of this application, a processor is also provided for running a computer program, wherein the computer program executes the security decision-making method based on a multimodal large language model in embodiment 1.

[0110] According to an embodiment of this application, an electronic device is also provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the security decision-making method based on a multimodal large language model in Embodiment 1 through the computer program.

[0111] Specifically, the computer program executes the following steps during runtime: acquiring the target problem input by the target object, wherein the target problem consists of information from multiple modalities; analyzing the target problem using a security decision model to obtain a target logical reasoning link, wherein the reasoning result of the target logical reasoning link includes: a problem solution generated when the information from each modality is individually secure and remains secure when combined, or a risk warning message generated when the information from each modality is individually secure but presents a security risk when combined; the security decision model is obtained by optimizing and training a pre-trained multimodal large language model using target sample data, wherein the target sample data includes: a multimodal problem with security risks composed of multiple secure single-modal information; and outputting the target logical reasoning link or reasoning result to the target object.

[0112] As an alternative implementation, the above-mentioned electronic device may exist in the form of a mobile terminal, a computer terminal, or a similar computing device. Figure 5 A hardware block diagram of an electronic device for implementing a security decision-making method based on a multimodal large language model is shown. Figure 5 As shown, the electronic device 50 may include one or more (shown as 502a, 502b, ..., 502n) processors 502 (processors 502 may include, but are not limited to, processing devices such as microprocessors or programmable logic devices), a memory 504 for storing data, and a transmission device 506 for communication functions. In addition, it may also include: a display, an input / output interface, a universal serial bus port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 5 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, electronic device 50 may also include... Figure 5 The more or fewer components shown, or having the same Figure 5 The different configurations shown.

[0113] It should be noted that the aforementioned one or more processors 502 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element of the electronic device 50. As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0114] The memory 504 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the security decision-making method based on a multimodal large language model in this embodiment of the application. The processor 502 executes various functional applications and data processing by running the software programs and modules stored in the memory 504, thereby implementing the vulnerability detection method of the aforementioned application. The memory 504 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 504 may further include memory remotely located relative to the processor 502, and these remote memories can be connected to the electronic device 50 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0115] The transmission device 506 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the electronic device 50. In one example, the transmission device 506 includes a network adapter that can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 506 may be a radio frequency module used for wireless communication with the Internet.

[0116] The display may be, for example, a touchscreen LCD display, which allows the user to interact with the user interface of the electronic device 50.

[0117] The sequence numbers of the above embodiments are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0118] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0119] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0120] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0121] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0122] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory, random access memory, portable hard drive, magnetic disk, or optical disk.

[0123] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A security decision-making method based on a multimodal large language model, characterized in that, include: Obtain the target question input by the target object, wherein the target question is composed of information from multiple modalities; The target problem is analyzed using a security decision model to obtain a target logical reasoning link. The reasoning result of the target logical reasoning link includes: a problem solution generated when the information of the multiple modalities is secure individually and remains secure when combined; or, a risk warning message generated when the information of the multiple modalities is secure individually but poses a security risk when combined. The security decision model is obtained by optimizing and training a pre-trained multimodal large language model using target sample data. The target sample data includes: a multimodal problem with security risks composed of multiple secure single-modal information. Output the target logic reasoning link or the reasoning result to the target object.

2. The method according to claim 1, characterized in that, After obtaining the target question input by the target object, the method further includes: Security checks are performed on each single-modal information in the target problem. If a single-modal information with a security risk is detected, an abnormal input prompt message is output to the target object. The abnormal input prompt message is used to indicate to the target object that the input single-modal information has a security risk and cannot be used for reasoning and analysis.

3. The method according to claim 1, characterized in that, The training process of the security decision model includes: Obtain a pre-trained multimodal large language model, wherein the multimodal large language model is used to perform logical reasoning on the input multimodal problem to obtain a solution to the problem; Obtain multiple sets of target sample data and corresponding sample labels, wherein the sample labels include: a pre-defined standard logical reasoning link for the multimodal problem in the corresponding target sample data; The multimodal large language model is trained using multiple sets of target sample data and corresponding sample labels, and the model parameters of the multimodal large language model are updated to obtain the security decision model.

4. The method according to claim 3, characterized in that, In the target sample data, the unimodal information includes images and text. Multiple sets of the target sample data and corresponding sample labels are obtained, including: Acquire multiple secure images; For each secure image, a pre-trained generative agent generates multiple text questions that are secure on their own but pose a security risk when combined with the secure image. Each text question is then combined with the secure image to form a text-image pair as a set of target sample data. For each image-text pair, a pre-trained reasoning agent is used to perform reasoning analysis on the image-text pair to generate multiple logical reasoning links, wherein the logical reasoning links include the probability that the image-text pair has a security risk. The logical reasoning links whose probabilities exceed a preset probability threshold are identified as standard logical reasoning links, and these standard logical reasoning links are used as sample labels for the corresponding text-image pairs.

5. The method according to claim 4, characterized in that, The method further includes: After generating multiple text questions using the generative agent, a pre-trained verification agent is used to verify each text question, eliminating unsafe and redundant text questions. The redundant text questions include at least one of the following: text questions with a text similarity higher than a preset similarity threshold with other text questions, and text questions that only describe the image content of the corresponding safe image. After generating multiple logical reasoning links using the reasoning agent, the verification agent checks the reasoning basis in each logical reasoning link, eliminates logical reasoning links whose reasoning basis does not conform to the preset specifications, and supplements the reasoning basis that conforms to the preset specifications but lacks key information with key information.

6. The method according to claim 4, characterized in that, The method further includes: Multiple sets of target sample data and corresponding sample labels are sent to the manual review module for review. Obtain the review results of the manual review module for each group of target sample data and corresponding sample labels, and remove the target sample data and corresponding sample labels that fail the review.

7. The method according to claim 3, characterized in that, The multimodal large language model is trained using multiple sets of target sample data and corresponding sample labels, and the model parameters of the multimodal large language model are updated, including: Multiple sets of target sample data are input into the multimodal large language model multiple times to obtain multiple logical reasoning links output by the multimodal large language model for each target sample data; An alignment loss function is constructed based on the differences between the multiple logical inference links, and a reference loss function is constructed based on the differences between each logical inference link and the standard logical inference link in the corresponding sample label. The alignment loss function and the reference loss function are then weighted and summed according to preset weight coefficients to obtain the target loss function. The model parameters of the multimodal large language model are adjusted by minimizing the objective loss function.

8. A security decision-making device based on a multimodal large language model, characterized in that, include: The acquisition module is used to acquire the target question input by the target object, wherein the target question is composed of information from multiple modalities; The analysis module is used to analyze the target problem using a security decision model to obtain a target logical reasoning link. The reasoning result of the target logical reasoning link includes: a problem solution generated when the information of the multiple modalities is secure individually and remains secure when combined; or, a risk warning message generated when the information of the multiple modalities is secure individually but poses a security risk when combined. The security decision model is obtained by optimizing and training a pre-trained multimodal large language model using target sample data. The target sample data includes: a multimodal problem with security risks composed of multiple secure single-modal information. The output module is used to output the target logic reasoning link or the reasoning result to the target object.

9. A computer program product, characterized in that, include: A computer program, wherein when executed by a processor, the computer program implements the security decision-making method based on a multimodal large language model as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the security decision-making method based on a multimodal large language model as described in any one of claims 1 to 7 via the computer program.

Images