Multimodal risk quantification assessment method and apparatus

Abstract

This invention provides a method for multimodal risk quantification assessment. Based on a predetermined set of risk dimensions for multimodal risk quantification assessment, it generates structured risk assessment criteria for multimodal risk and stores them in a criterion library. Multiple expert models are invoked to perform parallel annotation of a first multimodal risk sample to be labeled, generating a risk dataset with structured annotations. The multimodal base model is then fine-tuned under multi-task supervision based on the risk dataset to obtain a corresponding multimodal risk quantification assessment model. The multimodal risk quantification assessment model is deployed to an online assessment system, and a quantification assessment is performed on a second multimodal risk sample to be assessed based on the criterion library. This invention also provides a device, storage medium, and electronic device based on multimodal risk quantification assessment. Therefore, this invention enables refined assessment of multimodal input risks.

Description

Technical Field

[0001] This invention relates to the field of multimodal risk detection technology, and in particular to a method, apparatus, storage medium and electronic device based on multimodal risk quantification assessment. Background Technology

[0002] With the development of multimodal large models, multimodal information has been widely applied in scenarios such as intelligent question answering, retrieval enhancement generation, and human-computer interaction. Compared with single text input, image-text combined input has stronger semantic expression capabilities, but it is also more likely to carry risky information through cross-modal collaboration, contextual misleading, etc., posing a greater challenge to content security identification.

[0003] Existing multimodal risk detection technologies can be broadly categorized as follows:

[0004] (1) Risk detection methods based on rules or keywords.

[0005] These methods typically target sensitive words, risky phrases, fixed templates, or explicit violations in images. Implementation methods mainly include keyword matching, pattern matching, and rule filtering for text; target detection, image classification, or feature matching for images; and a method of first detecting each text / image combination separately and then performing a simple fusion to provide the final result. For example, Chinese invention patent CN106445998A, "A Text Content Review Method and System Based on Sensitive Words," calculates violation weights using a sensitive word database, synonyms or near-synonyms, and keywords co-occurring with sensitive words to determine whether the text violates regulations. Chinese invention patent CN110674255A, "Text Content Review Method and Device," further utilizes an AI review model for secondary review based on keyword matching. These methods are simple to implement and highly efficient, but they are mainly applicable to explicit risky content and have weak detection capabilities for implicit risks and cross-modal combination risks.

[0006] (2) Multimodal risk detection method based on supervised learning.

[0007] These methods typically train a classification model based on existing labeled data, mapping text and image inputs to risk category labels or safe / unsafe binary labels. For example, Chinese invention patent CN115512259A, "A Multimodal Short Video Review Method," splits short videos into audio and image parts, analyzing them separately using automatic speech recognition, optical character recognition, and facial recognition, and then combining keyword recognition to determine whether there is a violation. Chinese invention patent CN118013144A, "A Multimodal Method, Apparatus, Device, and Medium for Detecting Malicious Webpages," further integrates text features formed from webpage titles, webpage text, webpage tags, and webpage addresses, image features formed from webpage screenshots, and text-image position features, using a multimodal self-attention network to complete the detection of malicious webpages. Chinese invention patent CN116778376B, "A Content Security Detection Model Training Method, Detection Method, and Apparatus," for video content security detection, proposes extracting video compression domain information and multi-source features under the standard color gamut, and obtaining a content security detection model through cross-modal distillation learning. This type of method has stronger generalization ability than pure rule-based methods, but it usually suffers from problems such as coarse label granularity, inconsistent labeling standards, only able to determine whether a violation has occurred, difficulty in quantifying the degree of risk, and difficulty in identifying novel risks not covered in the training set.

[0008] (3) Risk assessment method based on large model discrimination.

[0009] In recent years, some technologies have begun to utilize large language models or visual language models to directly assess the risk of input samples, and even use multiple models for cross-scoring. For example, Chinese invention patent CN119339419A, "A Risk Content Recognition Method Based on a Multimodal Large Model," inputs the outputs of audio segmentation models and image segmentation models, along with image features, into a large language model. The large language model then summarizes risk points for multi-source information fusion and discrimination in specific risk scenarios such as forgery and fraud identification. Chinese invention patent CN120012776B, "Content Security Recognition Method Based on the Integration of Multiple Large Language Models," retrieves relevant text from a sensitive database and constructs security identification prompts. It inputs at least two large language models and integrates their outputs to improve the accuracy of text content security identification. While these methods can improve risk assessment capabilities in complex semantic scenarios to some extent, they also suffer from problems such as opaque model judgment criteria, inherent model bias, and susceptibility to the influence of prompt methods.

[0010] Overall, existing technologies have covered areas such as rule-based text moderation, multimodal content moderation based on audio / image / text, cross-modal distillation training for video scenarios, risk scoring for data compliance, and risk identification based on large models or multi-model integration. However, most of these solutions are still based on "whether it is a violation" or specific scenario classification, lacking unified, structured, and transferable multimodal risk assessment criteria. They rarely output risk category, risk intensity, modal contribution, and criterion basis simultaneously, and their ability to generalize assessment of unseen risk types remains insufficient.

[0011] Problem 1: Lack of unified, structured multimodal risk criteria. Existing risk assessment standards are usually scattered among human experience, dataset label descriptions, or model hints, leading to inconsistent understanding among labelers, difficulty in sharing the same assessment criteria across different models, and difficulty in tracing the specific risk basis of model outputs.

[0012] Question 2: Existing models are mostly coarse-grained classifications, which cannot achieve risk quantification and attribution. Existing multimodal risk models usually only output classification labels or binary results, making it difficult to further answer which dimension the risk belongs to, what level the risk intensity is, whether the risk mainly comes from image, text or graphic-text interaction modality, and what the basis for risk determination is.

[0013] Question 3: Lack of ability to assess unseen risk types. Existing methods typically rely on training data coverage, and are prone to missed detections or misjudgments when faced with novel expressions, variant expressions, evasive expressions, and implicitly suggestive content.

[0014] Question 4: Lack of a complete and systematic implementation path. Existing technologies often only focus on one aspect, such as dataset construction, model evaluation, or content review, and lack a closed-loop solution that covers risk dimension definition, structured criterion generation, sample pair construction, multi-expert model annotation, multi-task fine-tuning, criterion-driven inference, risk assessment, and result interpretation.

[0015] In conclusion, the existing technology obviously has inconveniences and defects in practical use, so it is necessary to improve it. Summary of the Invention

[0016] To address the aforementioned shortcomings, the present invention aims to provide a method, apparatus, storage medium, and electronic device for multimodal risk quantification assessment, which can achieve a refined assessment of multimodal input risks.

[0017] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0018] In a first aspect, embodiments of the present invention provide a method for multimodal risk quantification assessment, comprising the following steps:

[0019] The criteria generation step involves generating multimodal structured risk assessment criteria based on a predetermined set of risk dimensions for multimodal risk quantification assessment and storing them in the criteria library.

[0020] The risk labeling step involves calling multiple expert models to perform parallel labeling of the first multimodal risk sample to be labeled, generating a risk dataset with structured labels.

[0021] The supervised fine-tuning step involves performing multi-task supervised fine-tuning on the multimodal base model based on the risk dataset to obtain the corresponding multimodal risk quantification assessment model.

[0022] The risk assessment step involves deploying the multimodal risk quantification assessment model to the online assessment system and performing a quantification assessment on the second multimodal risk sample to be assessed based on the criterion library.

[0023] According to the multimodal risk quantification assessment method of the present invention, the criterion generation step further includes:

[0024] The dimension definition sub-step defines the set of risk dimensions used for multimodal risk quantification assessment, the set of risk dimensions including multiple risk quantification assessment dimensions;

[0025] The criteria generation sub-step generates risk assessment criteria under different dimensions based on the set of risk dimensions, including image criteria, text criteria, and / or image-text combination criteria.

[0026] The criteria structuring sub-step maps each of the aforementioned risk assessment criteria to a structured risk assessment criterion that includes triggering conditions, exclusion conditions, scoring mapping functions, and / or output interpretations.

[0027] The standard entry sub-step involves storing the structured risk assessment standards into the standard database.

[0028] According to the multimodal risk quantification assessment method of the present invention, the dimension definition sub-step further includes:

[0029] Define the set of risk dimensions D for multimodal risk quantification assessment;

[0030]

[0031] The risk dimension set D includes multiple risk quantification assessment dimensions d, wherein, This indicates the concealment of malicious intent. This indicates that the content is executable. Indicates the scope of the influence. This indicates the sensory concreteness and impact. This indicates the attitude or inclination expressed in the text. This indicates whether the content is true or misleading;

[0032] The criterion generation sub-step further includes:

[0033] For any sample x to be evaluated, it can be expressed as:

[0034]

[0035] Where I represents image input and T represents text input;

[0036] Based on the sample x to be evaluated, a six-dimensional risk score vector needs to be output:

[0037]

[0038] in, , representing the score of the sample x to be evaluated on the i-th dimension of the risk quantification assessment;

[0039] The criterion structuring sub-step further includes:

[0040] Let the modal set be:

[0041]

[0042] in, Represents image modality, Represents text modality, Represents a combined text and image mode;

[0043] For each of the aforementioned risk quantification assessment dimensions For each modal form m, construct the structured risk assessment criterion C:

[0044]

[0045] in: For the criteria identification; As a dimension for risk quantification assessment; For applicable modal types; The triggering condition; The exclusion condition; The rating mapping function; For explaining the template;

[0046] The criterion entry sub-step further includes:

[0047] The structured risk assessment criteria are stored in the criteria database. :

[0048] .

[0049] According to the multimodal risk quantification assessment method of the present invention, the risk labeling step further includes:

[0050] The annotation sub-step involves calling the multiple expert models to perform parallel annotation of the first multimodal risk sample to be annotated, generating the structured annotation;

[0051] The aggregation sub-step involves summarizing the structured annotations to generate candidate aggregation results, which are then sent to the multiple expert models.

[0052] The judgment sub-step involves the multiple expert models determining whether to accept the candidate aggregation result.

[0053] The voting sub-step performs a consensus vote on the candidate aggregation results to determine the final annotation results;

[0054] The set sub-step generates the risk dataset with the final labeled results.

[0055] According to the multimodal risk quantification assessment method described above, the annotation sub-step further includes:

[0056] The multiple expert models are invoked for the first multimodal risk sample X to be labeled;

[0057] ;

[0058] For each sample The multiple expert models are called as the expert model set to process the sample. Perform parallel evaluation;

[0059] Let the multiple expert models be:

[0060]

[0061] Each of the expert models The sample received the same input And a unified annotation prompt template, and output the sample. The structured annotations mentioned above:

[0062]

[0063] in: Indicates the first A six-dimensional risk score is given by an expert model; This indicates the overall risk level given by the expert model; This indicates the risk source modality, with values ​​ranging from image, text, or a combination of image and text. Indicates the type of risk; Indicates confidence level; This indicates explanatory information or the basis for judgment;

[0064] The voting sub-step further includes:

[0065] Risk level Source modality and risk type Perform majority voting separately:

[0066] ;

[0067] ;

[0068] ;

[0069] Where Mode is the mode function used to count the number of occurrences;

[0070] Define a consistent subset:

[0071]

[0072] Then, the continuous risk scores across the six dimensions are aggregated using a weighted aggregation method within a consistent subset:

[0073]

[0074] The sample was thus obtained. The final annotation results

[0075] ;

[0076] The set sub-step further includes:

[0077] Generate the risk dataset D with the final labeled results:

[0078] .

[0079] According to the multimodal risk quantification assessment method of the present invention, the supervised fine-tuning step further includes:

[0080] The multimodal base model is fine-tuned under multi-task supervision based on the risk dataset.

[0081] For training samples Construct a unified supervision objective:

[0082]

[0083] in, It can represent the structured criterion number or criterion set that was hit;

[0084] Generate the corresponding multimodal risk quantification assessment model;

[0085]

[0086] in: This is the predicted value for the six-dimensional risk score; This is the predicted value for the overall risk level; The predicted values ​​are from the source modal; For risk type predictions; This refers to either the criterion matching result or the criterion interpretation result.

[0087] According to the multimodal risk quantification assessment method of the present invention, the risk assessment step further includes:

[0088] The risk assessment steps include deploying the multimodal risk quantification assessment model to the online assessment system and performing a quantification assessment on the second multimodal risk sample to be assessed based on the criterion library.

[0089] Deploy the multimodal risk quantification assessment model to the online assessment system;

[0090] The online assessment system receives the second multimodal risk sample to be assessed. ;

[0091] The multimodal risk quantification assessment model yields the following:

[0092]

[0093] in:

[0094]

[0095] The overall risk score is calculated based on the six-dimensional risk assessment:

[0096]

[0097] in, This indicates the weight of each dimension in the quantitative risk assessment.

[0098] For unseen samples, the system can still perform evaluations based on structured criteria:

[0099] Let the confidence level of the risk type output by the model be:

[0100]

[0101] If the following conditions exist:

[0102]

[0103] Given samples and criteria The matching function is:

[0104]

[0105] when When the sample is identified as an unseen sample that can be evaluated according to the criteria, its six-dimensional risk score, overall risk level, and corresponding criterion description are output.

[0106] The high-confidence unseen samples obtained from the online assessment are added to the existing risk dataset.

[0107] Let the new sample set be:

[0108]

[0109] The updated risk dataset is then:

[0110] .

[0111] Secondly, embodiments of the present invention provide a multimodal risk quantification assessment device constructed based on any one of the methods described above, the device comprising:

[0112] The criteria generation module is used to generate multimodal structured risk assessment criteria based on a predetermined set of risk dimensions for multimodal risk quantification assessment and store them in the criteria library;

[0113] The risk labeling module is used to call multiple expert models to perform parallel labeling of the first multimodal risk sample to be labeled, generating a risk dataset with structured labels;

[0114] The supervised fine-tuning module is used to perform multi-task supervised fine-tuning of the multimodal base model based on the risk dataset to obtain the corresponding multimodal risk quantification assessment model.

[0115] The risk assessment module is used to deploy the multimodal risk quantification assessment model to the online assessment system and perform quantification assessment on the second multimodal risk sample to be assessed based on the criterion library.

[0116] Thirdly, embodiments of the present invention provide a storage medium for storing a computer program for performing any of the methods described herein.

[0117] Fourthly, embodiments of the present invention provide an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement any of the methods described above.

[0118] Therefore, this invention is based on multimodal risk quantification assessment technology, including: (1) generating multimodal structured risk assessment criteria according to a predetermined set of risk dimensions and storing them in a criterion library. This achieves a unified expression of the judgment criteria, enabling the criteria to be used for data construction, automatic labeling, model training, and online inference, thereby improving consistency, interpretability, and generalization ability. (2) calling multiple expert models to perform parallel labeling of the first multimodal risk sample to be labeled, generating a risk dataset with structured labels. This reduces the bias caused by a single model or a single labeler, improves the stability of boundary sample labeling, and provides highly reliable quantification labels for subsequent multi-task training. (3) performing multi-task supervised fine-tuning of the multimodal basic model according to the risk dataset to obtain the corresponding multimodal risk quantification assessment model. Compared with traditional binary classification or single-task classification models, it can more comprehensively and finely represent risks, improving the assessment ability of complex samples. (4) deploying the multimodal risk quantification assessment model to an online assessment system, and performing quantification assessment on the second multimodal risk sample to be assessed according to the criterion library. This invention supports the assessment of unseen risks based on criteria, forming a deployable, interpretable, and sustainably iterative closed-loop system. Therefore, this invention enables refined assessment of multimodal input risks and possesses the ability to generalize to unseen risk types. Attached Figure Description

[0119] Figure 1 This is a flowchart illustrating the multimodal risk quantification assessment method provided in Embodiment 1 of the present invention.

[0120] Figure 2 This is a flowchart illustrating the multimodal risk quantification assessment method provided in Embodiment 2 of the present invention;

[0121] Figure 3 This is an overall flowchart of the multimodal risk quantification assessment method provided in Embodiment 3 of the present invention;

[0122] Figure 4 This is a schematic diagram of the structure of the multimodal risk quantification assessment device provided in Embodiment 4 of the present invention;

[0123] Figure 5 This is a schematic diagram of the structure of the electronic device provided in Embodiment 5 of the present invention. Detailed Implementation

[0124] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0125] It should be noted that references to "an embodiment," "embodiment," "example embodiment," etc., in this specification refer to the described embodiment including specific features, structures, or characteristics, but not every embodiment must include these specific features, structures, or characteristics. Furthermore, such expressions do not refer to the same embodiment. Moreover, when describing specific features, structures, or characteristics in conjunction with embodiments, whether or not explicitly described, it is indicated that incorporating such features, structures, or characteristics into other embodiments is within the knowledge of those skilled in the art.

[0126] Furthermore, certain terms are used in the specification and subsequent claims to refer to specific components or parts. Those skilled in the art will understand that manufacturers may use different names or terms to refer to the same component or part. This specification and subsequent claims do not distinguish components or parts by differences in name, but rather by differences in function. The terms "comprising" and "including" used throughout the specification and subsequent claims are open-ended and should be interpreted as "including but not limited to." Additionally, the term "connection" here includes any direct and indirect electrical connection means. Indirect electrical connection means include connections made through other means.

[0127] The multimodal risk quantification assessment method provided by the present invention will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0128] In its research on the security evaluation of multimodal large models, this invention discovered two fundamental shortcomings in existing multimodal risk assessment methods. First, most existing technologies treat risk assessment as a fixed-category identification problem, directly predicting which risk category a given input belongs to, without further addressing "why it constitutes a risk." Especially in image-text combined scenarios, risk is often not directly manifested by a single modality, but rather formed by the combined relationships of image semantics, text semantics, and context. Second, most existing technologies treat risk labeling as a binary classification label assignment problem, without evaluating it from a more granular perspective.

[0129] This invention further reveals that what truly determines a model's risk identification capability is often not obviously risky samples far from the boundary, but rather the safe-risk transition sample pairs generated after minimal modification to the original samples. For example: pairing a safe image with suggestive text may constitute a risk; making slight semantic substitutions to risky text may cause it to fall back into the safe zone; replacing, occluding, or cropping local areas of an image may change the overall risk level; an image and text may appear safe individually, but their combination may trigger new risks.

[0130] Based on the above findings, this invention proposes a multimodal risk quantification assessment system and method that combines risk dimension-driven structured criterion automatic generation, multi-expert model joint annotation, multi-task fine-tuning, and criterion-driven reasoning. This approach does not simply judge results from the model output, but designs a complete chain from "risk criteria—sample annotation—quantification assessment—interpretive output," thereby achieving refined assessment of multimodal input risks and possessing the ability to generalize to unseen risk types.

[0131] Figure 1 This is a flowchart illustrating the multimodal risk quantification assessment method provided in Embodiment 1 of the present invention. The method includes the following steps:

[0132] Step S101, the criterion generation step, generates multimodal structured risk assessment criteria based on a predetermined set of risk dimensions for multimodal risk quantification assessment and stores them in a criterion library. This achieves a unified expression of the judgment criteria, enabling the criteria to be used for data construction, automatic annotation, model training, and online inference, thereby improving consistency, interpretability, and generalization ability.

[0133] Step S102, the risk labeling step, involves calling multiple expert models to perform parallel labeling of the first multimodal risk samples to be labeled, generating a risk dataset with structured annotations. This reduces the bias caused by a single model or a single labeler, improves the stability of boundary sample labeling, and provides highly reliable quantifiable labels for subsequent multi-task training.

[0134] Step S103, supervised fine-tuning step, involves performing multi-task supervised fine-tuning on the multimodal base model based on the risk dataset to obtain the corresponding multimodal risk quantification assessment model. Compared to traditional binary classification or single-task classification models, this model can represent risks more comprehensively and with finer granularity, improving the assessment capability for complex samples.

[0135] Step S104, Risk Assessment Step: Deploy the multimodal risk quantification assessment model to the online assessment system, and perform a quantification assessment on the second multimodal risk sample to be assessed based on the criterion library. This supports the assessment of unseen risks based on criteria, forming a deployable, interpretable, and sustainably iterative closed-loop system.

[0136] The technical solutions of this invention collectively constitute a complete technical chain, from risk rule abstraction to data annotation, from data to model training, and from model to online evaluation. Therefore, this invention enables refined evaluation of multimodal input risks and possesses the ability to generalize to unseen risk types. Specific technical effects include:

[0137] 1. Greater uniformity: Through structured criteria, multimodal risk assessment is transformed from experience-based judgment into a unified technical framework that is reusable, trainable, and reasonable.

[0138] 2. Higher labeling quality: Through joint labeling by multi-expert models, consistency verification, and boundary sample pair labeling, the accuracy and stability of risk labels and risk scores can be improved.

[0139] 3. Finer granularity of assessment: It can not only output safe / unsafe conclusions, but also risk category, risk intensity, modal contribution, risk source and corresponding explanation.

[0140] 4. Stronger generalization ability: Through the criterion-driven mechanism, it can assess unseen risk types and reduce dependence on fixed-label datasets.

[0141] 5. More suitable for model analysis and governance: It can be used to analyze the differences in the identification capabilities of different multimodal models across different risk dimensions, and support targeted model optimization and security governance strategy formulation.

[0142] Figure 2 This is a flowchart illustrating the multimodal risk quantification assessment method provided in Embodiment 2 of the present invention. The method includes the following steps:

[0143] Step S201, Dimension Definition Sub-step, defines a set of risk dimensions for multimodal risk quantification assessment, which includes multiple risk quantification assessment dimensions.

[0144] Preferably, the criterion generation sub-step further includes:

[0145] For any sample x to be evaluated, it can be expressed as:

[0146]

[0147] Where I represents image input and T represents text input.

[0148] Based on the sample x to be evaluated, a six-dimensional risk score vector needs to be output:

[0149]

[0150] in, , representing the score of the sample x to be evaluated on the i-th dimension of risk quantification assessment.

[0151] Step S202, the criteria generation sub-step, generates risk assessment criteria under different dimensions based on the risk dimension set, including image criteria, text criteria, and / or image-text combination criteria.

[0152] Preferably, the criterion structuring sub-step further includes:

[0153] Let the modal set be:

[0154]

[0155] in, Represents image modality, Represents text modality, This indicates a combined text and image mode.

[0156] For each dimension of risk quantification assessment For each modal form m, construct the structured risk assessment criterion C:

[0157]

[0158] in: This serves as a standard identifier. This serves as a dimension for risk quantification assessment. For applicable modal types. This is the trigger condition. To exclude conditions. This is the rating mapping function. This is for explaining the template.

[0159] Step S203, the criteria structuring sub-step, maps each risk assessment criterion into a structured risk assessment criterion containing triggering conditions, exclusion conditions, scoring mapping functions, and / or output interpretations.

[0160] Step S204, the criteria entry sub-step, involves storing the structured risk assessment criteria into the criteria database.

[0161] Preferably, the criteria entry sub-step further includes:

[0162] Store structured risk assessment criteria in the criteria database. :

[0163] .

[0164] Preferably, the risk labeling step further includes:

[0165] Step S205, the annotation sub-step, calls multiple expert models to perform parallel annotation of the first multimodal risk sample to be annotated, generating structured annotations.

[0166] Preferably, the annotation sub-step further includes:

[0167] Multiple expert models are invoked to label the first multimodal risk sample X.

[0168] .

[0169] For each sample Multiple expert models are called as an expert model set for this sample. Perform parallel evaluation.

[0170] Suppose there are multiple expert models:

[0171]

[0172] Each expert model Receive the same input samples And a unified annotation prompt template, and output samples. Structured annotation:

[0173]

[0174] in: Indicates the first A six-dimensional risk score is given by an expert model. This indicates the overall risk level given by the expert model. This indicates the risk source modality, with values ​​that can be images, text, or a combination of images and text. Indicates the type of risk. Indicates the confidence level. It indicates explanatory information or the basis for judgment.

[0175] Step S206, the aggregation sub-step, involves summarizing the structured annotations to generate candidate aggregation results, which are then sent to multiple expert models.

[0176] Step S207, the judgment sub-step, involves multiple expert models determining whether to accept the candidate aggregation result.

[0177] Step S208, voting sub-step, performs consistency voting on the candidate aggregation results to determine the final annotation result.

[0178] Preferably, the voting sub-step further includes:

[0179] Risk level Source modality and risk type Perform majority voting separately:

[0180] .

[0181] .

[0182] .

[0183] Mode is the mode function used to count the number of occurrences.

[0184] Define a consistent subset:

[0185]

[0186] Then, the continuous risk scores across the six dimensions are aggregated using a weighted aggregation method within a consistent subset:

[0187]

[0188] This yields the sample. Final annotation results

[0189] .

[0190] Step S209, the set sub-step, generates a risk dataset with final labeled results.

[0191] Preferably, the set sub-step further includes:

[0192] Generate the risk dataset D with final annotation results:

[0193] .

[0194] Step S210, supervised fine-tuning step, performs multi-task supervised fine-tuning of the multimodal base model based on the risk dataset to obtain the corresponding multimodal risk quantification assessment model.

[0195] Preferably, the monitoring and fine-tuning step further includes:

[0196] The multimodal base model is fine-tuned under multi-task supervision based on the risk dataset.

[0197] For training samples Construct a unified supervision objective:

[0198]

[0199] in, It can represent the structured criterion number or criterion set that was hit.

[0200] Generate the corresponding multimodal risk quantification assessment model.

[0201]

[0202] in: This is the predicted value for the six-dimensional risk score. This is the predicted value for the overall risk level. The predicted value is the source modality. This is the predicted value for the risk type. This refers to either the criterion matching result or the criterion interpretation result.

[0203] Step S211, risk assessment step: deploy the multimodal risk quantification assessment model to the online assessment system, and perform quantification assessment on the second multimodal risk sample to be assessed according to the criterion library.

[0204] Preferably, the risk assessment step further includes:

[0205] Deploy a multimodal risk quantification assessment model into the online assessment system.

[0206] The online assessment system receives the second multimodal risk sample to be assessed. .

[0207] The results obtained through the multimodal risk quantification assessment model are as follows:

[0208]

[0209] in:

[0210]

[0211] The overall risk score is calculated based on the six-dimensional risk assessment:

[0212]

[0213] in, This indicates the weight of each dimension in the quantitative risk assessment.

[0214] For unseen samples, the system can still perform evaluations based on structured criteria:

[0215] Let the confidence level of the risk type output by the model be:

[0216]

[0217] If the following conditions exist:

[0218]

[0219] Given samples and criteria The matching function is:

[0220]

[0221] when When the sample is identified as an unseen sample that can be evaluated according to the criteria, its six-dimensional risk score, overall risk level, and corresponding criterion description are output.

[0222] The high-confidence unseen samples obtained from the online assessment are added to the existing risk dataset.

[0223] Let the new sample set be:

[0224] The updated risk dataset is then: .

[0225] Figure 3 This is an overall flowchart of the multimodal risk quantification assessment method provided in Embodiment 3 of the present invention. This embodiment provides a method for multimodal risk quantification assessment that accepts image, text, and image-text combination inputs, continuously scores the input samples across multiple risk quantification assessment dimensions, and outputs the risk level, source modality, risk type, and criterion basis. The method mainly includes the following steps:

[0226] Step S301: Automatic generation of risk-dimensional driven structured criteria

[0227] First, we define the set of dimensions for multimodal risk quantification assessment.

[0228]

[0229] in: This indicates the concealment of malicious intent; Indicates content executability; Indicates the scope of influence; It indicates the vividness and impact of sensory experiences; Indicates the text's attitude or inclination; This indicates whether the content is true or misleading.

[0230] For any sample to be evaluated It can be represented as:

[0231]

[0232] in, Indicates image input, This represents text input; the corresponding position can be empty if no image or text exists. Based on this sample, the system needs to output a six-dimensional risk score vector:

[0233]

[0234] in, , indicating that the sample is in the th Scores on each dimension of risk quantification assessment.

[0235] Furthermore, to enable the assessment criteria to be automatically invoked, labeled, and learned by the model, this method extends each dimension of risk quantification assessment into structured criteria applicable to different modalities. Let the modality set be:

[0236]

[0237] in, Represents image modality, Represents text modality, This indicates a combined text and image mode.

[0238] For each dimension of risk quantification assessment and each modal form Constructing structured criteria:

[0239]

[0240] in: For the criteria identification; For corresponding risk quantification assessment dimensions; For applicable modal types; For the set of triggering conditions; To exclude the set of conditions; For rating mapping function; This is for explaining the template.

[0241] In other words, this method does not directly provide an abstract "scoring dimension", but constructs image criteria, text criteria and image-text combination criteria for each dimension, so that the system can distinguish whether the risk comes from the image, the text, or the semantic coupling of the image and text.

[0242] In summary, the automatic criterion generation process includes:

[0243] 1. Read the predefined risk quantification assessment dimensions description;

[0244] 2. Generate image criteria, text criteria, and image-text combination criteria for that dimension based on the dimension description;

[0245] 3. Map each criterion to a structure of "trigger condition - exclusion condition - scoring range - output explanation";

[0246] 4. Store all criteria in the criteria library to form a set of callable structured risk assessment criteria.

[0247] For example, regarding the concealment of malicious intent The following criteria can be generated: (1) The triggering conditions may include: text containing jailbreak-related veiled attacks, metaphorical offensive expressions, ironic inducements, euphemisms for circumventing censorship, and directive text disguised as study or research but actually pointing to harmful behavior. When the text is clearly positive, safe, and does not contain hidden inducements, the corresponding scoring range is... When the text contains obvious evasive expressions, puns leading to conclusions, or jailbreak-related elements, the corresponding score range is... (2) The triggering conditions may include: the presence of visually injected elements in the image, embedded prompts, pseudo-instructions in the form of screenshots, spliced ​​inducement information, and the use of image layout to hide malicious prompts. When the image content is natural, the semantics are clear and there is no inducement embedded information, the score is low; when the image contains visual prompts that can guide the model to deviate from normal safe behavior, the score is high. (3) This criterion is used to determine situations where an image and text may appear safe on their own, but when combined, they form a malicious intent. For example, if the text appears to be a normal question, but the image embeds actual attack targets, operational conditions, or evasion prompts, causing the overall semantics to transform into a harmful intent, then the sample will be rated as high on the dimension of malicious intent concealment.

[0248] Similarly, regarding content executability It can generate image criteria, text criteria, and combined image-text criteria to determine whether a sample provides harmful guidance that can be directly implemented in the real or digital world; and to assess the authenticity and misleading nature of the content. It can generate rules for combining text and graphics against deepfakes, fake context splicing, fake documents, fake political information, and fake medical information.

[0249] In this embodiment, the system ultimately forms a criteria library:

[0250]

[0251] This criteria library serves as a unified basis for subsequent automatic annotation, model fine-tuning, and risk assessment of unseen cases.

[0252] Step S302: Automatic labeling of risk data based on multi-expert model

[0253] After the criterion library is built, the system obtains the multimodal risk sample set to be labeled:

[0254]

[0255] For each sample The system calls multiple mainstream commercial model APIs as expert models to perform parallel evaluation of the sample. Let the set of expert models be:

[0256]

[0257] Each expert model Receive the same input samples It also provides a unified annotation prompt template and outputs the structured annotation results for the sample:

[0258]

[0259] in: Indicates the first A six-dimensional risk score is given by an expert model; This indicates the overall risk level given by the expert model; This indicates the risk source modality, with values ​​ranging from image, text, or a combination of image and text. Indicates the type of risk; Indicates confidence level; It indicates explanatory information or the basis for judgment.

[0260] In one specific implementation, to improve the quality of data annotation, the system adopts a two-stage annotation mechanism:

[0261] (1) Independent labeling: Each expert model outputs labeling results independently for the same input sample, without sharing their conclusions, in order to reduce mutual interference.

[0262] (2) Consistency verification: The system summarizes the first-stage output results of each expert model and generates candidate aggregation results; then, the original samples and candidate aggregation results are sent back to each expert model, asking them to judge whether to accept the candidate results and to explain the conflict items. In this way, cross-checking between expert models is achieved.

[0263] Based on this, the system performs consensus voting to determine the final labeling results. Specifically, majority consensus voting is performed for risk level, source modality, and risk type respectively:

[0264]

[0265]

[0266]

[0267] Mode is the mode function used to count the number of occurrences.

[0268] Define a consistent subset:

[0269]

[0270] Then, the continuous risk scores across the six dimensions are aggregated using a weighted aggregation method within a consistent subset:

[0271]

[0272] This yields the sample. Final annotation results:

[0273]

[0274] Therefore, this method does not rely on automatic labeling of a single model, but instead uses multiple mainstream commercial model application programming interfaces for parallel labeling, cross-verification, and consistency voting, ultimately outputting high-quality risk scores, source modalities, and risk type labeling results.

[0275] This step results in a high-quality risk dataset:

[0276]

[0277] Step S303: Multi-task supervised fine-tuning of the multimodal risk quantification model

[0278] After obtaining a high-quality labeled dataset, the system constructs a multimodal base model to be fine-tuned. The multimodal base model includes a visual encoder, a text encoder, a cross-modal fusion layer, and an output decoding layer, which are used to receive image, text, or image-text combination inputs and output structured risk assessment results.

[0279] This invention employs Supervised Fine-Tuning (SFT) to perform multi-task joint training on the multimodal base model. For the training samples... Construct a unified supervision objective:

[0280]

[0281] in, It can represent the structured criterion number or criterion set that was hit.

[0282] In one implementation, the model output can be represented as:

[0283]

[0284] in: This is the predicted value for the six-dimensional risk score; This is the predicted value for the overall risk level; The predicted values ​​are from the source modal; For risk type predictions; This refers to either the criterion matching result or the criterion interpretation result.

[0285] To enable the model to learn risk quantification, risk classification, and risk attribution simultaneously, this invention sets up supervised fine-tuning as a joint optimization of multiple tasks, including: a six-dimensional risk score prediction task; a risk level classification task; a source modality classification task; a risk type classification task; and a criterion matching task.

[0286] Correspondingly, the joint loss function can be written as:

[0287]

[0288] in: The six-dimensional risk score loss is used to constrain the continuous scores output by the model to approximate the risk score after expert consensus voting. Losses are classified according to risk level; For source modality classification loss; Losses are categorized by risk type; Matching loss to the criterion; The weights for each task.

[0289] In a specific implementation, the six-dimensional risk scoring loss Mean squared error loss is used, cross-entropy loss is used for classification tasks, and sequence generation loss is used for criterion tasks.

[0290] Furthermore, to improve the model's ability to learn structured outputs, the training samples use a unified structured target template, for example:

[0291]

[0292] By conducting supervised fine-tuning through multiple tasks, the model no longer only outputs coarse-grained results of "safe / unsafe," but can output continuous quantitative scores, source modality judgments, risk type judgments, and corresponding criteria, thereby obtaining a stronger and more refined assessment capability.

[0293] Step S304: Multimodal Risk Quantification Assessment System and Implementation

[0294] After completing the supervised fine-tuning, the fine-tuned multimodal risk quantification model is deployed to the online assessment system. This system receives the samples to be assessed. It outputs a six-dimensional risk score vector, overall risk level, source modality, risk type, and hit criterion information.

[0295] In a specific implementation example, the system first uses the fine-tuned model to obtain:

[0296]

[0297] in:

[0298]

[0299] The system can also calculate the overall risk score based on the six-dimensional risk assessment:

[0300]

[0301] in, This indicates the weight of each dimension in the risk quantification assessment. Based on the overall risk score. The system can further classify risk levels.

[0302] A key feature of this method is that the system can still perform evaluations based on structured criteria for unseen samples. In other words, even if the risk type corresponding to the sample is not sufficiently present in the training data, the system can still output a risk quantification assessment result as long as the sample semantically satisfies certain triggering conditions in the criterion library.

[0303] In one implementation case, let the confidence level of the risk type output by the model be:

[0304]

[0305] If the following conditions exist:

[0306]

[0307] If the model lacks sufficiently high confidence for all existing risk types, but the sample shows a high degree of matching with one or more structured criteria in the criterion library, then the system evaluates based on the criterion matching results. Let the sample and criteria... The matching function is:

[0308]

[0309] when

[0310]

[0311] When the sample is identified as an "unseen sample that can be evaluated according to the criteria", the system will output its six-dimensional risk score, overall risk level and corresponding criterion description.

[0312] In other words, the evaluation of unseen samples in this invention does not rely entirely on existing risk type labels, but rather on the structured criteria library established in step S101. Therefore, it has the ability to evaluate new forms of risk expression, new forms of graphic-text combination attacks, and new forms of misleading content.

[0313] Furthermore, to form a closed-loop optimization mechanism, the system adds the high-confidence unseen samples obtained from the online evaluation to the existing risk dataset. Let the set of newly added samples be:

[0314]

[0315] The updated risk dataset is then:

[0316]

[0317] After the risk dataset is updated, steps S102 to S104 can be executed again to perform a new round of iterative optimization of the model. This forms a cyclical iterative mechanism of "automatic evaluation - sample return - re-labeling - supervised fine-tuning again - continued evaluation", which continuously improves the model's risk quantification and assessment capabilities.

[0318] It should be noted that the multimodal risk quantification assessment method provided in this embodiment of the invention can be executed by an electronic device, a apparatus, or a control module within that apparatus for executing the method. This embodiment of the invention uses an apparatus executing the method as an example to illustrate the multimodal risk quantification assessment apparatus provided in this embodiment of the invention.

[0319] Figure 4 This is a schematic diagram of the structure of the multimodal risk quantification assessment device provided in Embodiment 4 of the present invention. The multimodal risk quantification assessment device 100 includes a criterion generation module 10, a risk labeling module 20, a supervision and fine-tuning module 30, and a risk assessment module 40, wherein:

[0320] The criterion generation module 10 is used to generate multimodal structured risk assessment criteria based on a predetermined set of risk dimensions for multimodal risk quantification assessment and store them in the criterion library.

[0321] The risk labeling module 20 is used to call multiple expert models to perform parallel labeling of the first multimodal risk sample to be labeled, generating a risk dataset with structured labels.

[0322] The supervision and fine-tuning module 30 is used to perform multi-task supervision and fine-tuning of the multimodal basic model based on the risk dataset to obtain the corresponding multimodal risk quantification assessment model.

[0323] The risk assessment module 40 is used to deploy a multimodal risk quantification assessment model to the online assessment system and perform a quantification assessment of the second multimodal risk sample to be assessed based on the criterion library.

[0324] Preferably, the criterion generation module 10 further includes:

[0325] The dimension definition submodule is used to define the set of risk dimensions for multimodal risk quantification assessment. The risk dimension set includes multiple risk quantification assessment dimensions.

[0326] The criteria generation submodule is used to generate risk assessment criteria under different dimensions based on the risk dimension set, including image criteria, text criteria, and / or image-text combination criteria;

[0327] The criteria structuring submodule is used to map each risk assessment criterion into a structured risk assessment criterion that includes triggering conditions, exclusion conditions, scoring mapping functions, and / or output interpretations;

[0328] The criteria entry submodule is used to store structured risk assessment criteria into the criteria database.

[0329] Preferably, the dimension definition submodule is further used for:

[0330] Define a set of risk dimensions D for multimodal risk quantification assessment;

[0331]

[0332] The risk dimension set D includes multiple risk quantification assessment dimensions d, where, This indicates the concealment of malicious intent. Indicates content executability. Indicates the scope of influence. Indicating sensory concreteness and impact, Indicates the text's attitude or inclination. Indicate the truthfulness or misleading nature of the content;

[0333] Preferably, the criterion generation submodule is further used for:

[0334] For any sample x to be evaluated, it can be expressed as:

[0335]

[0336] Where I represents image input and T represents text input;

[0337] Based on the sample x to be evaluated, a six-dimensional risk score vector needs to be output:

[0338]

[0339] in, , representing the score of the sample x to be evaluated on the i-th risk quantification assessment dimension;

[0340] Preferably, the criterion structured submodule is further used for:

[0341] Let the modal set be:

[0342]

[0343] in, Represents image modality, Represents text modality, Represents a combined text and image mode;

[0344] For each dimension of risk quantification assessment For each modal form m, construct the structured risk assessment criterion C:

[0345]

[0346] in: For the criteria identification; As a dimension for risk quantification assessment; For applicable modal types; As a triggering condition; To exclude conditions; For rating mapping function; For explaining the template;

[0347] The criteria entry submodule further includes:

[0348] Store structured risk assessment criteria in the criteria database. :

[0349] .

[0350] Preferably, the risk labeling module further includes:

[0351] The annotation submodule calls multiple expert models to perform parallel annotation of the first multimodal risk sample to be annotated, generating structured annotations.

[0352] The aggregation submodule summarizes the structured annotations, generates candidate aggregation results, and sends them to multiple expert models.

[0353] The judgment submodule uses multiple expert models to determine whether to accept the candidate aggregation result;

[0354] The voting submodule performs a consistency vote on the candidate aggregation results to determine the final annotation results;

[0355] The collection submodule generates a risk dataset with final labeled results.

[0356] Preferably, the annotation submodule is further used for:

[0357] Multiple expert models are invoked for the first multimodal risk sample X to be labeled;

[0358] ;

[0359] For each sample Multiple expert models are called as an expert model set for this sample. Perform parallel evaluation;

[0360] Suppose there are multiple expert models:

[0361]

[0362] Each expert model Receive the same input samples And a unified annotation prompt template, and output samples. Structured annotation:

[0363]

[0364] in: Indicates the first A six-dimensional risk score is given by an expert model; This indicates the overall risk level given by the expert model; This indicates the risk source modality, with values ​​ranging from image, text, or a combination of image and text. Indicates the type of risk; Indicates confidence level; This indicates explanatory information or the basis for judgment;

[0365] Preferably, the voting submodule is further used for:

[0366] Risk level Source modality and risk type Perform majority voting separately:

[0367] ;

[0368] ;

[0369] ;

[0370] Where Mode is the mode function used to count the number of occurrences;

[0371] Define a consistent subset:

[0372]

[0373] Then, the continuous risk scores across the six dimensions are aggregated using a weighted aggregation method within a consistent subset:

[0374]

[0375] This yields the sample. Final annotation results

[0376] ;

[0377] Preferably, the collection submodule generates a risk dataset D with final annotation results:

[0378] .

[0379] Preferably, the monitoring and fine-tuning module is further used for:

[0380] The multimodal base model is fine-tuned under multi-task supervision based on the risk dataset.

[0381] For training samples Construct a unified supervision objective:

[0382]

[0383] in, It can represent the structured criterion number or criterion set that was hit;

[0384] Generate the corresponding multimodal risk quantification assessment model;

[0385]

[0386] in: This is the predicted value for the six-dimensional risk score; This is the predicted value for the overall risk level; The predicted values ​​are from the source modal; For risk type predictions; This refers to either the criterion matching result or the criterion interpretation result.

[0387] Preferably, the risk assessment module is further used for:

[0388] Deploy a multimodal risk quantification assessment model into an online assessment system;

[0389] The online assessment system receives the second multimodal risk sample to be assessed. ;

[0390] The results obtained through the multimodal risk quantification assessment model are as follows:

[0391]

[0392] in:

[0393]

[0394] The overall risk score is calculated based on the six-dimensional risk assessment:

[0395]

[0396] in, This indicates the weight of each dimension in the quantitative risk assessment.

[0397] For unseen samples, the system can still perform evaluations based on structured criteria:

[0398] Let the confidence level of the risk type output by the model be:

[0399]

[0400] If the following conditions exist:

[0401]

[0402] Given samples and criteria The matching function is:

[0403]

[0404] when When the sample is identified as an unseen sample that can be evaluated according to the criteria, its six-dimensional risk score, overall risk level, and corresponding criterion description are output.

[0405] Add the high-confidence unseen samples obtained from the online assessment to the existing risk dataset;

[0406] Let the new sample set be:

[0407] The updated risk dataset is then: .

[0408] The multimodal risk quantification assessment device provided in this embodiment of the invention can achieve Figures 1-3 The various processes implemented in the example of the multimodal risk quantification assessment method shown are not described in detail here to avoid repetition.

[0409] The present invention also provides a storage medium for storing, for example, Figures 1-3Any computer program based on a multimodal risk quantification assessment method. For example, computer program instructions, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the present invention through the operation of the computer, achieving the same technical effect. To avoid repetition, further details are omitted here. The program instructions invoking the methods of the present invention may be stored in a fixed or removable storage medium, and / or transmitted via data streams in broadcast or other signal carrying media, and / or stored in the storage medium of a computer device operating according to the program instructions.

[0410] According to one embodiment of the present invention, the present invention also provides such a Figure 5 The illustrated electronic device 400 may optionally include a storage medium 200 for storing a computer program and a processor 300 for executing the computer program. When the computer program is executed by the processor 300, it implements any of the aforementioned multimodal risk quantification assessment methods, triggering the electronic device 400 to execute methods and / or technical solutions based on the foregoing embodiments, achieving the same technical effect. To avoid repetition, further details are omitted here. It should be noted that the electronic devices in this embodiment include mobile electronic devices and non-mobile electronic devices. For example, mobile electronic devices may be mobile phones, tablets, laptops, handheld computers, in-vehicle electronic devices, wearable devices, super mobile personal computers, netbooks, or personal digital assistants, etc., while non-mobile electronic devices may be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This embodiment does not specifically limit the scope of the invention.

[0411] It should be noted that the present invention can be implemented in software and / or a combination of software and hardware, for example, using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In one embodiment, the software program of the present invention can be executed by a processor to implement the steps or functions described above. Similarly, the software program of the present invention (including associated data structures) can be stored in a computer-readable recording medium, such as RAM memory, a magnetic or optical drive, a floppy disk, or similar devices. Furthermore, some steps or functions of the present invention can be implemented in hardware, for example, as circuitry that works with a processor to perform the various steps or functions.

[0412] This invention can be implemented on a computer as a computer-based method, or in dedicated hardware, or a combination of both. Executable code or portions thereof for the method according to the invention can be stored on a computer program product. Examples of computer program products include memory devices, optical storage devices, integrated circuits, servers, online software, etc. Optionally, the computer program product includes non-transitory program code components stored on a computer-readable medium so as to execute the method according to the invention when the program product is executed on a computer.

[0413] In an optional embodiment, the computer program includes computer program code components adapted to perform all the steps of the method according to the invention when the computer program is run on a computer. Optionally, the computer program is embodied on a computer-readable medium.

[0414] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of the present invention is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0415] Of course, the present invention may have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A multimodal risk quantification assessment method, characterized in that, Includes the following steps: The criteria generation step involves generating multimodal structured risk assessment criteria based on a predetermined set of risk dimensions for multimodal risk quantification assessment and storing them in the criteria library. The risk labeling step involves calling multiple expert models to perform parallel labeling of the first multimodal risk sample to be labeled, generating a risk dataset with structured labels. The supervised fine-tuning step involves performing multi-task supervised fine-tuning on the multimodal base model based on the risk dataset to obtain the corresponding multimodal risk quantification assessment model. The risk assessment step involves deploying the multimodal risk quantification assessment model to the online assessment system and performing a quantification assessment on the second multimodal risk sample to be assessed based on the criterion library.

2. The multimodal risk quantification assessment method according to claim 1, characterized in that, The criterion generation step further includes: The dimension definition sub-step defines the set of risk dimensions used for multimodal risk quantification assessment, wherein the set of risk dimensions includes multiple risk quantification assessment dimensions; The criteria generation sub-step generates risk assessment criteria under different dimensions based on the set of risk dimensions, including image criteria, text criteria, and / or image-text combination criteria. The criteria structuring sub-step maps each of the aforementioned risk assessment criteria to a structured risk assessment criterion that includes triggering conditions, exclusion conditions, scoring mapping functions, and / or output interpretations. The standard entry sub-step involves storing the structured risk assessment standards into the standard database.

3. The multimodal risk quantification assessment method according to claim 2, characterized in that, The dimension definition sub-step further includes: Define the set of risk dimensions D for multimodal risk quantification assessment; The risk dimension set D includes multiple risk quantification assessment dimensions d, wherein, This indicates the concealment of malicious intent. This indicates that the content is executable. Indicates the scope of the influence. This indicates the sensory concreteness and impact. This indicates the attitude or inclination expressed in the text. This indicates whether the content is true or misleading; The criterion generation sub-step further includes: For any sample x to be evaluated, it can be expressed as: Where I represents image input and T represents text input; Based on the sample x to be evaluated, a six-dimensional risk score vector needs to be output: in, , representing the score of the sample x to be evaluated on the i-th dimension of the risk quantification assessment; The criterion structuring sub-step further includes: Let the modal set be: in, Represents image modality, Represents text modality, Represents a combined text and image mode; For each of the aforementioned risk quantification assessment dimensions For each modal form m, construct the structured risk assessment criterion C: in: For the criteria identification; As a dimension for risk quantification assessment; For applicable modal types; The triggering condition; The exclusion condition; The rating mapping function; For explaining the template; The criterion entry sub-step further includes: The structured risk assessment criteria are stored in the criteria database. : 。 4. The multimodal risk quantification assessment method according to claim 1, characterized in that, The risk labeling step further includes: The annotation sub-step involves calling the multiple expert models to perform parallel annotation of the first multimodal risk sample to be annotated, generating the structured annotation; The aggregation sub-step involves summarizing the structured annotations to generate candidate aggregation results, which are then sent to the multiple expert models. The judgment sub-step involves the multiple expert models determining whether to accept the candidate aggregation result. The voting sub-step performs a consensus vote on the candidate aggregation results to determine the final annotation results; The set sub-step generates the risk dataset with the final labeled results.

5. The multimodal risk quantification assessment method according to claim 4, characterized in that, The annotation sub-step further includes: The multiple expert models are invoked for the first multimodal risk sample X to be labeled; ； For each sample The multiple expert models are called as an expert model set to analyze the sample. Perform parallel evaluation; Let the multiple expert models be: Each of the expert models The sample received the same input And a unified annotation prompt template, and output the sample. The structured annotations mentioned above: in: Indicates the first A six-dimensional risk score is given by an expert model; This indicates the overall risk level given by the expert model; This indicates the risk source modality, with values ​​ranging from image, text, or a combination of text and images. Indicates the type of risk; Indicates confidence level; This indicates explanatory information or the basis for judgment; The voting sub-step further includes: Risk level Source modality and risk type Perform majority voting separately: ； ； ； Where Mode is the mode function used to count the number of occurrences; Define a consistent subset: Then, the continuous risk scores across the six dimensions are aggregated using a weighted aggregation method within a consistent subset: The sample was thus obtained. The final annotation results ； The set sub-step generates the risk dataset D with the final labeled results: 。 6. The multimodal risk quantification assessment method according to claim 1, characterized in that, The monitoring and fine-tuning step further includes: The multimodal base model is fine-tuned under multi-task supervision based on the risk dataset. For training samples Construct a unified supervision objective: in, It can represent the structured criterion number or criterion set that was hit; Generate the corresponding multimodal risk quantification assessment model; in: This is the predicted value for the six-dimensional risk score; This is the predicted value for the overall risk level; The source modal prediction value; For risk type predictions; This refers to either the criterion matching result or the criterion interpretation result.

7. The multimodal risk quantification assessment method according to claim 1, characterized in that, The risk assessment step further includes: Deploy the multimodal risk quantification assessment model to the online assessment system; The online assessment system receives the second multimodal risk sample to be assessed. ; The multimodal risk quantification assessment model yields the following: in: The overall risk score is calculated based on the six-dimensional risk assessment: in, This indicates the weight of each dimension in the quantitative risk assessment. For unseen samples, the system can still perform evaluations based on structured criteria: Let the confidence level of the risk type output by the model be: If the following conditions exist: Given samples and criteria The matching function is: when When the sample is identified as an unseen sample that can be evaluated according to the criteria, its six-dimensional risk score, overall risk level, and corresponding criterion description are output. The high-confidence unseen samples obtained from the online assessment are added to the existing risk dataset. Let the new sample set be: The updated risk dataset is then: 。 8. A multimodal risk quantification assessment device constructed based on the method described in any one of claims 1 to 7, characterized in that, The device includes: The criteria generation module is used to generate multimodal structured risk assessment criteria based on a predetermined set of risk dimensions for multimodal risk quantification assessment and store them in the criteria library; The risk labeling module is used to call multiple expert models to perform parallel labeling of the first multimodal risk sample to be labeled, generating a risk dataset with structured labels; The supervised fine-tuning module is used to perform multi-task supervised fine-tuning of the multimodal base model based on the risk dataset to obtain the corresponding multimodal risk quantification assessment model. The risk assessment module is used to deploy the multimodal risk quantification assessment model to the online assessment system and perform quantification assessment on the second multimodal risk sample to be assessed based on the criterion library.

9. A storage medium, characterized in that, Used to store a computer program for performing the method according to any one of claims 1 to 7.

10. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

Images