A Risk Content Identification Method Based on Multimodal Large Model

Abstract

This invention relates to the field of artificial intelligence technology and provides a risk content identification method based on a multimodal large model. The method includes: identifying forged parts in audio; extracting background noise features and high-frequency features from a target face image and inputting them into an image segmentation model to locate the forged region; segmenting the target face image into blocks, and inputting the resulting local region images and global images into a visual encoder to extract visual features; calculating the attention of text features and global image features to local image features, discarding local image features with low attention; and inputting the outputs of the audio segmentation model and image segmentation model, image features, and questions into a large language model to summarize the risk points. This invention improves the accuracy and robustness of risk identification by integrating multiple data sources and using a multimodal large model for risk identification. It can also effectively counter various fraud methods and solves the problems of existing technologies being unable to handle multimodal data and lacking interpretability.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for identifying risky content based on a multimodal large model. Background Technology

[0002] Traditional single-modal identification methods, such as text-based or voice-feature-based identification, are increasingly showing their limitations in the face of ever-evolving and sophisticated fraud tactics. They struggle to effectively combat scams using SMS, telephone calls, fake voices, and fake facial videos. Against this backdrop, there is an urgent need to explore more efficient and accurate methods for identifying risky content. Multimodal large-scale model-based risk content identification technology is gaining increasing attention and demonstrating significant application potential.

[0003] Traditional risk identification methods primarily rely on single-modal information, such as text, speech, and images, and employ techniques like rule engines and machine learning for identification. However, this approach has the following limitations:

[0004] (1) Limited information dimension, easily bypassed: Traditional risk identification methods only focus on information of a single modality, ignoring the correlation and complementarity between information of different modalities. Fraudsters can easily bypass the identification system by changing information of one modality. For example, in voice fraud, fraudsters can easily change their tone and intonation to evade identification based on voice features.

[0005] (2) Insufficient generalization ability, making it difficult to deal with new types of fraud: Traditional risk identification methods usually rely on manually formulated rules or patterns learned from historical data. Their ability to identify new types of fraud that have never appeared before is limited. For example, traditional image or video-based identification methods are often difficult to effectively identify fake face videos generated using deepfake technology.

[0006] (3) Lack of interpretability, difficulty in tracing and optimization: Traditional risk identification methods usually only provide the final identification result, but cannot explain the identification basis and reasoning process. This makes it difficult for users to understand the reliability of the identification result, and also hinders technicians from optimizing and improving the model.

[0007] This shows that traditional single-modal identification methods are no longer sufficient in the face of increasingly complex online fraud methods.

[0008] Therefore, overcoming the shortcomings of the existing technology is an urgent problem to be solved in this technical field. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to provide a method for identifying risky content based on a multimodal large model.

[0010] The present invention adopts the following technical solution:

[0011] In a first aspect, the present invention provides a risk content identification method based on a multimodal large model, the method comprising:

[0012] In step 201, the audio is input into the audio segmentation model LATDL. The audio segmentation model LATDL is used to convert the audio into text, identify the fake parts, and output the corresponding audio content.

[0013] In step 202, the target face image is processed to extract background noise features and high-frequency features, which are then input into the image segmentation model ForgerySAM to locate the fake regions in the target face image.

[0014] In step 203, a series of aspect ratios are predefined, including one or more of 1:1, 1:2, 1:3, 2:3, 3:4, 1:12, 2:1, 3:1, 3:2, 4:3 and 12:1;

[0015] In step 204, the aspect ratio of the target face image is calculated, and the closest aspect ratio is selected from the predefined aspect ratios. The target face image is then adjusted to the predefined resolution using interpolation to make it conform to the selected aspect ratio. The aspect ratio is a multiple of a fixed-size sliding window. The target face image is then segmented using the sliding window to obtain local region images. At the same time, the resolution of the target face image is adjusted to the size of the sliding window in order to obtain global visual information.

[0016] In step 205, the local region image and global image obtained by segmentation are input into a visual encoder to extract visual features; wherein, the visual features include local image features and global image features; wherein, the global image is the target face image;

[0017] In step 206, text features, local image features, and global image features are input into the large language model, so that the large language model calculates the attention of text features and global image features to local image features, and discards local image features with attention less than a preset value.

[0018] In step 207, the outputs of the audio segmentation model and the image segmentation model, along with image features and problems, are input into the large language model. The large language model summarizes the risk points and provides risk analysis and suggestions.

[0019] Preferably, the step of inputting the audio into the audio segmentation model LATDL, which is used to convert the audio into text, identify the fake parts, and output the corresponding audio content, specifically includes:

[0020] The audio segmentation model first uses a pre-trained wav2vec2 model to perform preliminary feature extraction on the input audio, converting the audio into a feature vector sequence containing rich information; wherein, the feature vector sequence captures the frequency changes, timbre features and temporal structure of the audio;

[0021] The audio segmentation model uses a multi-layer one-dimensional convolutional neural network ResNet to further extract deep features of the audio. Specifically, a local attention mechanism is introduced into the audio segmentation model, using the extracted timbre features as query Q and key K, and the feature vector sequence extracted by wav2vec2 as value V, so that the model can further capture the semantic features of the audio through the attention mechanism.

[0022] The audio segmentation model uses a multilayer perceptron to further process and fuse the extracted timbre and semantic features to generate a unified feature representation;

[0023] Train a classifier to classify each frame of audio based on the fused features, and output its forgery probability and the character corresponding to the audio, thereby obtaining the audio content and the forgery probability.

[0024] Preferably, the step of processing the target face image to extract background noise features and high-frequency features, and inputting them into the image segmentation model ForgerySAM to locate the forged region in the target face image, specifically includes:

[0025] In the acquired target face image, the noise feature and the high-frequency feature in the target face image are obtained through two steps: noise extraction and high-frequency information extraction.

[0026] The pre-processed target face image, background noise features, and high-frequency features are input into the image segmentation model to obtain the segmentation result of the fake region.

[0027] Preferably, in the acquired target face image, the noise feature and the high-frequency feature in the target face image are obtained through two steps: noise extraction and high-frequency information extraction. Specifically, this includes:

[0028] The image segmentation model first uses a self-supervised learning method to extract the background noise features in the target face image; the image segmentation model also uses a high-pass filter to extract the high-frequency features in the target face image.

[0029] Preferably, the step of inputting the preprocessed target face image, background noise features, and high-frequency features into the image segmentation model to obtain the segmentation result of the forged region specifically includes:

[0030] The image segmentation model utilizes a multi-condition input efficient fine-tuning architecture to fuse extracted background noise features and high-frequency features. The multi-condition input efficient fine-tuning architecture employs a weight sharing mechanism to accurately map the embedded features of multiple conditions to the same latent space, and utilizes a cross-attention mechanism to efficiently fuse background noise features and high-frequency features.

[0031] Lora was also used to efficiently fine-tune the SAM model to adapt it to the task of image forgery detection;

[0032] The image segmentation model uses a fine-tuned SAM model to segment the target face image, marking each pixel in the target face image as a fake region or a normal region to obtain the segmentation result;

[0033] Based on the segmentation results, if the segmentation results show that there is a fake region, the image segmentation model determines that the target face image is a fake image. Finally, the image segmentation module outputs the judgment of whether the target face image is fake and the coordinates of the fake region of the target face image.

[0034] Preferably, the aspect ratio of the target face image is calculated by selecting the closest predefined aspect ratio, adjusting the target face image to a predefined resolution using interpolation to match the selected aspect ratio, where the length and width are multiples of a fixed-size sliding window, and then using the sliding window to segment the target face image into local region images. Simultaneously, the resolution of the target face image is adjusted to the size of the sliding window to obtain global visual information. Specifically, this includes:

[0035] Calculate the aspect ratio of the target face images at different resolutions and select the predefined aspect ratio that is closest to the aspect ratio of the target face image as the final aspect ratio;

[0036] The image resolution is adjusted to a multiple of 448 using interpolation methods;

[0037] The uniformly adjusted target face image is segmented into local region images with a resolution of 448×448 using a sliding window of fixed size;

[0038] Meanwhile, the resolution of the target face image is adjusted to 448×448 to match the resolution of the local region image. The local region image with the adjusted resolution of 448×448 contains the local visual information to be processed, while the target face image adjusted to 448×448 provides global visual information.

[0039] Preferably, the step of inputting the local region image and global image obtained from the segmentation into the visual encoder to extract visual features specifically includes:

[0040] After feature extraction by the visual encoder, each block of the local region image and the global image is encoded into a feature vector of length 256.

[0041] These features are then processed by a two-layer perceptron to align the feature vectors into the feature space of the large language model, resulting in the corresponding visual features.

[0042] Preferably, the step of inputting text features, local image features, and global image features into a large language model, enabling the large language model to calculate the attention of text features and global image features to local image features, and discarding local image features with attention less than a preset value, specifically includes:

[0043] Text features, local image features, and global image features are pre-input into the first two layers of the large language model;

[0044] The first two layers of the model calculate the attention of text features and global image features to local image features, and discard local image features with attention less than a preset value.

[0045] Preferably, the outputs of the audio segmentation model and the image segmentation model, along with image features and problems, are input into the large language model. The large language model summarizes the risk points and provides risk analysis and suggestions, specifically including:

[0046] Text features, cropped local image features, and global image features are re-inputted into the large language model, enabling the large language model to summarize and analyze risky content and provide recommendations.

[0047] Preferably, the audio is preprocessed and denoised before being input into the audio segmentation model LATDL.

[0048] Secondly, the present invention also provides a risk content identification device based on a multimodal large model, used to implement the risk content identification method based on a multimodal large model described in the first aspect, the device comprising:

[0049] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor for performing the risk content identification method based on a multimodal large model as described in the first aspect.

[0050] Thirdly, the present invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors to perform the method described in the first aspect.

[0051] Fourthly, a chip is provided, comprising: a processor and an interface for calling and running a computer program stored in memory, performing the method as described in the first aspect.

[0052] Fifthly, a computer program product containing instructions is provided that, when executed on a computer or processor, causes the computer or processor to perform the method as described in the first aspect.

[0053] This invention effectively integrates multiple data sources and uses a risk content identification method based on a multimodal large model to identify risks. This not only improves the accuracy and robustness of identification, but also effectively addresses various new types of fraud such as SMS, chat, web page, and video fraud, thus effectively safeguarding social security. Furthermore, it solves the problems of existing technologies being unable to handle multimodal data and lacking interpretability. Attached Figure Description

[0054] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0055] Figure 1 This is a flowchart illustrating a risk content identification method based on a multimodal large model provided in an embodiment of the present invention;

[0056] Figure 2 This is a schematic diagram of a risk content identification method based on a multimodal large model provided in an embodiment of the present invention;

[0057] Figure 3 This is a schematic diagram of a risk content identification method based on a multimodal large model provided in an embodiment of the present invention;

[0058] Figure 4 This is a schematic diagram of a risk content identification method based on a multimodal large model provided in an embodiment of the present invention;

[0059] Figure 5 This is a schematic diagram of a risk content identification method based on a multimodal large model provided in an embodiment of the present invention;

[0060] Figure 6 This is a schematic diagram of a risk content identification method based on a multimodal large model provided in an embodiment of the present invention;

[0061] Figure 7 This is a schematic diagram of the architecture of a risk content identification device based on a multimodal large model provided in an embodiment of the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0063] Unless the context otherwise requires, throughout the specification and claims, the term "comprising" is interpreted as openly inclusive, meaning "including, but not limited to." In the description of the specification, terms such as "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this disclosure. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics mentioned may be included in any suitable manner in any one or more embodiments or examples; that is, although they may be incorporated into embodiments or examples using the above terms for reasons such as order and position, it does not limit them to be incorporated in combination by a single embodiment or example.

[0064] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more. Furthermore, for example, the description may use the prefix "A" or "B" to describe the same type of nouns as two independent entities. In this case, the corresponding features defined with "A" and "B" are used only to distinguish between similar entities and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features.

[0065] In the description of this invention, the expression “A and / or B” (where A and B are used to formally represent specific features) will be used. The corresponding expression includes the following three combinations: only A, only B, and a combination of A and B.

[0066] As used in this invention, “about,” “approximately,” or “approximately” includes the stated value and the average value within an acceptable range of deviation from a particular value, wherein the acceptable range of deviation is determined by a person skilled in the art taking into account the measurement under discussion and the error associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0067] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0068] Example 1:

[0069] In recent years, with the rapid development of deep learning technology, multimodal large-scale models have made breakthroughs in fields such as natural language processing and computer vision. Compared with traditional single-modal models, multimodal large-scale models can comprehensively process information from different modalities, such as text, speech, images, and videos, and learn the complex relationships between different modal information. They offer the following advantages: Strong information fusion capability and more accurate recognition: Multimodal large-scale models can comprehensively analyze information from different modalities. For example, they can jointly analyze speech content and intonation changes with text content and video footage, thereby more comprehensively and accurately identifying risky content. Strong generalization ability and more effective response to new types of fraud: Multimodal large-scale models can learn deep semantic relationships between different modal information, possessing stronger generalization ability and effectively identifying and responding to new and complex fraud methods. A degree of interpretability: Multimodal large-scale models can analyze input information and provide explanations for potential fraudulent elements, making them more acceptable to the public. In other words, multimodal large models have comprehensive information processing capabilities and excellent interpretability. Based on this characteristic, Embodiment 1 of this invention provides a risk content identification method based on a multimodal large model, such as... Figure 1 As shown, the method includes:

[0070] In step 201, the audio is input into the audio segmentation model LATDL. LATDL converts the audio into text, identifies forged portions, and outputs the corresponding audio content. This step ensures that we can accurately locate and identify any forged information that may exist in the speech (i.e., audio). Before inputting the audio into LATDL, preprocessing and noise reduction are performed on the audio. This preprocessing may include format adjustments or cropping of the audio.

[0071] In step 202, the target face image is processed to extract background noise features and high-frequency features, which are then input into the image segmentation model ForgerySAM to locate the forged regions in the target face image; this process helps to identify tampered or forged parts of the image.

[0072] In step 203, a series of aspect ratios are predefined, including one or more of 1:1, 1:2, 1:3, 2:3, 3:4, 1:12, 2:1, 3:1, 3:2, 4:3 and 12:1; these ratios provide the basis for subsequent image processing.

[0073] In step 204, the aspect ratio of the target face image is calculated, and the closest aspect ratio is selected from the predefined aspect ratios. The target face image is then adjusted to a predefined resolution using interpolation to match the selected aspect ratio. The aspect ratio is a multiple of a fixed-size sliding window. The target face image is then segmented using the sliding window to obtain local region images. Simultaneously, the resolution of the target face image is adjusted to the size of the sliding window to obtain global visual information. The fixed size is determined by those skilled in the art based on experience.

[0074] In step 205, the local region image and global image obtained by slicing are input into the visual encoder to extract visual features; wherein, the visual features include local image features and global image features; wherein, the global image is the target face image; this step ensures that we can obtain rich visual information from both local and global perspectives.

[0075] In step 206, text features, local image features, and global image features are input into the large language model. The large language model calculates the attention of the text features and global image features to the local image features, discarding local image features with attention less than a preset value; wherein, the preset value is obtained by those skilled in the art based on experience. This process reduces the number of features input into the large language model, thereby reducing inference overhead.

[0076] In step 207, the outputs of the audio segmentation model and the image segmentation model, along with image features and problems, are input into the large language model. The large language model summarizes the risk points and provides risk analysis and suggestions. This comprehensive analytical capability enables the system to provide users with comprehensive risk identification and response strategies.

[0077] This embodiment effectively integrates multiple data sources and uses a risk content identification method based on a multimodal large model to identify risks. This not only improves the accuracy and robustness of identification, but also effectively addresses various new types of fraud such as SMS, chat, web page, and video fraud, effectively safeguarding social security. Furthermore, it solves the problems of existing technologies being unable to handle multimodal data and lacking interpretability, and provides a solid technical guarantee for national security.

[0078] Specifically, the audio is input into the audio segmentation model LATDL, which converts the audio into text, identifies fake parts, and outputs the corresponding audio content.

[0079] The audio segmentation model first uses a pre-trained wav2vec2 model to perform preliminary feature extraction on the input audio, converting the audio into a feature vector sequence containing rich information; wherein, the feature vector sequence captures the frequency changes, timbre features and temporal structure of the audio.

[0080] The audio segmentation model uses a multi-layer one-dimensional convolutional neural network, ResNet, to further extract deep features of the audio, especially fine-grained frame-level timbre information. The residual structure of ResNet effectively avoids the vanishing gradient problem during deep network training, ensuring the model's learning ability. To more accurately focus on key information in the audio, a local attention mechanism is introduced into the audio segmentation model. The extracted timbre features are used as the query Q and key K, and the feature vector sequence extracted by wav2vec2 is used as the value V. This allows the model to further capture the semantic features of the audio through the attention mechanism and focus on specific time periods or feature segments, thereby more effectively detecting subtle changes in forged audio.

[0081] The audio segmentation model uses a multilayer perceptron to further process and fuse the extracted timbre and semantic features to generate a unified feature representation.

[0082] Train a classifier to classify each frame of audio based on fused features, output its forgery probability and the corresponding character in the audio, and achieve accurate detection of the entire audio or part of the forged audio, thereby obtaining the audio content and the forgery probability.

[0083] In some embodiments, processing the target face image to extract background noise features and high-frequency features, and inputting them into the image segmentation model ForgerySAM to locate the forged region in the target face image, specifically includes:

[0084] In the acquired target face image, the background noise features and high-frequency features in the target face image are obtained through two steps: background noise extraction and high-frequency information extraction.

[0085] The preprocessed target face image, background noise features, and high-frequency features are input into the image segmentation model to obtain the segmentation result of the forged region. The preprocessing involves scaling and cropping the image according to a predefined aspect ratio set to adapt to different image sizes and content, ensuring the integrity of image information.

[0086] In one optional implementation, the step of obtaining background noise features and high-frequency features in the target face image through two steps—background noise extraction and high-frequency information extraction—specifically includes:

[0087] The image segmentation model first uses a self-supervised learning method to extract background noise features from the target face image. These background noise features refer to the unique imprints left by factors such as device and environment during image generation, containing information that is not easily detected but is crucial. The image segmentation model also uses a high-pass filter to extract high-frequency features from the target face image. Other image processing algorithms can also be used to extract high-frequency features. These high-frequency features include edge features, details, and texture features. This high-frequency information is easily altered during image forgery, becoming an important clue for determining the authenticity of an image.

[0088] Specifically, the step of inputting the preprocessed target face image, background noise features, and high-frequency features into the image segmentation model to obtain the segmentation result of the forged region includes:

[0089] The image segmentation model utilizes a multi-condition input efficient fine-tuning architecture to fuse extracted background noise features and high-frequency features. The multi-condition input efficient fine-tuning architecture employs a weight sharing mechanism to accurately map the embedded features of multiple conditions to the same latent space, and utilizes a cross-attention mechanism to efficiently fuse background noise features and high-frequency features.

[0090] The SAM (Segment Anything Model) model was also efficiently fine-tuned using the Low-Rank Adaptation of Large Language Models (Lora), a technique used for fine-tuning large language models, to adapt it to the task of image forgery detection.

[0091] The image segmentation model uses a fine-tuned SAM model to segment the target face image, marking each pixel in the target face image as a fake region or a normal region, and obtains the segmentation result.

[0092] Based on the segmentation results, if the segmentation results show that there is a fake region, the image segmentation model determines that the target face image is a fake image. Finally, the image segmentation module outputs the judgment of whether the target face image is fake and the coordinates of the fake region of the target face image.

[0093] In practical applications, the aspect ratio of the target face image is calculated by selecting the closest predefined aspect ratio and adjusting the target face image to a predefined resolution using interpolation to match the selected aspect ratio. The aspect ratio is a multiple of a fixed-size sliding window. The sliding window is used to segment the target face image into local regions. Simultaneously, the resolution of the target face image is adjusted to the size of the sliding window to obtain global visual information. Specifically, this includes:

[0094] Calculate the aspect ratio of the target face images at different resolutions and select the predefined aspect ratio that is closest to the aspect ratio of the target face image as the final aspect ratio.

[0095] The resolution of an image is adjusted to a multiple of 448 using interpolation methods; for example, assuming the final aspect ratio is 3:4, an image with a resolution of (300, 400) will be adjusted to a size of (3×448, 4×448).

[0096] The uniformly adjusted target face image is segmented into local region images with a resolution of 448×448 using a fixed-size sliding window.

[0097] Simultaneously, the resolution of the target face image is adjusted to 448×448 to match the resolution of the local region image. The adjusted 448×448 local region image contains the local visual information to be processed, while the adjusted 448×448 target face image provides global visual information. This combination of local and global image information ensures the model's comprehensive understanding of the original image.

[0098] In one optional implementation, the step of inputting the segmented local region image and global image into a visual encoder to extract visual features specifically includes:

[0099] After feature extraction by the visual encoder, each block of the local and global images is encoded into a feature vector of length 256.

[0100] These features are then processed by a two-layer perceptron to align the feature vectors into the feature space of the large language model, resulting in the corresponding visual features. Specifically, these feature vectors undergo dimensionality transformation through a two-layer perceptron, mapping the visual features to the same 2048-dimensional feature space as the text feature space of the large language model.

[0101] In a practical application scenario, the step of inputting text features, local image features, and global image features into a large language model, enabling the large language model to calculate the attention of text features and global image features to local image features, and discarding local image features with attention less than a preset value, specifically includes: pre-inputting text features, local image features, and global image features into the first two layers of the large language model; the first two layers of the model calculating the attention of text features and global image features to local image features, discarding local image features with attention less than a preset value, and retaining important local image features.

[0102] In one optional implementation, the outputs of the audio segmentation model and the image segmentation model, along with image features and questions, are input into a large language model. The large language model summarizes the risk points and provides risk analysis and suggestions. Specifically, this includes re-inputting text features, cropped local image features, and global image features into the large language model, enabling it to summarize and analyze the risk content and provide suggestions. This part is mainly achieved by designing prompt words, allowing the large language model to perform deep fusion and understanding based on the input multimodal features. By designing prompt words, the large language model ultimately outputs results containing analysis, summaries, and suggestions regarding the risk content.

[0103] Example 2:

[0104] With the rapid development of information technology, especially the widespread adoption of mobile internet and social media platforms, new types of fraud are emerging, and their methods are becoming more diverse and covert. Traditional single-modal identification methods, such as those based solely on text analysis, are proving inadequate when faced with fraud involving multiple formats, including text messages, conversations, web pages, and videos. These methods often only analyze a specific type of information and cannot effectively identify fraudulent activities across modalities, posing a significant challenge to preventing and combating new types of fraud.

[0105] To address the challenge of traditional single-modal recognition methods struggling to simultaneously combat emerging fraud tactics such as SMS, chatbot, webpage, and video scams, this invention provides a risk content recognition method based on a multimodal large-scale model. The aim is to improve fraud detection rates and provide users with more comprehensive security by leveraging the modal collaboration of image, voice, and text, along with the powerful understanding capabilities of the multimodal large-scale model. This effectively counters various emerging fraud tactics, including SMS, chatbot, webpage, and video scams, thus safeguarding social security and resolving the limitations of existing technologies in handling multimodal data and lacking interpretability. The core of this invention lies in utilizing the powerful understanding and dialogue capabilities of a large language model to construct a multimodal large-scale model that integrates image, voice, and text information, enabling dialogue and risk warnings for users.

[0106] To achieve the above objectives, this invention proposes a risk content identification method based on a multimodal large model, comprising the following steps:

[0107] Step 1: Spoofed Speech Recognition: First, the speech is input into the LATDL audio segmentation model. This model is responsible for converting the audio content into text, identifying the forged parts, and outputting the corresponding audio content. This step ensures that we can accurately locate and identify any forged information that may exist in the speech.

[0108] Step Two: Forged Region Identification: Next, the target face image is processed. By extracting background noise features and high-frequency features, these are input into the image segmentation model ForgerySAM to accurately locate forged regions in the image. This process helps identify altered or forged parts of the image.

[0109] Step 3: Predefine Image Resolution: In this step, we predefine a series of image aspect ratios, such as 1:1, 1:2, 3:4, 1:12, 4:3, 12:1, etc. These ratios provide the foundation for subsequent image processing.

[0110] Step 4: Dynamic Image Resolution Slicing: Calculate the aspect ratio of the original image and select the closest one from predefined aspect ratios. Adjust the image to the predefined resolution using interpolation to match the selected aspect ratio, with the dimensions being multiples of a fixed-size sliding window. Then, slice the image using the sliding window. Simultaneously, adjust the original image resolution to the sliding window size to capture global visual information.

[0111] Step 5: Visual Encoding: Input the segmented local and global images into the visual encoder to extract visual features. This step ensures that we can obtain rich visual information from both local and global perspectives.

[0112] Step Six: Dynamic Visual Feature Cropping: First, text features, local image features, and global image features are input into the large language model. The model calculates the attention of text features and global image features to local image features, discarding local image features with less attention. This process reduces the number of features input into the large language model, thereby reducing inference overhead.

[0113] Step Seven: Large-Scale Model Analysis and Summary: Finally, the outputs of the audio segmentation model and image segmentation model, along with image features and problems, are input into the large language model. The large language model summarizes the risk points and provides risk analysis and suggestions. This comprehensive analytical capability enables the system to provide users with comprehensive risk identification and response strategies.

[0114] In one embodiment of the present invention, step one specifically includes: the model first uses a pre-trained wav2vec2 model to perform preliminary feature extraction on the input audio, converting the audio into a sequence of feature vectors containing rich information. These vectors capture key information such as frequency variations, timbre features, and temporal structure of the audio. Next, the model uses a multi-layer one-dimensional convolutional neural network ResNet to further extract deep features of the audio data, especially fine-grained frame-level timbre information. The residual structure of ResNet effectively avoids the gradient vanishing problem in deep network training, ensuring the model's learning ability. To more accurately focus on key information in the audio, the model introduces a local attention mechanism. Using the extracted timbre features as query Q and key K, and the original audio features extracted by wav2vec2 as value V, the model can further capture the semantic information of the audio through the attention mechanism and focus on specific time periods or feature segments, thereby more effectively detecting subtle changes in fake audio. Finally, the model uses a multilayer perceptron to further process and fuse the extracted timbre and semantic features to generate a unified feature representation. Then, a classifier is trained to classify each frame of audio data based on the fused features, outputting its forgery probability and the corresponding character in the audio, thereby achieving accurate detection of the entire audio or part of the forged audio. Finally, through the LATDL audio segmentation model, we obtain the audio content and the forgery probability.

[0115] In one embodiment of the present invention, step two specifically includes: extracting image "noise": The model first uses a self-supervised learning method to extract "noise" from the image. This "noise" refers to the unique imprint left by factors such as device and environment during the image generation process, containing information that is not easily noticed but is crucial. Extracting high-frequency information from the image: Simultaneously, the model uses a high-pass filter or other advanced image processing algorithms to extract high-frequency information from the image, such as edge, detail, and texture features. This high-frequency information is easily altered during image forgery, becoming an important clue for judging the authenticity of the image. Feature fusion and fine-tuning: The model uses a "multi-condition input efficient fine-tuning architecture" to fuse the extracted "noise" and high-frequency information. This architecture adopts a weight sharing mechanism to accurately map the embedded features of multiple conditions to the same latent space, and uses a cross-attention mechanism to efficiently fuse "noise" and "high-frequency" features. In addition, the model also uses LoRa to efficiently fine-tune the SAM model to adapt it to the task of image forgery detection. Image segmentation: The model uses the fine-tuned SAM model to segment the image, marking each pixel in the image as a forged region or a normal region. Output: Based on the segmentation results, if the segmentation results show obvious forged regions, the model determines that the image is a forged image. The final image segmentation module outputs the determination of whether the image is forged and the coordinates of the forged regions.

[0116] In one embodiment of the present invention, step three specifically includes: First, obtaining the original size of the image, and calculating the target size for each aspect ratio according to a predefined set of aspect ratios (1:1, 1:2, 1:3, 2:3, 3:4, 2:1, 3:1, 3:2, 4:3). Next, comparing the difference between the target size and the original size, and selecting the aspect ratio with the smallest difference as the final aspect ratio. Finally, based on the final aspect ratio, using an interpolation algorithm, adjusting the image to a multiple of 448 for both length and width.

[0117] In one embodiment of the present invention, step four specifically includes: first, acquiring the image processed in step three; then, using a fixed-size sliding window, cutting the adjusted image into multiple image blocks with a resolution of 448x448, and also adjusting the original image to a resolution of 448×448 through interpolation.

[0118] In one embodiment of the present invention, step five specifically includes: inputting each 448x448 resolution image patch obtained in step four into a visual encoder for feature extraction, resulting in a feature vector of length 256 for each image patch. These feature vectors undergo dimensionality transformation through a two-layer perceptron, mapping the visual features to the same dimension of 2048 as the text feature space of the large language model.

[0119] In one embodiment of the present invention, step six specifically includes: inputting text features, global image features, and local image features into the first two layers of the large language model, respectively. The first two layers of the model calculate the attention weights of the text features and global image features to each local image feature. Based on the attention weights, local image features with smaller attention weights are filtered out, while important local image features are retained.

[0120] In one embodiment of the present invention, step seven specifically includes: re-inputting the local image features, text features, and global image features selected in step six into the large language model. The large language model performs deep fusion and understanding based on the input multimodal features. By designing prompt words, the large language model ultimately outputs results containing analysis, summaries, and suggestions regarding risk content.

[0121] The multimodal risk content identification method described in this invention effectively integrates text, audio, and image information to achieve accurate identification of multimodal risk content. First, the method extracts the timbre and semantic features of audio using advanced audio processing technology and combines this with a fake audio detection model to obtain the audio content and the probability of forgery. Simultaneously, it utilizes self-supervised learning and image segmentation techniques to analyze the "background noise" and high-frequency information of the image, determining whether the image is forged and accurately locating the forged region. To integrate multimodal information, the method introduces a flexible image preprocessing step, scaling and cropping the image according to a predefined aspect ratio set to adapt to different image sizes and content, ensuring the integrity of image information. Then, a visual encoder extracts the features of image patches and maps them to the feature space of a large language model. Finally, the attention mechanism of the large language model filters important local image features, and the filtered multimodal features are input into the large language model for deep fusion and understanding, ultimately outputting an analysis, summary, and recommendations for risk content. This invention effectively solves the problem of insufficient understanding of multimodal data in traditional methods, significantly improving the accuracy and generalization ability of risk content identification.

[0122] Based on Embodiment 1 and the above method, the implementation process of the features of the present invention in specific application scenarios will be explained below using technical descriptions in relevant scenarios.

[0123] like Figure 2 As shown, Figure 2 The demonstration included: audio segmentation model, image segmentation model, visual encoder, and Big Prophet model. The entire recognition process included the following steps:

[0124] Step 1: Spoofed Speech Recognition: First, the speech is input into the LATDL audio segmentation model. This model is responsible for converting the audio content into text, identifying the forged parts, and outputting the corresponding audio content. This step ensures that we can accurately locate and identify any forged information that may exist in the speech.

[0125] Specifically, the audio segmentation model first uses a pre-trained wav2vec2 model to perform preliminary feature extraction on the input audio, converting it into a sequence of feature vectors rich in information. These vectors capture key information such as frequency variations, timbre features, and temporal structure. Next, the model uses a multi-layer one-dimensional convolutional neural network, ResNet, to further extract deeper features from the audio data, particularly fine-grained frame-level timbre information. The residual structure of ResNet effectively avoids the gradient vanishing problem in deep network training, ensuring the model's learning ability. To more accurately focus on key information in the audio, the model introduces a local attention mechanism. Using the extracted timbre features as query Q and key K, and the original audio features extracted by wav2vec2 as value V, the model can further capture the semantic information of the audio through the attention mechanism and focus on specific time periods or feature segments, thereby more effectively detecting subtle changes in fake audio. Finally, the model uses a multilayer perceptron to further process and fuse the extracted timbre and semantic features, generating a unified feature representation. Then, a classifier is trained to classify each frame of audio data based on the fused features, outputting its forgery probability and the corresponding character in the audio, thereby achieving accurate detection of the entire audio or part of the forged audio. Finally, through the LATDL audio segmentation model, we obtain the audio content and the forgery probability.

[0126] Step Two: Forged Region Identification: Next, the target face image is processed. By extracting background noise features and high-frequency features, these are input into the image segmentation model ForgerySAM to accurately locate forged regions in the image. This process helps identify altered or forged parts of the image.

[0127] Specifically, the image segmentation model extracts image "noise": The model first uses a self-supervised learning method to extract the "noise" in the image. This "noise" refers to the unique imprint left by factors such as device and environment during the image generation process, containing information that is not easily noticed but is crucial. It then extracts high-frequency information: Simultaneously, the model uses high-pass filters or other advanced image processing algorithms to extract high-frequency information in the image, such as edge, detail, and texture features. This high-frequency information is easily altered during image forgery, becoming an important clue for judging the authenticity of an image. Feature fusion and fine-tuning: The model uses a "multi-condition input efficient fine-tuning architecture" to fuse the extracted "noise" and high-frequency information. This architecture employs a weight-sharing mechanism to accurately map the embedded features of multiple conditions to the same latent space and uses a cross-attention mechanism to efficiently fuse "noise" and "high-frequency" features. Furthermore, the model uses LoRa to efficiently fine-tune the SAM model, adapting it to the task of image forgery detection. Image segmentation: The model uses the fine-tuned SAM model to segment the image, marking each pixel in the image as a forged or normal region. Output: Based on the segmentation results, if the segmentation results show obvious forged regions, the model determines that the image is a forged image. The final image segmentation module outputs the determination of whether the image is forged and the coordinates of the forged regions.

[0128] Step 3: Predefine Image Resolution: In this step, we predefine a series of image aspect ratios, such as 1:1, 1:2, 3:4, 1:12, 4:3, 12:1, etc. These ratios provide the foundation for subsequent image processing.

[0129] Specifically, this step ensures that the model can handle high-resolution image inputs, with a maximum resolution of 1344×1792, and that the model can capture the detailed features of the image, thereby more accurately identifying forged areas.

[0130] Step 4: Dynamic Image Resolution Slicing: Calculate the aspect ratio of the original image and select the closest one from predefined aspect ratios. Adjust the image to the predefined resolution using interpolation to match the selected aspect ratio, with the dimensions being multiples of a fixed-size sliding window. Then, slice the image using the sliding window. Simultaneously, adjust the original image resolution to the sliding window size to capture global visual information.

[0131] Specifically, each image patch has a resolution of 448×448 to accommodate the input size of the visual encoder. The complementarity between the global and local images helps the multimodal large model to fully understand the image content.

[0132] Step 5: Visual Encoding: Input the segmented local and global images into the visual encoder to extract visual features. This step ensures that we can obtain rich visual information from both local and global perspectives.

[0133] Specifically, the visual encoder encodes image patches with a length of 256 and a feature dimension of 1024. The visual features are then passed through a two-layer perceptron, which converts the feature dimension to 2048 and aligns it with the feature dimension of the language model.

[0134] Step Six: Dynamic Visual Feature Cropping: First, text features, local image features, and global image features are input into the large language model. The model calculates the attention of text features and global image features to local image features, discarding local image features with less attention. This process reduces the number of features input into the large language model, thereby reducing inference overhead.

[0135] Step Seven: Large-Scale Model Analysis and Summary: Finally, the outputs of the audio segmentation model and image segmentation model, along with image features and issues, are input into the large language model. The large language model summarizes the risk points and provides risk analysis and suggestions. This comprehensive analytical capability enables the system to provide users with comprehensive risk identification and response strategies. By setting different prompts for the entire model, the large language model generates risk assessment reports for specific scenarios. Users can use these reports to understand the potential risks in multimedia content and take appropriate measures to protect themselves from fraud or malicious behavior.

[0136] Through these steps, the method can effectively identify and analyze various types of risky content, including face tampering, voice tampering, risky websites, risky conversations, and risky text messages. The specific manifestations of these functions are as follows: Figure 3 As shown, for a tampered image, this method can accurately detect the location of the tampered face and provide segmentation results of the tampered region. Furthermore, the method provides a detailed analysis report explaining the location and extent of the tampering, helping users better understand the nature of the tampering behavior. Figure 4 As shown, when an image containing website information is input, this method can intelligently assess whether the website poses a fraud risk and provide specific reasons. If a risk is detected, the system will clearly point out the risk and remind the user to avoid visiting such websites to prevent personal information leakage or financial loss. Figure 5 As shown, in cases of voice tampering, this method can detect the tampered parts of the audio and perform detailed analysis. For example... Figure 6As shown, this method can quickly identify suspicious content in fraudulent text messages and point out potential risks. In addition, it provides targeted advice on how to safely handle such messages, effectively preventing users from suffering losses due to believing fraudulent information.

[0137] Example 3:

[0138] like Figure 7 The diagram shown is an architectural schematic of a risk content identification method device based on a multimodal large model according to an embodiment of the present invention. The risk content identification method device based on a multimodal large model in this embodiment includes one or more processors 21 and a memory 22. Figure 7 Take a processor 21 as an example.

[0139] Processor 21 and memory 22 can be connected via a bus or other means. Figure 7 Taking the example of a connection between China and Israel via a bus.

[0140] The memory 22, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the risk content identification method based on a multimodal large model in Embodiment 1. The processor 21 executes the risk content identification method based on a multimodal large model by running the non-volatile software programs and instructions stored in the memory 22.

[0141] Memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 22 may optionally include memory remotely located relative to processor 21, which can be connected to processor 21 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0142] The program instructions / modules are stored in the memory 22. When executed by one or more processors 21, they execute the risk content identification method based on a multimodal large model in Embodiment 1.

[0143] It is worth noting that the information interaction and execution process between the modules and units in the above-mentioned device and system are based on the same concept as the processing method embodiment of the present invention. For details, please refer to the description in the method embodiment of the present invention, and will not be repeated here.

[0144] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0145] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A risk content identification method based on a multimodal large model, characterized in that, The methods include: In step 201, the audio is input into the audio segmentation model LATDL. The audio segmentation model LATDL is used to convert the audio into text, identify the fake parts in the audio, and output the corresponding text features of the audio. In step 202, the target face image is preprocessed to extract background noise features and high-frequency features, which are then input into the image segmentation model ForgerySAM to locate the fake regions in the target face image. In step 203, a series of aspect ratios are predefined, including one or more of 1:1, 1:2, 1:3, 2:3, 3:4, 1:12, 2:1, 3:1, 3:2, 4:3 and 12:1; In step 204, the aspect ratio of the target face image is calculated, and the closest aspect ratio is selected from the predefined aspect ratios. The target face image is then adjusted to the predefined resolution using interpolation to make it conform to the selected aspect ratio. The aspect ratio is a multiple of a fixed-size sliding window. The target face image is then segmented using the sliding window to obtain a local image. At the same time, the resolution of the target face image is adjusted to the size of the sliding window to obtain the global image. In step 205, the local and global images obtained from the segmentation are input into a visual encoder to extract visual features; wherein, the visual features include local image features and global image features; In step 206, text features, local image features, and global image features are input into the large language model, which calculates the attention of text features and global image features to local image features, discards local image features with attention less than a preset value, and obtains cropped local image features. In step 207, the text features, cropped local image features, and global image features are input into the large language model. The large language model summarizes the risk points and provides risk analysis and suggestions.

2. The risk content identification method based on a multimodal large model according to claim 1, characterized in that, The audio is input into the audio segmentation model LATDL, which converts the audio into text, identifies fake parts in the audio, and outputs the corresponding text features. Specifically, this includes: The audio segmentation model first uses a pre-trained wav2vec2 model to perform preliminary feature extraction on the input audio, converting the audio into a feature vector sequence containing rich information; wherein, the feature vector sequence captures the frequency changes, timbre features and temporal structure of the audio; The audio segmentation model uses a multi-layer one-dimensional convolutional neural network ResNet to further extract deep features of the audio. Specifically, a local attention mechanism is introduced into the audio segmentation model, using the extracted timbre features as query Q and key K, and the feature vector sequence extracted by wav2vec2 as value V, so that the model can further capture the semantic features of the audio through the attention mechanism. The audio segmentation model uses a multilayer perceptron to further process and fuse the extracted timbre and semantic features to obtain fused features; Train a classifier to classify each frame of audio based on the fused features, and output its forgery probability and the corresponding text features of the audio.

3. The risk content identification method based on a multimodal large model according to claim 1, characterized in that, The preprocessing of the target face image, extracting background noise and high-frequency features, and inputting them into the image segmentation model ForgerySAM to locate the forged region in the target face image, specifically includes: In the acquired target face image, the noise feature and the high-frequency feature in the target face image are obtained through two steps: noise extraction and high-frequency information extraction. The pre-processed target face image, background noise features, and high-frequency features are input into the image segmentation model to obtain the segmentation result of the fake region.

4. The risk content identification method based on a multimodal large model according to claim 3, characterized in that, In the acquired target face image, the noise background features and high-frequency features in the target face image are obtained through two steps: noise background extraction and high-frequency information extraction. Specifically, this includes: The image segmentation model first uses a self-supervised learning method to extract the background noise features in the target face image; the image segmentation model also uses a high-pass filter to extract the high-frequency features in the target face image.

5. The risk content identification method based on a multimodal large model according to claim 3, characterized in that, The step of inputting the preprocessed target face image, background noise features, and high-frequency features into the image segmentation model to obtain the segmentation result of the forged region specifically includes: The image segmentation model utilizes a multi-condition input efficient fine-tuning architecture to fuse extracted background noise features and high-frequency features. The multi-condition input efficient fine-tuning architecture employs a weight sharing mechanism to accurately map the embedded features of multiple conditions to the same latent space, and utilizes a cross-attention mechanism to efficiently fuse background noise features and high-frequency features. Lora was also used to efficiently fine-tune the SAM model to adapt it to the task of image forgery detection; The image segmentation model uses a fine-tuned SAM model to segment the target face image, marking each pixel in the target face image as a fake region or a normal region to obtain the segmentation result; Based on the segmentation results, if the segmentation results show that there is a fake region, the image segmentation model determines that the target face image is a fake image. Finally, the image segmentation module outputs the judgment of whether the target face image is fake and the coordinates of the fake region of the target face image.

6. The risk content identification method based on a multimodal large model according to claim 1, characterized in that, The process involves calculating the aspect ratio of the target face image, selecting the closest predefined aspect ratio, adjusting the target face image to a predefined resolution using interpolation to match the selected aspect ratio (the aspect ratio being a multiple of a fixed-size sliding window), and segmenting the target face image using the sliding window to obtain local images. Simultaneously, the resolution of the target face image is adjusted to the size of the sliding window to obtain the global image. Specifically, this includes: Calculate the aspect ratio of the target face images at different resolutions and select the predefined aspect ratio that is closest to the aspect ratio of the target face image as the final aspect ratio; The image resolution is adjusted to a multiple of 448 using interpolation methods; The uniformly adjusted target face image is segmented into local images with a resolution of 448×448 using a sliding window of fixed size; Meanwhile, the resolution of the target face image is adjusted to 448×448 to match the resolution of the local image, thus obtaining the global image. The local image with the adjusted resolution of 448×448 contains the local visual information to be processed, while the target face image adjusted to 448×448 provides global visual information.

7. The risk content identification method based on a multimodal large model according to claim 1, characterized in that, The step of inputting the local and global images obtained from the segmentation into a visual encoder to extract visual features specifically includes: After feature extraction by the visual encoder, each block of the local and global images is encoded into a feature vector of length 256. These features are then processed by a two-layer perceptron to align the feature vectors into the feature space of the large language model, resulting in the corresponding visual features.

8. The risk content identification method based on a multimodal large model according to claim 1, characterized in that, The process of inputting text features, local image features, and global image features into a large language model, enabling the large language model to calculate the attention of text features and global image features to local image features, and discarding local image features with attention less than a preset value, specifically includes: Text features, local image features, and global image features are pre-input into the first two layers of the large language model; The first two layers of the model calculate the attention of text features and global image features to local image features, and discard local image features with attention less than a preset value.

9. The risk content identification method based on a multimodal large model according to claim 1, characterized in that, The text features, cropped local image features, and global image features are input into the large language model. The large language model summarizes the risk points and provides risk analysis and suggestions, specifically including: Text features, cropped local image features, and global image features are re-inputted into the large language model, enabling the large language model to summarize and analyze risky content and provide recommendations.

10. The risk content identification method based on a multimodal large model according to claim 1, characterized in that, Before the audio is input into the audio segmentation model LATDL, the audio is also preprocessed and denoised.

Images