Risk content identification method based on multi-modal large model

Abstract

The invention relates to the technical field of artificial intelligence, and provides a risk content identification method based on a multi-modal large model. The method comprises the following steps: identifying a forged part in audio; extracting bottom noise features and high-frequency features of the target face image, inputting the features into the image segmentation model, and positioning a forged area; dicing the target face image, and inputting a local area image and a global image obtained by dicing into a visual encoder to extract visual features; calculating the attention of the text features and the global image features on the local image features, and discarding the local image features with small attention; and inputting the output of the audio segmentation model and the image segmentation model, image features and questions into a large language model, and summarizing risk points. According to the method, multiple data sources are integrated, risk identification is carried out based on the multi-modal large model, the accuracy and robustness of identification are improved, multiple fraud means can be effectively coped with, and the problems that in the prior art, data of multiple modals cannot be coped with and interpretability is lacked are solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a risk content identification method based on a multimodal large model. Background Art

[0002] With the rapid development of the Internet and mobile communication technologies, online fraud activities are becoming increasingly rampant. Traditional single-modal recognition methods, such as text content-based recognition or voice feature-based recognition, have become increasingly limited in the face of ever-changing and innovative fraud methods. It is difficult to effectively deal with frauds that are carried out in various forms such as text messages, phone calls, fake voices, and fake face videos. In this context, there is an urgent need to explore more efficient and accurate risk content recognition methods, and risk content recognition technology based on multimodal large models is gaining increasing attention and showing great application potential.

[0003] Traditional risk identification methods mainly rely on single-modal information, such as text, voice, and images, and use rule engines, machine learning, and other technologies for identification. However, this method has the following limitations:

[0004] (1) Single information dimension, easy to be bypassed: Traditional risk identification methods only focus on information of a single modality, ignoring the correlation and complementarity between information of different modalities. Fraudsters can easily bypass the recognition system by changing the information of one modality. For example, in voice fraud, fraudsters can easily change their tone and intonation to evade recognition based on voice features.

[0005] (2) Insufficient generalization ability, making it difficult to deal with new fraud methods: Traditional risk identification methods usually rely on manually formulated rules or patterns learned from historical data, and their ability to identify new fraud methods that have never appeared before is limited. For example, when faced with fake face videos generated using deep fake technology, traditional image or video-based recognition methods are often difficult to effectively identify.

[0006] (3) Lack of explainability, difficulty in tracing and optimizing: Traditional risk identification methods can usually only give the final identification results, but cannot explain the identification basis and reasoning process. This makes it difficult for users to understand the reliability of the identification results and is not conducive to technical personnel to optimize and improve the model.

[0007] This shows that in the face of increasingly sophisticated online fraud methods, traditional single-modality recognition methods have become inadequate.

[0008] In view of this, overcoming the defects of the prior art is an urgent problem to be solved in the field of this technology. Summary of the invention

[0009] The technical problem to be solved by the present invention is to provide a risk content identification method based on a multimodal large model.

[0010] The present invention adopts the following technical solution:

[0011] In a first aspect, the present invention provides a method for identifying risk content based on a multimodal large model, the method comprising:

[0012] In step 201, the audio is input into the audio segmentation model LATDL, which is used to convert the audio into text, identify the forged part, and output the corresponding audio content;

[0013] In step 202, the target face image is processed to extract background noise features and high-frequency features, which are input into the image segmentation model ForgerySAM to locate the forged area in the target face image;

[0014] In step 203, a series of aspect ratios are predefined, including one or more of 1:1, 1:2, 1:3, 2:3, 3:4, 1:12, 2:1, 3:1, 3:2, 4:3 and 12:1;

[0015] In step 204, the aspect ratio of the target face image is calculated, and the closest one is selected from the predefined aspect ratios. The target face image is adjusted to a predefined resolution by interpolation so that it conforms to the selected aspect ratio, and the aspect ratio is a multiple of the fixed-size sliding window. The target face image is cut into pieces using the sliding window to obtain a local area image; at the same time, the resolution of the target face image is adjusted to the sliding window size so as to obtain global visual information;

[0016] In step 205, the local area image and the global image obtained by slicing are input into a visual encoder to extract visual features; wherein the visual features include local image features and global image features; wherein the global image is the target face image;

[0017] In step 206, the text features, local image features and global image features are input into the large language model, so that the large language model calculates the attention of the text features and the global image features to the local image features, and discards the local image features whose attention is less than a preset value;

[0018] In step 207, the outputs of the audio segmentation model and the image segmentation model as well as the image features and questions are input into the large language model, and the large language model summarizes the risk points and provides risk analysis and suggestions.

[0019] Preferably, the step of inputting the audio into an audio segmentation model LATDL, which is used to convert the audio into text, identify the forged part thereof, and output the corresponding audio content, specifically includes:

[0020] The audio segmentation model first uses the pre-trained wav2vec2 model to perform preliminary feature extraction on the input audio and convert the audio into a feature vector sequence containing rich information; wherein the feature vector sequence captures the frequency changes, timbre characteristics and temporal structure of the audio;

[0021] The audio segmentation model uses a multi-layer one-dimensional convolutional neural network ResNet to further extract deep features of the audio. In particular, a local attention mechanism is introduced into the audio segmentation model, using the extracted timbre features as the query Q and key K, and the feature vector sequence extracted by wav2vec2 as the value V, so that the model can further capture the semantic features of the audio through the attention mechanism.

[0022] The audio segmentation model uses a multi-layer perceptron to further process and fuse the extracted timbre features and semantic features to generate a unified feature representation;

[0023] Train a classifier to classify each frame of audio based on the fused features, output its forgery probability and the character corresponding to the audio, and thus obtain the audio content and forgery probability.

[0024] Preferably, the processing of the target face image, extracting background noise features and high-frequency features, and inputting them into the image segmentation model ForgerySAM to locate the forged area in the target face image specifically includes:

[0025] In the acquired target face image, the background noise feature and the high-frequency feature in the target face image are obtained through the two steps of background noise extraction and high-frequency information extraction respectively;

[0026] The preprocessed target face image, background noise features and high-frequency features are input into the image segmentation model to obtain the segmentation result of the forged area.

[0027] Preferably, in the acquired target face image, obtaining the background noise feature and the high-frequency feature in the target face image respectively through the two steps of background noise extraction and high-frequency information extraction, specifically includes:

[0028] The image segmentation model first uses a self-supervised learning method to extract the background noise features in the target face image; the image segmentation model also uses a high-pass filter to extract the high-frequency features in the target face image.

[0029] Preferably, the step of inputting the pre-processed target face image, background noise features and high-frequency features into an image segmentation model to obtain a segmentation result of the forged area specifically includes:

[0030] The image segmentation model uses a multi-condition input efficient fine-tuning architecture to fuse the extracted background noise features and high-frequency features; wherein the multi-condition input efficient fine-tuning architecture adopts a weight sharing mechanism to accurately map the embedded features of multiple conditions to the same latent space, and uses a cross-attention mechanism to efficiently fuse the background noise features and high-frequency features;

[0031] Lora is also used to efficiently fine-tune the SAM model to adapt it to the task of image forgery detection;

[0032] The image segmentation model uses the fine-tuned SAM model to segment the target face image, marking each pixel in the target face image as a forged area or a normal area to obtain the segmentation result;

[0033] According to the segmentation result, if the segmentation result shows that there is a forged area, the image segmentation model determines that the target face image is a forged image, and finally the image segmentation module outputs the judgment of whether the target face image is forged and the coordinates of the forged area of ​​the target face image.

[0034] Preferably, the aspect ratio of the target face image is calculated, the closest one is selected from predefined aspect ratios, the target face image is adjusted to a predefined resolution by interpolation so that it conforms to the selected aspect ratio, the aspect ratio is multiples of a fixed-size sliding window, and the target face image is sliced ​​using the sliding window to obtain a local area image; at the same time, the resolution of the target face image is adjusted to the sliding window size so as to obtain global visual information, specifically including:

[0035] Calculate the aspect ratio of the input target face images of different resolutions, and select the predefined aspect ratio closest to the aspect ratio of the target face image as the final aspect ratio;

[0036] The resolution of the image is adjusted to a multiple of 448 by interpolation method;

[0037] Use a fixed-size sliding window to cut the uniformly adjusted target face image into local area images with a resolution of 448×448;

[0038] At the same time, the resolution of the target face image is adjusted to 448×448 again to be consistent with the resolution of the local area image; wherein the local area image with the adjusted resolution of 448×448 contains the local visual information to be processed, while the target face image adjusted to 448×448 provides global visual information.

[0039] Preferably, the step of inputting the local area image and the global image obtained by slicing into a visual encoder to extract visual features specifically includes:

[0040] After the local area image and the global image are subjected to feature extraction by the visual encoder, each block is encoded into a feature vector of length 256;

[0041] These features are then passed through a two-layer perceptron to align the feature vectors into the feature space of the large language model to obtain the corresponding visual features.

[0042] Preferably, the step of inputting text features, local image features and global image features into a large language model, allowing the large language model to calculate the attention of text features and global image features to local image features, and discarding local image features whose attention is less than a preset value, specifically includes:

[0043] Pre-input text features, local image features, and global image features into the first two layers of the large language model;

[0044] The first two layers of the model calculate the attention of text features and global image features to local image features, and discard local image features whose attention is less than a preset value.

[0045] Preferably, the outputs of the audio segmentation model and the image segmentation model as well as the image features and problems are input into the large language model, and the large language model summarizes the risk points and provides risk analysis and suggestions, specifically including:

[0046] The text features, cropped local image features, and global image features are re-input into the large language model so that the large language model can summarize and analyze the risk content and give recommendations.

[0047] Preferably, before inputting the audio into the audio segmentation model LATDL, the audio is also preprocessed and denoised.

[0048] In a second aspect, the present invention further provides a risk content identification device based on a multimodal large model, which is used to implement the risk content identification method based on a multimodal large model described in the first aspect, and the device includes:

[0049] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the risk content identification method based on a multimodal large model as described in the first aspect.

[0050] In a third aspect, the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the method described in the first aspect.

[0051] In a fourth aspect, a chip is provided, comprising: a processor and an interface, for calling and running a computer program stored in the memory from a memory to execute the method of the first aspect.

[0052] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer or a processor, causes the computer or the processor to execute the method of the first aspect.

[0053] The present invention effectively integrates multiple data sources and performs risk identification based on a risk content identification method of a multimodal large model. It not only improves the accuracy and robustness of identification, but can also effectively deal with multiple new types of fraud methods such as text message, conversation, web page, and video fraud, effectively maintain social security, and solves the problem that existing technologies cannot deal with data in multiple modalities and lack interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 It is a flowchart of a risk content identification method based on a multimodal large model provided by an embodiment of the present invention;

[0056] Figure 2 is a schematic diagram of a risk content identification method based on a multimodal large model provided by an embodiment of the present invention;

[0057] Figure 3 is a schematic diagram of a risk content identification method based on a multimodal large model provided by an embodiment of the present invention;

[0058] Figure 4 is a schematic diagram of a risk content identification method based on a multimodal large model provided by an embodiment of the present invention;

[0059] Figure 5 is a schematic diagram of a risk content identification method based on a multimodal large model provided by an embodiment of the present invention;

[0060] Figure 6is a schematic diagram of a risk content identification method based on a multimodal large model provided by an embodiment of the present invention;

[0061] Figure 7 It is a schematic diagram of the architecture of a risk content identification device based on a multimodal large model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0063] Unless the context requires otherwise, throughout the specification and claims, the term "including" is to be interpreted as open inclusion, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "examples", "specific examples" or "some examples" and the like are intended to indicate that specific features, structures, materials or characteristics associated with the embodiment or example are included in at least one embodiment or example of the present disclosure. The schematic representation of the above terms does not necessarily refer to the same embodiment or example. In addition, the specific features, structures, materials or characteristics may be included in any one or more embodiments or examples in any appropriate manner, that is, although they may be carried in the embodiments or examples of the above terms due to reasons such as the order and position of appearance, it is not limited to that they can be carried in combination by one embodiment or example.

[0064] In the description of the present invention, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "multiple" is two or more. In addition, for example, the same type of nouns may be described as two independent individuals by adding "A" and "B" at the end. In this case, the corresponding features defined as "A" and "B" are only used to distinguish the same type of individuals for description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features.

[0065] In the description of the present invention, the expression "A and / or B" (where A and B are used to formally represent specific characteristic contents) will be involved, and the corresponding expressions include the following three combinations: only A, only B, and a combination of A and B.

[0066] As used herein, "about," "substantially," or "approximately" includes the stated value and an average value that is within an acceptable range of deviation from the particular value as determined by one of ordinary skill in the art taking into account the measurements in question and the errors associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0067] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0068] Embodiment 1:

[0069] In recent years, with the rapid development of deep learning technology, multimodal large models have made breakthrough progress in natural language processing, computer vision and other fields. Compared with traditional single-modal models, multimodal large models can comprehensively process information from different modalities, such as text, voice, image, video, etc., and learn the complex associations between different modal information. They have the following advantages: Strong information fusion ability and more accurate recognition: multimodal large models can comprehensively analyze information from different modalities, for example, jointly analyze voice content and intonation changes with text content and video images, so as to more comprehensively and accurately identify risk content. Strong generalization ability and more effective response to new types of fraud: multimodal large models can learn the deep semantic associations between different modal information, have stronger generalization ability, and can effectively identify and respond to new and complex fraud methods. Have a certain degree of interpretability: multimodal large models can analyze the input information to give explanations of possible fraud locations, which are more acceptable to people. That is, the multimodal large model has comprehensive information processing capabilities and excellent interpretability. Based on this characteristic of the multimodal large model, Embodiment 1 of the present invention provides a risk content identification method based on the multimodal large model, such as Figure 1 As shown, the method includes:

[0070] In step 201, the audio is input into the audio segmentation model LATDL, which is used to convert the audio into text, identify the forged parts, and output the corresponding audio content; this step ensures that we can accurately locate and identify the forged information that may exist in the speech (i.e., audio). Before the audio is input into the audio segmentation model LATDL, the audio is also preprocessed and denoised. The preprocessing can be format adjustment or cropping of the audio.

[0071] In step 202, the target face image is processed to extract background noise features and high-frequency features, which are input into the image segmentation model ForgerySAM to locate the forged area in the target face image; this process helps to identify the tampered or forged parts of the image.

[0072] In step 203, a series of aspect ratios are predefined, including one or more of 1:1, 1:2, 1:3, 2:3, 3:4, 1:12, 2:1, 3:1, 3:2, 4:3 and 12:1; these ratios provide a basis for subsequent image processing.

[0073] In step 204, the aspect ratio of the target facial image is calculated, and the closest one is selected from the predefined aspect ratios. The target facial image is adjusted to a predefined resolution by interpolation so that it conforms to the selected aspect ratio. The aspect ratio is a multiple of a fixed-size sliding window. The target facial image is cut into pieces using the sliding window to obtain a local area image. At the same time, the resolution of the target facial image is adjusted to the sliding window size so as to obtain global visual information. The fixed size is obtained by a technician in this field based on empirical analysis.

[0074] In step 205, the local area image and the global image obtained by slicing are input into the visual encoder to extract visual features; wherein the visual features include local image features and global image features; wherein the global image is the target face image; this step ensures that we can obtain rich visual information from both local and overall aspects.

[0075] In step 206, the text features, local image features and global image features are input into the large language model, so that the large language model calculates the attention of the text features and global image features to the local image features, and discards the local image features whose attention is less than a preset value, that is, discards the local image features with small attention; wherein the preset value is obtained by those skilled in the art based on empirical analysis. This process reduces the number of features input into the large language model, thereby reducing the inference consumption.

[0076] In step 207, the outputs of the audio segmentation model and the image segmentation model, as well as the image features and problems, are input into the large language model, which summarizes the risk points and provides risk analysis and suggestions. This comprehensive analysis capability enables the system to provide users with comprehensive risk identification and response strategies.

[0077] This embodiment effectively integrates multiple data sources and performs risk identification based on a risk content identification method of a multimodal large model. It not only improves the accuracy and robustness of identification, but can also effectively deal with multiple new types of fraud such as text message, conversation, web page, and video fraud, effectively maintaining social security, and solving the problem that existing technologies are unable to cope with data in multiple modalities and lack explainability, and also provides a solid technical guarantee for national security.

[0078] The audio is input into the audio segmentation model LATDL, and the audio segmentation model LATDL is used to convert the audio into text, identify the forged part, and output the corresponding audio content, which specifically includes:

[0079] The audio segmentation model first uses the pre-trained wav2vec2 model to perform preliminary feature extraction on the input audio and converts the audio into a feature vector sequence containing rich information; wherein the feature vector sequence captures the frequency changes, timbre characteristics and timing structure of the audio.

[0080] The audio segmentation model uses a multi-layer one-dimensional convolutional neural network ResNet to further extract deep features of the audio, especially fine-grained frame-level timbre information; among them, the residual structure of ResNet effectively avoids the gradient vanishing problem in deep network training and ensures the learning ability of the model. In order to focus more accurately on the key information in the audio, a local attention mechanism is also introduced in the audio segmentation model, using the extracted timbre features as the query Q and key K, and the feature vector sequence extracted by wav2vec2 as the value V, so that the model can further capture the semantic features of the audio through the attention mechanism and focus on specific time periods or feature segments, thereby more effectively detecting subtle changes in forged audio.

[0081] The audio segmentation model uses a multi-layer perceptron to further process and fuse the extracted timbre features and semantic features to generate a unified feature representation.

[0082] Train a classifier to classify each frame of audio based on the fused features, output its forgery probability and the character corresponding to the audio, and achieve accurate detection of the entire audio or part of the forged audio, thereby obtaining the audio content and forgery probability.

[0083] In some embodiments, the processing of the target face image, extracting background noise features and high-frequency features, and inputting them into the image segmentation model ForgerySAM to locate the forged area in the target face image specifically includes:

[0084] In the acquired target face image, the background noise features and the high-frequency features in the target face image are obtained through the two steps of background noise extraction and high-frequency information extraction respectively.

[0085] The preprocessed target face image, background noise features and high-frequency features are input into the image segmentation model to obtain the segmentation result of the forged area. The preprocessing is: scaling and cropping the image according to a predefined aspect ratio set to adapt to different image sizes and contents to ensure the integrity of the image information.

[0086] In an optional implementation, in the acquired target face image, obtaining the background noise feature and the high-frequency feature in the target face image respectively through the two steps of background noise extraction and high-frequency information extraction, specifically includes:

[0087] The image segmentation model first uses a self-supervised learning method to extract the background noise features in the target face image; the background noise features refer to the unique imprints left by factors such as equipment and environment during the image generation process, which contain information that is not easily perceived but is crucial. The image segmentation model also uses a high-pass filter to extract high-frequency features in the target face image. Other image processing algorithms can also be used to extract high-frequency features. The high-frequency features include features such as edges, details, and textures. These high-frequency information are easily changed during the image forgery process and become an important clue to determine the authenticity of the image.

[0088] The step of inputting the pre-processed target face image, background noise features and high-frequency features into the image segmentation model to obtain the segmentation result of the forged area specifically includes:

[0089] The image segmentation model uses a multi-condition input efficient fine-tuning architecture to fuse the extracted background noise features and high-frequency features; wherein, the multi-condition input efficient fine-tuning architecture adopts a weight sharing mechanism to accurately map the embedded features of multiple conditions to the same latent space, and uses a cross-attention mechanism to efficiently fuse the background noise features and high-frequency features.

[0090] The SAM model (Segment Anything Model) is also efficiently fine-tuned using the Low-Rank Adaptation of Large Language Models (Lora), which is used to fine-tune large language models, to adapt it to the task of image forgery detection.

[0091] The image segmentation model uses the fine-tuned SAM model to segment the target face image, marks each pixel in the target face image as a forged area or a normal area, and obtains the segmentation result.

[0092] According to the segmentation result, if the segmentation result shows that there is a forged area, the image segmentation model determines that the target face image is a forged image, and finally the image segmentation module outputs the judgment of whether the target face image is forged and the coordinates of the forged area of ​​the target face image.

[0093] In actual application scenarios, the aspect ratio of the target face image is calculated, the closest one is selected from predefined aspect ratios, and the target face image is adjusted to a predefined resolution by interpolation to conform to the selected aspect ratio, where the aspect ratio is a multiple of a fixed-size sliding window, and the target face image is sliced ​​using the sliding window to obtain a local area image; at the same time, the resolution of the target face image is adjusted to the sliding window size so as to obtain global visual information, specifically including:

[0094] The aspect ratio of the input target face images of different resolutions is calculated, and the predefined aspect ratio closest to the aspect ratio of the target face image is selected as the final aspect ratio.

[0095] The image resolution is adjusted to a multiple of 448 through interpolation; for example, assuming the final aspect ratio is 3:4, an image with a resolution of (300, 400) will be adjusted to a size of (3×448, 4×448).

[0096] The uniformly adjusted target face image is cut into local area images with a resolution of 448×448 using a sliding window of a fixed size.

[0097] At the same time, the resolution of the target face image is adjusted to 448×448 to match the resolution of the local region image; the local region image with the adjusted resolution of 448×448 contains the local visual information to be processed, while the target face image adjusted to 448×448 provides global visual information. The combination of local image information and global image information ensures that the model has a comprehensive understanding of the original image.

[0098] In an optional implementation, the step of inputting the local region image and the global image obtained by slicing into a visual encoder to extract visual features specifically includes:

[0099] After the local area image and the global image are subjected to feature extraction by the visual encoder, each block is encoded into a feature vector of length 256.

[0100] These features are then passed through a two-layer perceptron to align the feature vectors into the feature space of the large language model to obtain the corresponding visual features. Specifically: These feature vectors are transformed in dimension through a two-layer perceptron to map the visual features to the same dimension of 2048 as the text feature space of the large language model.

[0101] In an actual application scenario, the method of inputting text features, local image features and global image features into a large language model so that the large language model calculates the attention of text features and global image features to local image features, and discards local image features whose attention is less than a preset value, specifically includes: inputting text features, local image features and global image features into the first two layers of the large language model in advance; the first two layers of the model calculate the attention of text features and global image features to local image features, discards local image features whose attention is less than a preset value, and retains important local image features.

[0102] In an optional implementation, the output of the audio segmentation model, the image segmentation model, the image features and the questions are input into the large language model, and the large language model summarizes the risk points and provides risk analysis and suggestions, specifically including: re-inputting text features, cropped local image features and global image features into the large language model, so that the large language model summarizes and analyzes the risk content and gives suggestions. This part is mainly achieved by designing prompt words, so that the large language model can deeply integrate and understand the input multimodal features, and finally output the results including analysis, summary and suggestions of the risk content by designing prompt words.

[0103] Embodiment 2:

[0104] With the rapid development of information technology, especially the popularity of mobile Internet and social media platforms, new types of fraud are increasing, and their methods are becoming more diverse and covert. Traditional single-modal recognition methods, such as those based only on text analysis, are powerless when faced with multiple forms of fraud such as text messages, conversations, web pages, videos, etc. These methods can often only analyze a specific form of information, and cannot effectively identify fraud across modalities, which poses a great challenge to preventing and combating new types of fraud.

[0105] In view of the problem that the traditional single-modality recognition method is difficult to simultaneously deal with a series of new fraud methods such as text messages, conversations, web pages, and video frauds, the present invention provides a risk content recognition method based on a multimodal large model, the purpose of which is to not only help improve the recognition rate of fraudulent behavior through the modal collaboration of images, voices, and texts and the powerful understanding ability of the multimodal large model, but also provide users with more comprehensive security protection, so that it can effectively deal with a variety of new fraud methods such as text messages, conversations, web pages, and video frauds at the same time, effectively maintain social security, and solve the problem that the existing technology cannot deal with multiple modal data and lacks interpretability. The core of the present invention is to use the powerful understanding and dialogue capabilities of the large language model to build a multimodal large model that can integrate image, voice, and text information, so as to complete dialogue exchanges with users and risk warnings.

[0106] To achieve the above object, the present invention proposes a risk content identification method based on a multimodal large model, comprising the following steps:

[0107] Step 1: Forged speech recognition: First, the speech is input into the audio segmentation model LATDL. The model is responsible for converting the audio content into text, identifying the forged parts, and outputting the corresponding audio content. This step ensures that we can accurately locate and identify possible forged information in the speech.

[0108] Step 2: Forged area identification: Next, the target face image is processed. By extracting the background noise features and high-frequency features and inputting them into the image segmentation model ForgerySAM, the forged areas in the image can be accurately located. This process helps to identify the tampered or forged parts of the image.

[0109] Step 3: Predefine image resolution: In this step, we predefine a series of image aspect ratios, such as 1:1, 1:2, 3:4, 1:12, 4:3, 12:1, etc. These ratios provide the basis for subsequent image processing.

[0110] Step 4: Dynamic resolution slicing of the image: Calculate the aspect ratio of the original image and select the closest one from the predefined aspect ratios. Adjust the image to the predefined resolution through interpolation to make it match the selected aspect ratio, where the length and width are multiples of the fixed-size sliding window. Then, use the sliding window to slice the image. At the same time, adjust the original image resolution to the sliding window size to obtain global visual information.

[0111] Step 5: Visual encoding: The local image and the global image obtained by cutting are input into the visual encoder to extract visual features. This step ensures that we can obtain rich visual information from both local and overall aspects.

[0112] Step 6: Dynamic visual feature cropping: First, text features, local image features, and global image features are input into the large language model. The model calculates the attention of text features and global image features to local image features, and discards local image features with less attention. This process reduces the number of features input into the large language model, thereby reducing the cost of inference.

[0113] Step 7: Large model analysis summary: Finally, the output of the audio segmentation model and image segmentation model, as well as image features and problems, are input into the large language model. The large language model summarizes the risk points and provides risk analysis and suggestions. This comprehensive analysis capability enables the system to provide users with comprehensive risk identification and response strategies.

[0114] In one embodiment of the present invention, the step 1 specifically includes: the model first uses the pre-trained wav2vec2 model to perform preliminary feature extraction on the input audio, and converts the audio into a feature vector sequence containing rich information. These vectors capture key information such as frequency changes, timbre characteristics, and temporal structure of the audio. Then, the model uses a multi-layer one-dimensional convolutional neural network ResNet to further extract deep features of the audio data, especially fine-grained frame-level timbre information. The residual structure of ResNet effectively avoids the gradient vanishing problem in deep network training and ensures the learning ability of the model. In order to focus more accurately on the key information in the audio, the model introduces a local attention mechanism. Using the extracted timbre features as the query Q and key K, and the original audio features extracted by wav2vec2 as the value V, the model can further capture the semantic information of the audio through the attention mechanism, and focus on specific time periods or feature segments, thereby more effectively detecting subtle changes in forged audio. Finally, the model uses a multi-layer perceptron to further process and fuse the extracted timbre and semantic features to generate a unified feature representation. Then, we train a classifier to classify each frame of audio data based on the fused features, and output its forgery probability and the character corresponding to the audio, so as to achieve accurate detection of the entire audio or part of the forged audio. Finally, we get the audio content and forgery probability through the LATDL audio segmentation model.

[0115] In one embodiment of the present invention, the step 2 specifically includes: extracting the "background noise" of the image: the model first uses a self-supervised learning method to extract the "background noise" in the image. These "background noises" refer to the unique imprints left by factors such as equipment and environment during the generation process of the image, which contain information that is not easily perceived but is crucial. Extracting high-frequency information of the image: At the same time, the model uses a high-pass filter or other advanced image processing algorithms to extract high-frequency information in the image, such as features such as edges, details, and textures. These high-frequency information is easy to change during the image forgery process and becomes an important clue to determine the authenticity of the image. Feature fusion and fine-tuning: The model uses the "multi-condition input efficient fine-tuning architecture" to fuse the extracted "background noise" and high-frequency information. The architecture uses a weight sharing mechanism to accurately map the embedded features of multiple conditions to the same latent space, and uses a cross-attention mechanism to efficiently fuse the "background noise" and "high-frequency" features. In addition, the model also uses Lora to efficiently fine-tune the SAM model to adapt it to the task of image forgery detection. Image segmentation: The model uses the fine-tuned SAM model to segment the image and mark each pixel in the image as a forged area or a normal area. Output: Based on the segmentation results, if the segmentation results show that there are obvious forged areas, the model determines that the image is a forged image. The final image segmentation module outputs the judgment of whether the image is forged and the coordinates of the forged area of ​​the image.

[0116] In one embodiment of the present invention, the step three specifically includes: first, obtaining the original size of the image, and calculating the target size under each aspect ratio according to a predefined aspect ratio set (1:1, 1:2, 1:3, 2:3, 3:4, 2:1, 3:1, 3:2, 4:3). Then, comparing the difference between the target size and the original size, selecting the aspect ratio with the smallest difference as the final aspect ratio. Finally, according to the final aspect ratio, using an interpolation algorithm to adjust the image to a multiple of 448 in both length and width.

[0117] In one embodiment of the present invention, step 4 specifically includes: first, obtaining the image processed in step 3. Then, using a sliding window of a fixed size, cutting the adjusted image into multiple image blocks with a resolution of 448x448, and adjusting the original image to a resolution of 448×448 by interpolation.

[0118] In one embodiment of the present invention, step 5 specifically includes: inputting each image block with a resolution of 448x448 obtained in step 4 into a visual encoder for feature extraction, and obtaining a feature vector with a length of 256 for each image block. These feature vectors are transformed in dimension through a two-layer perceptron to map the visual features to the same dimension 2048 as the text feature space of the large language model.

[0119] In one embodiment of the present invention, step six specifically includes: inputting text features, global image features and local image features into the first two layers of the large language model respectively. The first two layers of the model will calculate the attention weights of the text features and the global image features to each local image feature. According to the attention weights, the local image features with smaller attention weights are filtered out, and the important local image features are retained.

[0120] In one embodiment of the present invention, step 7 specifically includes: re-inputting the local image features, text features, and global image features filtered in step 6 into the large language model. The large language model performs deep fusion and understanding based on the input multimodal features. By designing prompt words, the large language model finally outputs results including analysis, summary, and suggestions of risk content.

[0121] The multimodal risk content identification method of the present invention can effectively integrate text, audio and image information to achieve accurate identification of multimodal risk content. First, the method extracts the timbre and semantic features of the audio through advanced audio processing technology, and combines the forged audio detection model to obtain the audio content and forgery probability. At the same time, the self-supervised learning and image segmentation technology are used to analyze the "bottom noise" and high-frequency information of the image, determine whether the image is forged, and accurately locate the forged area. In order to fuse multimodal information, the method introduces a flexible image preprocessing step, scales and crops the image according to a predefined aspect ratio set to adapt to different image sizes and contents, and ensure the integrity of image information. Then, the visual encoder is used to extract the features of the image block and map it to the feature space of the large language model. Finally, the attention mechanism of the large language model is used to screen important local image features, and the screened multimodal features are input into the large language model for deep fusion and understanding, and finally output the analysis, summary and suggestions of the risk content. The present invention effectively solves the problem that the traditional method lacks understanding of multimodal data, and significantly improves the accuracy and generalization ability of risk content identification.

[0122] Based on Example 1 and the above method, the following is combined with specific application scenarios and uses technical descriptions in related scenarios to explain the implementation process of the present invention in the characteristic scenarios.

[0123] like Figure 2 As shown, Figure 2 The content displayed includes: audio segmentation model, image segmentation model, visual encoder and large prediction model. The entire recognition process includes the following steps:

[0124] Step 1: Forged speech recognition: First, the speech is input into the audio segmentation model LATDL. The model is responsible for converting the audio content into text, identifying the forged parts, and outputting the corresponding audio content. This step ensures that we can accurately locate and identify possible forged information in the speech.

[0125] Specifically, the audio segmentation model first uses the pre-trained wav2vec2 model to perform preliminary feature extraction on the input audio and convert the audio into a sequence of feature vectors containing rich information. These vectors capture key information such as frequency changes, timbre characteristics, and temporal structure of the audio. Then, the model uses a multi-layer one-dimensional convolutional neural network ResNet to further extract deep features of the audio data, especially fine-grained frame-level timbre information. The residual structure of ResNet effectively avoids the gradient vanishing problem in deep network training and ensures the learning ability of the model. In order to focus on the key information in the audio more accurately, the model introduces a local attention mechanism. Using the extracted timbre features as the query Q and key K, and the original audio features extracted by wav2vec2 as the value V, the model can further capture the semantic information of the audio through the attention mechanism and focus on specific time periods or feature segments, thereby more effectively detecting subtle changes in forged audio. Finally, the model uses a multi-layer perceptron to further process and fuse the extracted timbre and semantic features to generate a unified feature representation. Then, we train a classifier to classify each frame of audio data based on the fused features, and output its forgery probability and the character corresponding to the audio, so as to achieve accurate detection of the entire audio or part of the forged audio. Finally, we get the audio content and forgery probability through the LATDL audio segmentation model.

[0126] Step 2: Forged area identification: Next, the target face image is processed. By extracting the background noise features and high-frequency features and inputting them into the image segmentation model ForgerySAM, the forged areas in the image can be accurately located. This process helps to identify the tampered or forged parts of the image.

[0127] Specifically, the image segmentation model extracts the "bottom noise" of the image: the model first uses a self-supervised learning method to extract the "bottom noise" in the image. These "bottom noises" refer to the unique imprints left by factors such as equipment and environment during the image generation process, which contain information that is not easily perceived but is crucial. Extract high-frequency information of the image: At the same time, the model uses high-pass filters or other advanced image processing algorithms to extract high-frequency information in the image, such as features such as edges, details, and textures. These high-frequency information is easy to change during the image forgery process and becomes an important clue to determine the authenticity of the image. Feature fusion and fine-tuning: The model uses the "multi-condition input efficient fine-tuning architecture" to fuse the extracted "bottom noise" and high-frequency information. The architecture uses a weight sharing mechanism to accurately map the embedded features of multiple conditions to the same latent space, and uses a cross-attention mechanism to efficiently fuse the "bottom noise" and "high-frequency" features. In addition, the model also uses Lora to efficiently fine-tune the SAM model to adapt it to the task of image forgery detection. Image segmentation: The model uses the fine-tuned SAM model to segment the image and mark each pixel in the image as a forged area or a normal area. Output: Based on the segmentation results, if the segmentation results show that there are obvious forged areas, the model determines that the image is a forged image. The final image segmentation module outputs the judgment of whether the image is forged and the coordinates of the forged area of ​​the image.

[0128] Step 3: Predefine image resolution: In this step, we predefine a series of image aspect ratios, such as 1:1, 1:2, 3:4, 1:12, 4:3, 12:1, etc. These ratios provide the basis for subsequent image processing.

[0129] Specifically, this step ensures that the model can process the input of large-resolution images, with the highest resolution reaching 1344×1792, which can ensure that the model can capture the detailed features of the image, thereby being able to more accurately identify forged areas.

[0130] Step 4: Dynamic resolution slicing of the image: Calculate the aspect ratio of the original image and select the closest one from the predefined aspect ratios. Adjust the image to the predefined resolution through interpolation to make it match the selected aspect ratio, where the length and width are multiples of the fixed-size sliding window. Then, use the sliding window to slice the image. At the same time, adjust the original image resolution to the sliding window size to obtain global visual information.

[0131] Specifically, the resolution of each image block is 448×448 to adapt to the input size of the visual encoder. The mutual complementation of the global image and the local image helps the multimodal large model to fully understand the image content.

[0132] Step 5: Visual encoding: The local image and the global image obtained by cutting are input into the visual encoder to extract visual features. This step ensures that we can obtain rich visual information from both local and overall aspects.

[0133] Specifically, the visual encoder encodes the image block into a length of 256 and a feature dimension of 1024, and then the visual features pass through a two-layer perceptron to convert the feature dimension to 2048 and align it to the feature dimension of the language model.

[0134] Step 6: Dynamic visual feature cropping: First, text features, local image features, and global image features are input into the large language model. The model calculates the attention of text features and global image features to local image features, and discards local image features with less attention. This process reduces the number of features input into the large language model, thereby reducing the cost of inference.

[0135] Step 7: Big model analysis summary: Finally, the output of the audio segmentation model, image segmentation model, image features and problems are input into the big language model. The big language model summarizes the risk points and provides risk analysis and suggestions. This comprehensive analysis capability enables the system to provide users with comprehensive risk identification and response strategies. By setting different prompt words for the entire model, the big language model is helped to generate risk assessment reports for specific scenarios. Users can understand the potential risks in multimedia content based on these reports and take appropriate measures to protect themselves from fraud or malicious behavior.

[0136] After these steps, the method can effectively identify and analyze various risky contents, including face tampering, voice tampering, risky websites, risky conversations, and risky text messages. The specific performance of these functions is as follows: Figure 3 As shown in the figure, for a tampered image, this method can accurately detect the location of the tampered face and provide the segmentation result of the tampered area. In addition, this method will also provide a detailed analysis report to explain the location and degree of tampering, helping users to better understand the nature of tampering. Figure 4 As shown in Figure 1, when a picture containing website information is input, the method can intelligently evaluate whether the website has a fraud risk and give specific reasons. If a risk is found, the system will clearly point out the risk and remind users to avoid visiting such websites to prevent personal information leakage or economic losses. Figure 5 As shown in Figure 2, in the case of voice tampering, this method can detect the tampered part of the audio and perform a detailed analysis on it. Figure 6As shown in the figure, for fraudulent text messages, this method can quickly identify suspicious content in the text messages and point out possible risk points. In addition, the method will also give targeted suggestions to guide users on how to safely handle such information, effectively preventing users from suffering losses due to believing in fraudulent information.

[0137] Embodiment 3:

[0138] like Figure 7 , is a schematic diagram of the architecture of a risk content identification method and device based on a multimodal large model according to an embodiment of the present invention. The risk content identification method and device based on a multimodal large model according to this embodiment includes one or more processors 21 and a memory 22. Figure 7 A processor 21 is taken as an example.

[0139] The processor 21 and the memory 22 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0140] The memory 22 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs and non-volatile computer executable programs, such as the risk content identification method based on the multimodal large model in Example 1. The processor 21 executes the risk content identification method based on the multimodal large model by running the non-volatile software programs and instructions stored in the memory 22.

[0141] The memory 22 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 22 may optionally include a memory remotely arranged relative to the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0142] The program instructions / modules are stored in the memory 22, and when executed by the one or more processors 21, the risk content identification method based on the multimodal large model in the above-mentioned embodiment 1 is executed.

[0143] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific contents can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0144] A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0145] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A risk content identification method based on a multimodal large model, characterized in that: Methods include: In step 201, the audio is input into the audio segmentation model LATDL, which is used to convert the audio into text, identify the forged part, and output the corresponding audio content; In step 202, the target face image is processed to extract background noise features and high-frequency features, which are input into the image segmentation model ForgerySAM to locate the forged area in the target face image; In step 203, a series of aspect ratios are predefined, including one or more of 1:1, 1:2, 1:3, 2:3, 3:4, 1:12, 2:1, 3:1, 3:2, 4:3 and 12:1; In step 204, the aspect ratio of the target face image is calculated, and the closest one is selected from the predefined aspect ratios. The target face image is adjusted to a predefined resolution by interpolation so that it conforms to the selected aspect ratio, and the aspect ratio is a multiple of the fixed-size sliding window. The target face image is cut into pieces using the sliding window to obtain a local area image; at the same time, the resolution of the target face image is adjusted to the sliding window size so as to obtain global visual information; In step 205, the local area image and the global image obtained by slicing are input into a visual encoder to extract visual features; wherein the visual features include local image features and global image features; wherein the global image is the target face image; In step 206, the text features, local image features and global image features are input into the large language model, so that the large language model calculates the attention of the text features and the global image features to the local image features, and discards the local image features whose attention is less than a preset value; In step 207, the outputs of the audio segmentation model and the image segmentation model as well as the image features and questions are input into the large language model, and the large language model summarizes the risk points and provides risk analysis and suggestions.

2. The risk content identification method based on multimodal large model according to claim 1 is characterized in that: The audio is input into the audio segmentation model LATDL, which is used to convert the audio into text, identify the forged part, and output the corresponding audio content, specifically including: The audio segmentation model first uses the pre-trained wav2vec2 model to perform preliminary feature extraction on the input audio and convert the audio into a feature vector sequence containing rich information; wherein the feature vector sequence captures the frequency changes, timbre characteristics and temporal structure of the audio; The audio segmentation model uses a multi-layer one-dimensional convolutional neural network ResNet to further extract deep features of the audio. In particular, a local attention mechanism is introduced into the audio segmentation model, using the extracted timbre features as the query Q and key K, and the feature vector sequence extracted by wav2vec2 as the value V, so that the model can further capture the semantic features of the audio through the attention mechanism. The audio segmentation model uses a multi-layer perceptron to further process and fuse the extracted timbre features and semantic features to generate a unified feature representation; Train a classifier to classify each frame of audio based on the fused features, output its forgery probability and the character corresponding to the audio, and thus obtain the audio content and forgery probability.

3. The risk content identification method based on multimodal large model according to claim 1 is characterized in that: The target face image is processed to extract background noise features and high-frequency features, which are input into the image segmentation model ForgerySAM to locate the forged area in the target face image, specifically including: In the acquired target face image, the background noise feature and the high-frequency feature in the target face image are obtained through the two steps of background noise extraction and high-frequency information extraction respectively; The preprocessed target face image, background noise features and high-frequency features are input into the image segmentation model to obtain the segmentation result of the forged area.

4. The risk content identification method based on multimodal large model according to claim 3 is characterized in that: In the acquired target face image, the background noise feature and the high-frequency feature in the target face image are obtained respectively through the two steps of background noise extraction and high-frequency information extraction, specifically including: The image segmentation model first uses a self-supervised learning method to extract the background noise features in the target face image; the image segmentation model also uses a high-pass filter to extract the high-frequency features in the target face image.

5. The risk content identification method based on multimodal large model according to claim 3 is characterized in that: The pre-processed target face image, background noise features and high-frequency features are input into the image segmentation model to obtain the segmentation result of the forged area, specifically including: The image segmentation model uses a multi-condition input efficient fine-tuning architecture to fuse the extracted background noise features and high-frequency features; wherein the multi-condition input efficient fine-tuning architecture adopts a weight sharing mechanism to accurately map the embedded features of multiple conditions to the same latent space, and uses a cross-attention mechanism to efficiently fuse the background noise features and high-frequency features; Lora is also used to efficiently fine-tune the SAM model to adapt it to the task of image forgery detection; The image segmentation model uses the fine-tuned SAM model to segment the target face image, marking each pixel in the target face image as a forged area or a normal area to obtain the segmentation result; According to the segmentation result, if the segmentation result shows that there is a forged area, the image segmentation model determines that the target face image is a forged image, and finally the image segmentation module outputs the judgment of whether the target face image is forged and the coordinates of the forged area of ​​the target face image.

6. The risk content identification method based on multimodal large model according to claim 1 is characterized in that: The aspect ratio of the target face image is calculated, the closest one is selected from predefined aspect ratios, and the target face image is adjusted to a predefined resolution by interpolation so that it conforms to the selected aspect ratio, where the aspect ratio is a multiple of a fixed-size sliding window, and the target face image is sliced ​​using the sliding window to obtain a local area image; at the same time, the resolution of the target face image is adjusted to the sliding window size so as to obtain global visual information, specifically including: Calculate the aspect ratio of the input target face images of different resolutions, and select the predefined aspect ratio closest to the aspect ratio of the target face image as the final aspect ratio; The resolution of the image is adjusted to a multiple of 448 by interpolation method; Use a fixed-size sliding window to cut the uniformly adjusted target face image into local area images with a resolution of 448×448; At the same time, the resolution of the target face image is adjusted to 448×448 again to be consistent with the resolution of the local area image; wherein the local area image with the adjusted resolution of 448×448 contains the local visual information to be processed, while the target face image adjusted to 448×448 provides global visual information.

7. The risk content identification method based on multimodal large model according to claim 1 is characterized in that: The local area image and the global image obtained by slicing are input into a visual encoder to extract visual features, specifically including: After the local area image and the global image are subjected to feature extraction by the visual encoder, each block is encoded into a feature vector of length 256; These features are then passed through a two-layer perceptron to align the feature vectors into the feature space of the large language model to obtain the corresponding visual features.

8. The risk content identification method based on multimodal large model according to claim 1 is characterized in that: The step of inputting text features, local image features and global image features into a large language model so that the large language model calculates the attention of text features and global image features to local image features, and discarding local image features whose attention is less than a preset value, specifically includes: Pre-input text features, local image features, and global image features into the first two layers of the large language model; The first two layers of the model calculate the attention of text features and global image features to local image features, and discard local image features whose attention is less than a preset value.

9. The risk content identification method based on multimodal large model according to claim 1 is characterized in that: The outputs of the audio segmentation model and the image segmentation model as well as the image features and problems are input into the large language model. The large language model summarizes the risk points and provides risk analysis and suggestions, including: The text features, cropped local image features, and global image features are re-input into the large language model so that the large language model can summarize and analyze the risk content and give recommendations.

10. The risk content identification method based on multimodal large model according to claim 1 is characterized in that: Before the audio is input into the audio segmentation model LATDL, the audio is also preprocessed and denoised.

Images