Abnormal information detection method and device based on multimodal large model

Abstract

The present application provides a method and device for detecting abnormal information based on a multimodal large model. In this embodiment, on the premise of determining the existence of abnormal information, the specific content of the abnormal information is detected. Compared with directly detecting the specific details of the abnormal information on the target file, on the one hand, the detection of abnormal information and the detection of the specific content of the abnormal information are no longer performed simultaneously, which makes it possible to focus the entire computing power of the abnormal information detection model on whether abnormal information exists when detecting whether abnormal information exists, and improve the accuracy of whether abnormal information exists. On the other hand, when detecting the specific content of abnormal information, the prior condition of the existence of abnormal information can be fully utilized, and all computing power is focused on the specific content of the abnormal information, which can also accurately detect the specific content of the abnormal information.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for detecting abnormal information based on a multimodal large model. Background Art

[0002] In specific applications, it is common that an image contains private information such as personal identity information, or the face in the image is illegally tampered with to obtain a forged image, or there is violence and other illegal information in the image. The abnormal information in these images will cause personal and property losses to users.

[0003] Based on this, how to accurately check abnormal information is a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, the embodiments of the present application provide an abnormal information detection method and device based on a multimodal large model, so as to accurately detect abnormal information.

[0005] The present application provides a method for detecting abnormal information based on a multimodal large model, the method comprising:

[0006] The current target file to be detected and the first question prompt for performing anomaly detection on the target file are input into the anomaly information detection model trained based on the multimodal large model to obtain a first answer for answering the first question prompt; the first question prompt is used to ask whether the target file has abnormal information; the first answer is used to answer whether the target file has abnormal information;

[0007] If there is abnormal information in the target file, the target file and a second question prompt for detecting abnormalities in the target file are input into the abnormal information detection model to obtain a second answer for answering the second question prompt; the second question prompt is used to inquire about the specific content of the abnormal information in the target file; the second answer is used to answer the specific content of the abnormal information in the target file;

[0008] The specific content of the abnormal information existing in the target file is obtained based on the second answer.

[0009] The embodiment of the present application also provides an abnormal information detection device based on a multimodal large model, the device comprising:

[0010] A first detection module is used to input the current target file to be detected and a first question prompt for performing anomaly detection on the target file into an anomaly information detection model trained based on a multimodal large model, so as to obtain a first answer for answering the first question prompt; the semantics of the first question prompt is used to ask whether the target file has abnormal information; the first answer is used to answer whether the target file has abnormal information;

[0011] A second detection module is used for inputting the target file and a second question prompt for performing anomaly detection on the target file into an anomaly information detection model if the target file has anomaly information, so as to obtain a second answer for answering the second question prompt; the semantics of the second question prompt is used to inquire about the specific content of the anomaly information in the target file; and the second answer is used to answer the specific content of the anomaly information in the target file;

[0012] The obtaining module is used to obtain specific content of the abnormal information existing in the target file based on the second answer.

[0013] An embodiment of the present application also provides an electronic device, including: a processor and a memory for storing computer program instructions, and when the computer program instructions are executed by the processor, the processor executes the steps of the above method.

[0014] An embodiment of the present application also provides a machine-readable storage medium, which stores computer program instructions. When the computer program instructions are executed, the steps of the above method can be implemented.

[0015] It can be seen from the above technical scheme that in the embodiment of the present application, the target file to be detected and the first question prompt for asking whether there is abnormal information in the target file are input into the trained abnormal information detection model, so that the abnormal information detection model can perform targeted detection on whether there is an abnormality in the target file based on the first question prompt, so as to obtain a first answer for answering the first question prompt. When the first answer indicates that the target is abnormal, the target file and the second question prompt for asking the specific content of the abnormal information in the target file are input into the abnormal information detection model, so that the abnormal information detection model can further detect the specific content of the abnormal information in the target file based on the second question prompt, so as to describe the specific content of the existing abnormal information. In this way, not only can it be determined whether there is abnormal information in the target file, but also a description of the specific content of the abnormal information can be obtained.

[0016] Furthermore, this method of detecting the specific content of abnormal information after determining the existence of abnormal information is compared to directly detecting the specific details of abnormal information in the target file. On the one hand, it is no longer necessary to detect abnormal information and the specific content of abnormal information at the same time. This allows the entire computing power of the abnormal information detection model to be focused on whether abnormal information exists when detecting whether abnormal information exists, which can improve the accuracy of whether abnormal information exists. On the other hand, when detecting the specific content of abnormal information, the prior condition of the existence of abnormal information can be fully utilized, and all computing power can be focused on the specific description of the abnormal information, which can also accurately detect the specific content of the abnormal information.

[0017] Furthermore, combining the abnormal information detection model is a prerequisite for the multimodal large model, which enables the abnormal information detection model to fully integrate the features of different modalities when working to extract more valuable features, thereby improving the accuracy of abnormal information detection and abnormal information content detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of a process for a method provided in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of the process of training a large multimodal model provided in an embodiment of the present application;

[0020] Figure 3 A schematic diagram of a process for fine-tuning a large multimodal model provided in an embodiment of the present application;

[0021] Figure 4 Another schematic diagram of a method according to an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of the structure of the device provided in the embodiment of the present application;

[0023] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and understandable, the technical solutions in the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings.

[0025] See also Figure 1 , Figure 1 A flowchart of the method provided in the embodiment of the present application.

[0026] like Figure 1 As shown, the process may include the following steps:

[0027] S101, input the current target file to be detected and the first question prompt for performing anomaly detection on the target file into the trained anomaly information detection model to obtain a first answer for answering the first question prompt; the semantics of the first question prompt is used to ask whether there is abnormal information in the target file; the first answer is used to answer whether there is abnormal information in the target file.

[0028] In this embodiment, it is known that the multimodal large model can process and understand a variety of different types of input data (such as text, image, video, and audio, etc.), and the abnormal information detection model is obtained by fine-tuning the multimodal large model (the specific process of obtaining the abnormal information detection model will be detailed later and will not be repeated here). Therefore, the abnormal information detection model can also support inputs of different modalities. Thus, the target file can be one of the image modality, video modality, and audio modality, that is, the target file can be an image, video, or audio.

[0029] In this embodiment, the trained abnormal information detection model has the ability to perform abnormal detection on the target file when the target file and the first question prompt are input, and then output a first answer to whether the target file contains abnormal information. The abnormal information detection model extracts features from the target file and the first question prompt, and infers based on the features corresponding to the first question prompt and the features of the target file, and finally generates a relevant answer to the first question prompt (that is, the first answer). If the first answer indicates that abnormal information exists, it is determined that the target file contains abnormal information. If the first answer indicates that abnormal information does not exist, it is determined that the target file does not contain abnormal information.

[0030] During this process, all the computing power and attention of the abnormal information detection model are focused on detecting whether there is abnormal information, which improves the accuracy of the first answer.

[0031] S102, if there is abnormal information in the target file, the target file and the second question prompt for performing abnormality detection on the target file are input into the abnormal information detection model to obtain a second answer for answering the second question prompt; the semantics of the second question prompt is used to inquire about the specific content of the abnormal information existing in the target file; the second answer is used to reply to the specific content of the abnormal information existing in the target file.

[0032] In this embodiment, the trained abnormal information detection model also has the ability to perform abnormality detection on the specific content of the abnormal information of the target file when the target file and the second question prompt are input, and then output the second answer whether the abnormal information exists in the target file. The abnormal information detection model extracts features from the target file and the second question prompt, and infers based on the features corresponding to the second question prompt and the features of the target file, and finally generates the relevant answer to the second question prompt (that is, the second answer).

[0033] During this process, all the computing power and attention of the anomaly information detection model are focused on detecting the specific content of the abnormal information, and the specific content of the abnormal information is detected on the premise that the existence of abnormal information has been determined, which enables the anomaly information detection model to generate a second answer more accurately and quickly.

[0034] S103, obtaining specific content of the abnormal information existing in the target file based on the second answer.

[0035] In this embodiment, the second answer describes in detail the specific content of the abnormal information in the target file, and therefore, the specific content of the abnormal information can be obtained from the second answer.

[0036] In this embodiment, the abnormal information includes at least one of the following information: forged information, private information, and illegal information. Here, the specific content of the abnormal information is related to the category of the abnormal information. For example, if the abnormal information is forged information, the specific content may refer to the location information and the type of forgery of the forged area; if the abnormal information is private information, the specific content may refer to the content of the private information; if the abnormal information is illegal information, the specific content may refer to the content of the illegal information.

[0037] For example, if the abnormal information is private information, the first question prompt may be "Does the file contain private information?", and the first answer may be "Yes" or "No." The second question prompt may be "What private information does the file contain?", and the output second answer may be "The file includes the following private information: XXXX (such as ID number)."

[0038] If the abnormal information is forged information, the first question prompt may be "Does the file contain forged information?", and the first answer may be "Yes" or "No". The second question prompt may be "What forged information does the file contain?", and the second answer output may be "The forged information in the file is forged by splicing, and the location information of the spliced ​​forged area is output."

[0039] If the abnormal information is illegal information, the first question prompt may be "Does the file contain illegal information?", and the first answer may be "Yes" or "No." The second question prompt may be "What illegal information does the file contain?", and the second answer may be "The file includes the following forged information: XXXX (such as violent information)."

[0040] So far, completed Figure 1 The process shown.

[0041] pass Figure 1In the process shown, the target file to be detected and the first question prompt for asking whether the target file contains abnormal information are input into the trained abnormal information detection model, so that the abnormal information detection model can perform targeted detection on whether the target file contains abnormalities based on the first question prompt, so as to obtain a first answer for answering the first question prompt. When the first answer indicates that the target contains abnormalities, the target file and the second question prompt for asking the specific content of the abnormal information in the target file are input into the abnormal information detection model, so that the abnormal information detection model can further detect the specific content of the abnormal information in the target file based on the second question prompt, so as to describe the specific content of the existing abnormal information. In this way, not only can it be determined whether the target file contains abnormal information, but also a description of the specific content of the abnormal information can be obtained.

[0042] Furthermore, this method of detecting the specific content of abnormal information after determining the existence of abnormal information is compared to directly detecting the specific details of abnormal information in the target file. On the one hand, it is no longer necessary to detect abnormal information and the specific content of abnormal information at the same time. This allows the entire computing power of the abnormal information detection model to be focused on whether abnormal information exists when detecting whether abnormal information exists, which can improve the accuracy of whether abnormal information exists. On the other hand, when detecting the specific content of abnormal information, the prior condition of the existence of abnormal information can be fully utilized, and all computing power can be focused on the specific description of the abnormal information, which can also accurately detect the specific content of the abnormal information.

[0043] Furthermore, combining the abnormal information detection model is a prerequisite for the multimodal large model, which enables the abnormal information detection model to fully integrate the features of different modalities when working to extract more valuable features, thereby improving the accuracy of abnormal information detection and abnormal information content detection.

[0044] As an example, there are many types of multimodal large models. The input of the multimodal large model includes but is not limited to the following situations:

[0045] The first case: image modality and text modality;

[0046] The second case: video mode and text mode;

[0047] The third case: audio mode and text mode;

[0048] The fourth case: image modality, audio modality and text modality.

[0049] As an embodiment, the multimodal large model is a visual question answering (VQA) large model, which is a popular image-text cross-modal task that integrates natural language processing and computer vision technology. The task is mainly aimed at intelligent computer recognition and retrieval of image content and giving accurate answers, integrating and applying multiple technologies such as target recognition and detection, intelligent question answering, image attribute classification, and scene analysis. Optionally, as an implementation, common VQA large models include: MiniGPT-4, InstructBLIP, LLaVA, SPHINX, etc.

[0050] The VQA model supports both image and text modes. The abnormal information detection model is obtained by fine-tuning the VQA model. On this basis, the target file can be set to image mode, and the first question prompt and the second question prompt can be set to text mode.

[0051] In the specific implementation, the abnormal information is privacy information. The target image to be detected and the text of "Does the file contain privacy information?" (that is, the first question prompt in text mode) are input into the abnormal information detection model. The abnormal information detection model analyzes the target image to extract image features, and converts the first question prompt into text features. It combines the image features and the first text features to detect whether there is privacy information in the image, and finally outputs the first answer (the first answer is in text mode).

[0052] Furthermore, if the first answer indicates that there is private information in the image, the target image and the text of "What private information does the file contain" (that is, the second question prompt in text mode) are input into the abnormal information detection model. The abnormal information detection model analyzes the target image to extract image features, and converts the second question prompt into text features. The specific content of the private information in the image is detected by combining the image features and the second text features, and finally the second answer (the second answer is in text mode) is output, indicating what private information exists, such as an ID card.

[0053] In this embodiment, the two core tasks of privacy information and privacy information content description are decoupled through the above method, so that all the computing power and attention of the model are focused on the two core tasks respectively, thereby improving the detection rate of privacy data and the accuracy of privacy content description.

[0054] It should be noted that, although the first question prompt and the second question prompt are both in text mode, in one implementation, the first question prompt and the second question prompt may be texts recognized by a large language model from the natural language input by the user.

[0055] The following is a detailed description of the above-mentioned abnormal information detection model:

[0056] See also Figure 2 , Figure 2 A schematic diagram of the process of training a large multimodal model provided in an embodiment of the present application.

[0057] like Figure 2 As shown, the process may include the following steps:

[0058] S201, obtaining a first training sample set, wherein each training sample in the first training sample set includes at least a first file sample, a first question sample for asking whether the first file sample contains abnormal information, and a first answer sample for answering whether the first file sample contains abnormal information; the first question sample has the same semantics as the first question prompt.

[0059] Preferably, as an embodiment, the first question sample and the first question prompt are not only semantically identical but also completely identical, that is, for example, the first question sample is "Does the file contain private information?" and the first question prompt is also "Does the file contain private information?"

[0060] S202, obtaining a second training sample set, wherein each training sample in the second training sample set includes at least a second file sample that has been determined to have abnormal information, a second question sample for asking questions about the specific content of the abnormal information in the second file sample, and a second answer sample for answering the specific content of the abnormal information in the second file sample; the second question sample has the same semantics as the second question prompt.

[0061] In this embodiment, the construction of the first training sample set and the second training sample set is related to the purpose of the abnormal information detection model finally obtained, that is, the content of the first training sample set and the content of the second training sample set match the content of the abnormal information to be detected.

[0062] For example, taking the VQA large model as the base model, if the abnormal information is private information, each training sample in the first training sample set includes a first image sample, a first question sample in text mode "Does the first image sample contain private information", and a first answer sample in text mode "The first image sample contains private information" or "The first image sample does not contain private information".

[0063] Then each training sample in the second training sample set includes a second image sample with existing private information, a second question sample in text mode "What private information does the second image sample contain", and a second answer sample in text mode "The second image sample contains the private information of the ID card" or "The second image sample contains the private information of the face" in text mode.

[0064] Preferably, as an embodiment, the second question sample and the second question prompt are not only semantically identical, but also completely identical. For example, the second question sample is "What private information does the file contain", and the second question prompt is "What private information does the file contain".

[0065] In an optional embodiment, the second image samples in the second training sample set may be the first image samples in the first training sample set that have been determined to contain private information. Of course, the second image samples in the second training sample set may also be image samples that have nothing to do with the first training sample set.

[0066] It should be noted that, depending on the purpose of the abnormal information detection model (such as privacy information detection, illegal information detection, forged information detection, etc.), the first training sample set and the second training sample set constructed are slightly different, but both use the method described below to fine-tune the multimodal large model, which will not be repeated here.

[0067] S203, based on the first training sample set and the second training sample set, fine-tune the initial multimodal large model using a preset fine-tuning algorithm to obtain an abnormal information detection model.

[0068] In this embodiment, the initial multimodal large model is a trained large model (that is, a pre-trained multimodal large model), that is, it has the function of inputting data of different modalities of a conventional multimodal large model, but cannot detect abnormal information. Although the multimodal large model at this time can process information of different modalities, it is not applicable to abnormal information detection. When the user inputs an image (if the image contains abnormal information) and a text question of "Does the image contain abnormal information", the multimodal large model will output "Sorry, I can't answer your question". Therefore, it is necessary to use the first training sample set and the second training sample set constructed above to fine-tune the multimodal large model. The fine-tuning training can allow the multimodal large model to be retrained on abnormal information (forged information, privacy information, and illegal information, etc.), so that it can better learn to process abnormal information, thereby obtaining the above-mentioned abnormal information detection model.

[0069] In other words, in this embodiment, the abnormal information detection model is obtained by fine-tuning the initial multimodal large model using the first training sample set corresponding to the first question prompt, the second training sample set corresponding to the second question prompt, and the preset fine-tuning algorithm. This is different from the related technology of image abnormal information detection based on traditional deep neural networks such as graph neural networks. It does not need to rely on the learning of neural networks from scratch, but performs fine-tuning training on the basis of the pre-trained multimodal large model. Learning and training on a certain basis can obtain the abnormal information detection model faster, which can improve the model training speed.

[0070] The following is combined with Figure 3 How to fine-tune the training of a large multimodal model is described in detail:

[0071] See also Figure 3 , Figure 3 A schematic diagram of the process of fine-tuning the multimodal large model provided in an embodiment of the present application.

[0072] like Figure 3 As shown, the process may include the following steps:

[0073] S301, obtain a pre-trained initial multimodal large model, and select some parameters from all training parameters of the initial multimodal large model as a first fine-tuning parameter subset based on a preset fine-tuning algorithm; the relationship between the number of parameters in the first fine-tuning parameter subset and the total number of parameters of the initial multimodal large model meets the first set parameter requirements.

[0074] In this embodiment, the preset fine-tuning algorithm may be the LORA fine-tuning algorithm or other fine-tuning algorithms in the Swift fine-tuning tool library.

[0075] The first parameter setting requirement means that the ratio of the number of parameters in the first fine-tuning parameter subset to the total number of parameters of the initial multimodal large model is less than or equal to a first threshold, such as 10%.

[0076] The selected part of parameters in the above step S301 may be randomly selected.

[0077] S302, iteratively training parameters in a first fine-tuning parameter subset based on a first training sample set until a first iteration stop condition is met to obtain first adaptation layer parameters, and merging the first adaptation layer parameters with the initial multimodal large model.

[0078] In this embodiment, the parameters in the first fine-tuning parameter subset are iteratively trained based on the first training sample set. In specific implementation, each training sample in the first training sample set is input into the initial multimodal large model. During iterative training, the LORA fine-tuning algorithm allows only the parameters in the first fine-tuning parameter subset to be iteratively optimized, while the remaining parameters participate in the training but are not optimized. Iterative training is performed until the first iteration stop condition is met (for example, the iterative optimization rounds reach the set rounds, such as 100 rounds). At this time, the optimized parameters in the first fine-tuning parameter subset that meet the first iteration stop condition are determined as the first adaptation layer parameters.

[0079] In this embodiment, there are many specific ways to implement the merging of the first adaptation layer parameters with the initial multimodal large model. For example, as an embodiment, for each parameter in the first adaptation layer, the original parameter corresponding to the parameter in the initial multimodal large model and the parameter are operated according to a specified operation (such as addition), and then the operation result is used to replace the original parameter corresponding to the parameter in the initial multimodal large model, and so on, thereby realizing the merging of the first adaptation layer parameters with the initial multimodal large model.

[0080] For example, all parameters of the initial multimodal large model are represented by matrix W, and the first fine-tuning parameter subset is represented by two matrices A and B. When the first iteration stop condition is met, the first adaptation layer parameter expression is B'A', and the parameters of the merged multimodal large model are W merged = W + B'A'.

[0081] S303, selecting some parameters from the merged multimodal large model as a second fine-tuning parameter subset based on a preset fine-tuning algorithm; the relationship between the number of parameters in the second fine-tuning parameter subset and the total number of parameters of the initial multimodal large model meets the second set parameter requirements.

[0082] In this embodiment, the second setting parameter requirement refers to that the ratio of the number of parameters in the second fine-tuning parameter subset to the total number of parameters of the initial multimodal large model is less than or equal to a second threshold, such as 10%. The selected part of the parameters in the above step S303 can be randomly selected.

[0083] It should be noted that the parameters in the first fine-tuning parameter subset and the parameters in the second fine-tuning parameter subset may be completely different, or may partially overlap.

[0084] In this embodiment, the second image samples in the second training sample set may be the portion of the first image samples in the first training sample set that have been determined to contain private information.

[0085] S304, iteratively training the parameters in the second fine-tuning parameter subset based on the second training sample set until the second iteration stop condition is met to obtain the second adaptation layer parameters, and then merging the second adaptation layer parameters with the merged multimodal large model to obtain the abnormal information detection model.

[0086] In this embodiment, the parameters in the second fine-tuning parameter subset are iteratively trained based on the second training sample set. In specific implementation, the following steps are performed: inputting every two training samples in the second training sample set into the merged multimodal large model. During iterative training, the LORA fine-tuning algorithm allows only the parameters in the second fine-tuning parameter subset to be iteratively optimized, while the remaining parameters participate in the training but are not optimized. The iterative training is performed until the second iteration stop condition is met (for example, the iterative optimization rounds reach the set rounds, such as 250 rounds). At this time, the optimized parameters in the second fine-tuning parameter subset that meet the second iteration stop condition are determined as the second adaptation layer parameters.

[0087] In this embodiment, the specific implementation method of merging the second adaptation layer parameters with the merged multimodal large model is similar to the specific implementation method of merging the first adaptation layer parameters with the multimodal large model, which will not be repeated here.

[0088] In another embodiment, fine-tuning the initial multimodal large model is achieved by the following steps:

[0089] A pre-trained initial multimodal large model is obtained, and some parameters are selected from all training parameters of the initial multimodal large model as a third fine-tuning parameter subset based on a preset fine-tuning algorithm; the relationship between the number of parameters in the third fine-tuning parameter subset and the number of all parameters of the initial multimodal large model meets the third setting parameter requirement;

[0090] Iteratively training the parameters in the third fine-tuning parameter subset based on the second training sample set until a third iteration stop condition is met to obtain third adaptation layer parameters, and merging the third adaptation layer parameters with the initial multimodal large model;

[0091] Selecting some parameters from the merged multimodal large model as a fourth fine-tuning parameter subset based on a preset fine-tuning algorithm; the relationship between the number of parameters in the fourth fine-tuning parameter subset and the total number of parameters of the initial multimodal large model meets the fourth setting parameter requirement;

[0092] The parameters in the fourth fine-tuning parameter subset are iteratively trained based on the first training sample set until the fourth iteration stop condition is met to obtain the fourth adaptation layer parameters, and the fourth adaptation layer parameters are further merged with the merged multimodal large model to obtain the abnormal information detection model.

[0093] In this embodiment, the second image samples in the second training sample set are also image samples that have nothing to do with the first training sample set.

[0094] It should be noted that this embodiment is different from Figure 3 In the illustrated embodiment, only the training order of the first training sample set and the second training sample set is swapped, and other specific implementations are similar and will not be elaborated here.

[0095] The above describes in detail how to fine-tune the multimodal large model. In the above embodiment, the first training sample set and the second training sample set are used as training samples to fine-tune the multimodal large model. In this way, during the training process, the features of the object samples (i.e., video, image, or audio) and the text features of the questions and answers are fully integrated, so that the obtained abnormal information detection model has higher accuracy and stronger detection generalization ability when detecting abnormal information.

[0096] Furthermore, in an embodiment, based on the first training sample set and the second training sample set, a preset fine-tuning algorithm is used to perform fine-tuning training on the initial multimodal large model. Compared with traditional full parameter training, an abnormal information detection model can be obtained after very few rounds of training, which can improve the model training speed.

[0097] To explain in more detail, Figure 4 The solution provided in this application is described in more detail by way of specific embodiments:

[0098] In this embodiment, the abnormal information detection model is used to detect privacy information, that is, the abnormal information detection model is essentially a privacy information detection model (for the sake of ease of explanation, the privacy information detection model is used for explanation in this embodiment), and the initial multimodal large model is a VQA large model, which supports image modality and text modality.

[0099] In this embodiment, it specifically includes a model training phase and a privacy information detection model application phase.

[0100] The model training phase specifically includes the following steps:

[0101] 1. Construct a first training sample set, wherein each training sample in the first training sample set includes at least a first image sample, a first question sample in a text mode for asking whether the first image sample has abnormal information, and a first answer sample in a text mode for answering whether the first image sample has abnormal information.

[0102] For example, in the first question-and-answer text sample, the question is “Does the image contain private information?” and the answer is “Yes” or “No”.

[0103] 2. Construct a second training sample set, where each training sample in the second training sample set includes at least a second image sample that has been determined to have abnormal information, a second question sample for asking questions about the specific content of the abnormal information in the second image sample, and a second answer sample for answering the specific content of the abnormal information in the second image sample.

[0104] For example, in the second question-and-answer text sample, the question is “What private information does the image contain?” and the answer is “The image includes the following private information: XXXX (such as ID card number).

[0105] 3. Based on the first training sample set and the second training sample set, the initial multimodal large model is fine-tuned using the fine-tuning algorithm in the LORA or Swift fine-tuning tool library to obtain a privacy information detection model.

[0106] 4. Output the trained privacy information detection model.

[0107] For example, the VQA model used in this example is BLIP2.

[0108] The specific training process includes the following steps:

[0109] (1) Load the pre-trained BLIP2 model.

[0110] (2) Set the configuration corresponding to the LoRA fine-tuning method in the fine-tuning platform, such as r value = 16, lora_alpha = 32, etc.

[0111] (3) Set the data paths of the first training sample set and the second training sample set, and execute the model fine-tuning procedure.

[0112] (4) During the execution of the model fine-tuning program, the first adaptation layer parameters (i.e., the first Adapter model weights) are obtained, which are merged with the BLIP2 model. Then, the second adaptation layer parameters (i.e., the second Adapter model weights) are obtained, which are merged with the merged BLIP2 model to finally obtain the privacy information detection model.

[0113] (5) Deploy the privacy information detection model for operation.

[0114] The application phase of the privacy information detection model includes the following steps:

[0115] 5. Input the image to be detected into the privacy information detection model.

[0116] 6. Input the first question prompt into the privacy information detection model, where the first question prompt is the same as the first question sample, to obtain a first answer.

[0117] 7. Determine whether the first answer indicates that there is private information in the image. If yes, execute the following step 8, otherwise, end the process and output the first answer.

[0118] 8. Input a second question prompt into the privacy information detection model, where the second question prompt is the same as the second question sample, to obtain a second answer.

[0119] 9. Output the second answer, that is, output the private content description information.

[0120] At this point, the description of the method provided in this embodiment is completed. The following describes the device provided in this embodiment of the application:

[0121] See also Figure 5 , Figure 5 A schematic diagram of the structure of the device provided in an embodiment of the present application.

[0122] like Figure 5 As shown, the device 500 includes: a first detection module 501, a second detection module 502, and an acquisition module 503.

[0123] The first detection module 501 is used to input the current target file to be detected and the first question prompt for performing anomaly detection on the target file into the anomaly information detection model trained based on the multimodal large model to obtain a first answer for answering the first question prompt; the semantics of the first question prompt is used to ask whether the target file has abnormal information; the first answer is used to answer whether the target file has abnormal information;

[0124] The second detection module 502 is used for inputting the target file and the second question prompt for performing anomaly detection on the target file into the anomaly information detection model if the target file has anomaly information, so as to obtain a second answer for answering the second question prompt; the semantics of the second question prompt is used to inquire about the specific content of the anomaly information in the target file; the second answer is used to answer the specific content of the anomaly information in the target file;

[0125] The obtaining module 503 is used to obtain specific content of the abnormal information existing in the target file based on the second answer.

[0126] As an embodiment, the multimodal big model includes a visual question answering VQA big model; the VQA big model supports image modality and text modality;

[0127] The target file is in image mode, and the first question prompt and the second question prompt are in text mode.

[0128] As an embodiment, the abnormal information includes at least one of the following information: forged information, private information, and illegal information.

[0129] As an embodiment, the abnormal information detection model is obtained by fine-tuning the initial multimodal large model using a first training sample set corresponding to the first question prompt, a second training sample set corresponding to the second question prompt, and a preset fine-tuning algorithm;

[0130] Each training sample in the first training sample set at least includes a first file sample, a first question sample for asking whether the first file sample contains abnormal information, and a first answer sample for answering whether the first file sample contains abnormal information; the first question sample has the same semantics as the first question prompt;

[0131] Each training sample in the second training sample set includes at least a second file sample that has been determined to have abnormal information, a second question sample for asking questions about the specific content of the abnormal information in the second file sample, and a second answer sample for answering the specific content of the abnormal information in the second file sample; the second question sample has the same semantics as the second question prompt.

[0132] As an embodiment, fine-tuning the initial multimodal large model is implemented by the following steps:

[0133] A pre-trained initial multimodal large model is obtained, and some parameters are selected from all training parameters of the initial multimodal large model as a first fine-tuning parameter subset based on a preset fine-tuning algorithm; the relationship between the number of parameters in the first fine-tuning parameter subset and the number of all parameters of the initial multimodal large model meets the first set parameter requirement;

[0134] Iteratively training the parameters in the first fine-tuning parameter subset based on the first training sample set until a first iteration stop condition is met to obtain first adaptation layer parameters, and merging the first adaptation layer parameters with the initial multimodal large model;

[0135] Selecting some parameters from the merged multimodal large model as a second fine-tuning parameter subset based on a preset fine-tuning algorithm; the relationship between the number of parameters in the second fine-tuning parameter subset and the total number of parameters of the initial multimodal large model meets the second set parameter requirement;

[0136] The parameters in the second fine-tuning parameter subset are iteratively trained based on the second training sample set until the second iteration stop condition is met to obtain the second adaptation layer parameters, and the second adaptation layer parameters are further merged with the merged multimodal large model to obtain the abnormal information detection model.

[0137] As an embodiment, fine-tuning the initial multimodal large model can also be achieved by the following steps:

[0138] A pre-trained initial multimodal large model is obtained, and some parameters are selected from all training parameters of the initial multimodal large model as a third fine-tuning parameter subset based on a preset fine-tuning algorithm; the relationship between the number of parameters in the third fine-tuning parameter subset and the number of all parameters of the initial multimodal large model meets the third setting parameter requirement;

[0139] Iteratively training the parameters in the third fine-tuning parameter subset based on the second training sample set until a third iteration stop condition is met to obtain third adaptation layer parameters, and merging the third adaptation layer parameters with the initial multimodal large model;

[0140] Selecting some parameters from the merged multimodal large model as a fourth fine-tuning parameter subset based on a preset fine-tuning algorithm; the relationship between the number of parameters in the fourth fine-tuning parameter subset and the total number of parameters of the initial multimodal large model meets the fourth setting parameter requirement;

[0141] The parameters in the fourth fine-tuning parameter subset are iteratively trained based on the first training sample set until the fourth iteration stop condition is met to obtain the fourth adaptation layer parameters, and the fourth adaptation layer parameters are further merged with the merged multimodal large model to obtain the abnormal information detection model.

[0142] So far, completed Figure 5 Structural description of the device shown.

[0143] See also Figure 6 , Figure 6 This is a structural diagram of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0144] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0145] Exemplarily, the above-mentioned machine-readable storage medium can be any electronic, magnetic, optical or other physical storage device, which can contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium can be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as optical disk, DVD, etc.), or similar storage medium, or a combination thereof.

[0146] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for detecting abnormal information based on a multimodal large model, characterized in that: The method comprises: Inputting the current target file to be detected and the first question prompt for performing anomaly detection on the target file into the anomaly information detection model trained based on the multimodal large model to obtain a first answer for answering the first question prompt; the semantics of the first question prompt is used to ask whether the target file has abnormal information; the first answer is used to answer whether the target file has abnormal information; If there is abnormal information in the target file, the target file and a second question prompt for performing abnormality detection on the target file are input into the abnormal information detection model to obtain a second answer for answering the second question prompt; the semantics of the second question prompt is used to inquire about the specific content of the abnormal information in the target file; the second answer is used to answer the specific content of the abnormal information in the target file; the semantics of the first question prompt is different from the semantics of the second question prompt; the abnormal information detection model decouples the detection of abnormal information and the description of abnormal information; The specific content of the abnormal information existing in the target file is obtained based on the second answer.

2. The method according to claim 1, characterized in that The multimodal big model includes a visual question answering (VQA) big model; the VQA big model supports image modality and text modality; The target file is in image mode, and the first question prompt and the second question prompt are in text mode.

3. The method according to claim 1, characterized in that The abnormal information includes at least one of the following information: forged information, private information, and illegal information.

4. The method according to claim 1, characterized in that: The abnormal information detection model is obtained by fine-tuning the initial multimodal large model using a first training sample set corresponding to the first question prompt, a second training sample set corresponding to the second question prompt, and a preset fine-tuning algorithm; Each training sample in the first training sample set at least includes a first file sample, a first question sample for asking whether the first file sample has abnormal information, and a first answer sample for answering whether the first file sample has abnormal information; the first question sample has the same semantics as the first question prompt; Each training sample in the second training sample set includes at least a second file sample that has been determined to have abnormal information, a second question sample for asking questions about the specific content of the abnormal information in the second file sample, and a second answer sample for answering the specific content of the abnormal information in the second file sample; the second question sample has the same semantics as the second question prompt.

5. The method according to claim 4, characterized in that The fine-tuning training of the initial multimodal large model is achieved by the following steps: Obtain a pre-trained initial multimodal large model, and select some parameters from all training parameters of the initial multimodal large model as a first fine-tuning parameter subset based on the preset fine-tuning algorithm; the relationship between the number of parameters in the first fine-tuning parameter subset and the number of all parameters of the initial multimodal large model meets the first set parameter requirement; Iteratively training the parameters in the first fine-tuning parameter subset based on the first training sample set until a first iteration stop condition is satisfied to obtain first adaptation layer parameters, and merging the first adaptation layer parameters with the initial multimodal large model; Selecting some parameters from the merged multimodal large model as a second fine-tuning parameter subset based on the preset fine-tuning algorithm; The relationship between the number of parameters in the second fine-tuning parameter subset and the total number of parameters of the initial multimodal large model satisfies the second setting parameter requirement; Iteratively training the parameters in the second fine-tuning parameter subset based on the second training sample set until a second iteration stop condition is met to obtain second adaptation layer parameters, and merging the second adaptation layer parameters with the merged multimodal large model to obtain the abnormal information detection model; or, Obtain a pre-trained initial multimodal large model, and select some parameters from all training parameters of the initial multimodal large model as a third fine-tuning parameter subset based on the preset fine-tuning algorithm; the relationship between the number of parameters in the three fine-tuning parameter subsets and the number of all parameters of the initial multimodal large model meets the third setting parameter requirement; Iteratively training the parameters in the third fine-tuning parameter subset based on the second training sample set until a third iteration stop condition is satisfied to obtain third adaptation layer parameters, and merging the third adaptation layer parameters with the initial multimodal large model; Selecting some parameters from the merged multimodal large model as a fourth fine-tuning parameter subset based on the preset fine-tuning algorithm; The relationship between the number of parameters in the fourth fine-tuning parameter subset and the total number of parameters of the initial multimodal large model satisfies the fourth setting parameter requirement; Iteratively train the parameters in the fourth fine-tuning parameter subset based on the first training sample set until the fourth iteration stop condition is met to obtain fourth adaptation layer parameters, and then merge the fourth adaptation layer parameters with the merged multimodal large model to obtain the abnormal information detection model.

6. An abnormal information detection device based on a multimodal large model, characterized in that: The device comprises: A first detection module is used to input the target file to be detected and a first question prompt for performing anomaly detection on the target file into an anomaly information detection model trained based on the multimodal large model, so as to obtain a first answer for answering the first question prompt; the semantics of the first question prompt is used to ask whether the target file has abnormal information; the first answer is used to answer whether the target file has abnormal information; A second detection module is used for inputting the target file and a second question prompt for performing anomaly detection on the target file into the anomaly information detection model if there is abnormal information in the target file, so as to obtain a second answer for answering the second question prompt; the semantics of the second question prompt is used to inquire about the specific content of the abnormal information in the target file; the second answer is used to answer the specific content of the abnormal information in the target file; the semantics of the first question prompt is different from the semantics of the second question prompt; the abnormal information detection model decouples the detection of abnormal information and the description of abnormal information; An obtaining module is used to obtain specific content of the abnormal information existing in the target file based on the second answer.

7. The device according to claim 6, characterized in that The multimodal big model includes a visual question answering (VQA) big model; the VQA big model supports image modality and text modality; The target file is in image mode, and the first question prompt and the second question prompt are in text mode; and / or, The abnormal information includes at least one of the following information: forged information, private information, and illegal information.

8. The device according to claim 6, characterized in that The abnormal information detection model is obtained by fine-tuning the initial multimodal large model using a first training sample set corresponding to the first question prompt, a second training sample set corresponding to the second question prompt, and a preset fine-tuning algorithm; Each training sample in the first training sample set at least includes a first file sample, a first question sample for asking whether the first file sample has abnormal information, and a first answer sample for answering whether the first file sample has abnormal information; the first question sample has the same semantics as the first question prompt; Each training sample in the second training sample set at least includes a second file sample that has been determined to have abnormal information, a second question sample for asking questions about the specific content of the abnormal information in the second file sample, and a second answer sample for answering the specific content of the abnormal information in the second file sample; the second question sample has the same semantics as the second question; and / or, The fine-tuning training of the initial multimodal large model is achieved by the following steps: Obtain a pre-trained initial multimodal large model, and select some parameters from all training parameters of the initial multimodal large model as a first fine-tuning parameter subset based on the preset fine-tuning algorithm; the relationship between the number of parameters in the first fine-tuning parameter subset and the number of all parameters of the initial multimodal large model meets the first set parameter requirement; Iteratively training the parameters in the first fine-tuning parameter subset based on the first training sample set until a first iteration stop condition is satisfied to obtain first adaptation layer parameters, and merging the first adaptation layer parameters with the initial multimodal large model; Selecting some parameters from the merged multimodal large model as a second fine-tuning parameter subset based on the preset fine-tuning algorithm; the relationship between the number of parameters in the second fine-tuning parameter subset and the total number of parameters of the initial multimodal large model meets the second setting parameter requirement; Iteratively training the parameters in the second fine-tuning parameter subset based on the second training sample set until a second iteration stop condition is met to obtain second adaptation layer parameters, and merging the second adaptation layer parameters with the merged multimodal large model to obtain the abnormal information detection model; or, Obtain a pre-trained initial multimodal large model, and select some parameters from all training parameters of the initial multimodal large model as a third fine-tuning parameter subset based on the preset fine-tuning algorithm; the relationship between the number of parameters in the three fine-tuning parameter subsets and the number of all parameters of the initial multimodal large model meets the third setting parameter requirement; Iteratively training the parameters in the third fine-tuning parameter subset based on the second training sample set until a third iteration stop condition is satisfied to obtain third adaptation layer parameters, and merging the third adaptation layer parameters with the initial multimodal large model; Selecting some parameters from the merged multimodal large model as a fourth fine-tuning parameter subset based on the preset fine-tuning algorithm; the relationship between the number of parameters in the fourth fine-tuning parameter subset and the total number of parameters of the initial multimodal large model meets the fourth setting parameter requirement; Iteratively train the parameters in the fourth fine-tuning parameter subset based on the first training sample set until the fourth iteration stop condition is met to obtain fourth adaptation layer parameters, and then merge the fourth adaptation layer parameters with the merged multimodal large model to obtain the abnormal information detection model.

9. An electronic device, characterized in that: The electronic device includes: Processor; and A computer-readable storage medium having computer program instructions stored therein, wherein the computer program instructions, when executed by the processor, cause the processor to perform the steps of the method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, enable the processor to perform the steps of any one of the methods of claims 1 to 5.

Images