A document risk identification method, apparatus, device, medium, and product.

By identifying multimodal data in documents and fusing modal data below a similarity threshold, and utilizing a large model architecture, the problem of low accuracy in existing document risk identification is solved, achieving more efficient document risk identification.

CN123068635APending Publication Date: 2026-11-10INDUSTRIAL AND COMMERCIAL BANK OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611500155.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-24
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing document risk identification methods mainly target text content for risk identification, resulting in low accuracy and an inability to effectively utilize multimodal data to improve identification performance.

Method used

By identifying multimodal data in documents, machine learning models are used to identify risks in each single modality. When the similarity threshold is below a certain threshold, multimodal data is fused for risk identification. A large model architecture is used to reduce computational load, and modal data with a large amount of data is selected for fusion to improve identification accuracy.

Benefits of technology

It improves the accuracy and efficiency of document risk identification, reduces the computational cost of multimodal data fusion, and enhances the security of document content.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123068635A_ABST
    Figure CN123068635A_ABST
Patent Text Reader

Abstract

This application provides a document risk identification method, apparatus, device, medium, and product, which can be applied to the fields of artificial intelligence technology and fintech. The method includes: determining target multimodal data contained in a target document; the target multimodal data includes N target unimodal data; N is a positive integer; based on a first risk identification model, determining unimodal risk identification results for different target unimodal data among the N target unimodal data; if, among the determined unimodal risk identification results, the similarity between different unimodal risk identification results is less than a preset similarity threshold, determining the multimodal risk identification result of the target document based on a second risk identification model, according to the different target unimodal data among the N target unimodal data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence technology and fintech, specifically to a document risk identification method, apparatus, device, medium, and product. Background Technology

[0002] Many current business processes require documentation for explanation or interaction, and the multimodal data within these documents can improve business processing efficiency. For example, product documentation can combine text, images, and videos to explain a specific product.

[0003] To improve document content security, risk identification is necessary. This includes identifying whether the document poses data security risks or contains content errors.

[0004] However, current document risk identification methods typically only identify risks based on the text content within the document, resulting in low accuracy in document risk identification. Summary of the Invention

[0005] In view of the above problems, embodiments of this application provide a document risk identification method, apparatus, device, medium, and product that can improve the accuracy of document risk identification.

[0006] According to a first aspect of this application, a document risk identification method is provided, comprising: determining target multimodal data contained in a target document; the target multimodal data including N target unimodal data; N being a positive integer; determining unimodal risk identification results for different target unimodal data in the N target unimodal data based on a first risk identification model; and determining the multimodal risk identification result of the target document based on a second risk identification model, according to the different target unimodal data in the N target unimodal data, if the similarity between different unimodal risk identification results is less than a preset similarity threshold.

[0007] According to an embodiment of this application, determining the multimodal risk identification result of the target document based on the second risk identification model and different target single-modal data in the N target single-modal data includes: determining the multimodal risk identification result of the target document based on the second risk identification model and modal data features extracted from the first risk identification model for different target single-modal data in the N target single-modal data.

[0008] According to an embodiment of this application, determining the multimodal risk identification result of the target document based on the second risk identification model and different target single-modal data among the N target single-modal data includes: determining M target single-modal data among the N target single-modal data as M second single-modal data; where M is a positive integer, M < N; and determining the multimodal risk identification result of the target document based on the second risk identification model and the M second single-modal data.

[0009] According to an embodiment of this application, determining M target single-mode data points out of the N target single-mode data points as M second single-mode data points includes: determining the noise data ratio in each of the N target single-mode data points; and determining the M target single-mode data points with a noise data ratio less than a preset noise ratio threshold as M second single-mode data points.

[0010] According to embodiments of this application, different data modalities correspond to different lower limits of data volume; determining M target single-modal data from the N target single-modal data as M second single-modal data includes: among the N target single-modal data, determining M target single-modal data whose data volume is greater than the lower limit of data volume corresponding to their respective data modalities as M second single-modal data.

[0011] According to an embodiment of this application, the method further includes: if the minimum similarity between different single-modal risk identification results is greater than or equal to the preset similarity threshold among the determined single-modal risk identification results, then determining the multimodal risk identification result of the target document based on the determined single-modal risk identification result.

[0012] According to a second aspect of this application, a document risk identification device is provided, comprising: a data determination module, configured to determine target multimodal data contained in a target document; the target multimodal data includes N target unimodal data; N is a positive integer; a first identification module, configured to determine unimodal risk identification results for different target unimodal data in the N target unimodal data based on a first risk identification model; and a second identification module, configured to determine the multimodal risk identification result of the target document based on the second risk identification model and according to the different target unimodal data in the N target unimodal data when the similarity between different unimodal risk identification results is less than a preset similarity threshold in the determined unimodal risk identification results.

[0013] According to a third aspect of this application, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0014] According to a fourth aspect of this application, a computer-readable storage medium is also provided, on which a computer program or instructions are stored, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0015] According to a fifth aspect of this application, a computer program product is also provided, comprising a computer program or instructions that, when executed by a processor, implement the steps of the above-described method. Attached Figure Description

[0016] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0017] Figure 1 This illustration schematically depicts an application scenario of a document risk identification method according to an embodiment of this application.

[0018] Figure 2 A flowchart illustrating a document risk identification method according to an embodiment of this application is shown schematically;

[0019] Figure 3 This schematic diagram illustrates a structural block diagram of a document risk identification device according to an embodiment of this application;

[0020] Figure 4 A block diagram schematically illustrates an electronic device suitable for implementing a document risk identification method according to an embodiment of this application. Detailed Implementation

[0021] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0022] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0023] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0024] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0025] Many current business processes require documentation for explanation or interaction, and the multimodal data within these documents can improve business processing efficiency. For example, product documentation can combine text, images, and videos to explain a specific product. To enhance the security of document content, risk identification is necessary. This includes identifying potential data security risks or content errors. However, current document risk identification methods typically only target the text content, resulting in relatively low accuracy.

[0026] To address the aforementioned technical problems, embodiments of this application provide a document risk identification method. This method acquires multimodal data contained within a document, specifically including text data, audio data, image data, and video data; furthermore, it identifies document risks based on this multimodal data. This method improves the accuracy of document risk identification compared to methods that only utilize text data. Specifically, document risk identification can employ a pre-trained machine learning model. The multimodal data contained within the document is input into the machine learning model to obtain the risk identification prediction results output by the model. The machine learning model can specifically be a large multimodal model.

[0027] In the methods described above, considering the significant computational cost of risk identification for multimodal data, especially if risk identification is performed after fusion of multimodal data, the computational cost of fusion between different modalities is even greater. To reduce the computational cost of document risk identification using multimodal data, document risk identification can be divided into two stages. In the first stage, document risk identification can be performed separately for each single modal data contained in the multimodal data to obtain single-modal risk identification results. Typically, the risk within the same document can be consistent across different modalities, especially those with inherent relationships. For example, the text content in the same document often includes explanations of table content or image content. If the table content or image content contains errors, the text content usually also contains corresponding errors. The degree of error risk in the text content within the same document can be consistent with the degree of error risk in other content (table content or image content). Therefore, if the consistency between the obtained unimodal risk identification results is higher than a threshold, the results can be combined to obtain the final document risk identification result, also known as the multimodal risk identification result. This eliminates the need for a second stage, thus avoiding the need to fuse multimodal data for risk identification and reducing the computational cost associated with fusing different modalities. The consistency between the unimodal risk identification results can be measured by the degree of similarity or difference between them.

[0028] In some cases, the risks within the same document can vary significantly across different modalities of data, leading to substantial differences in the single-modal risk identification results obtained from different single-modal data sets. For example, within the same document, if the image content contains watermark information, the data security risk is relatively high, while the text content typically does not contain watermark information, resulting in a relatively low level of data security risk. Given these significant differences in single-modal risk identification results, a second stage can be implemented to improve the accuracy of document risk identification. In this second stage, document risk identification can be performed based on multimodal data, which can involve fusing different modalities for document risk identification. By fusing different modalities, the correlations between them can be explored, thereby improving the accuracy of the document risk identification results.

[0029] In the above method, machine learning models can be used for document risk identification in both the first and second stages, specifically large models can be used. Correspondingly, in the second stage, the multimodal data can be fused by reusing the modal data features extracted from the multimodal data in the first stage, which can reduce the computational cost of the second stage.

[0030] In the above method, in the second stage, some modal data can be selected from the multimodal data in the first stage for multimodal fusion to perform subsequent document risk identification. By selecting modal data, the amount of modal data that needs to be fused can be reduced, thereby reducing the computational cost of the second stage. The specific selection method can be random selection, or selection based on the risk confidence level identified by each modal data in the first stage, selecting modal data with a risk confidence level higher than the threshold, or selection based on the noise level of each modal data itself, selecting modal data with a noise level less than the threshold.

[0031] Furthermore, considering that a larger volume of single-modal data leads to higher accuracy in document risk identification, we can select modal data with larger volumes to perform the second-stage multimodal fusion and document risk identification. For different modalities, such as text, image, and video modalities, the data volume varies significantly. Therefore, different data volume thresholds can be set for each modality, selecting modal data with volumes exceeding the corresponding thresholds. Using this selected, larger-volume multimodal data for the second-stage multimodal fusion and document risk identification can improve the accuracy of document risk identification.

[0032] It should be noted that the document risk identification method and apparatus provided in the embodiments of this application can be applied to the fields of artificial intelligence technology and fintech. For example, in financial institutions such as banks, for scenarios requiring document risk identification, such as document interaction scenarios, the document risk identification method provided in the embodiments of this application can be used to identify document risks and take corresponding risk handling measures. The document risk identification method and apparatus provided in the embodiments of this application can also be applied to any field other than fintech, and the application field of the document risk identification method and apparatus provided in the embodiments of this application is not limited.

[0033] As used in this paper, the term "model" refers to a model that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs using multiple layers of processing units. A neural network model is an example of a deep learning-based model. In this paper, "model" may also be referred to as a "machine learning model," "learning model," "machine learning network," or "learning network," and these terms are used interchangeably.

[0034] It's important to note that the term "neural network" can refer to a machine learning network based on deep learning. A neural network processes input and provides corresponding output, typically consisting of an input layer, an output layer, and one or more hidden layers between them. Neural networks used in deep learning applications often include many hidden layers, increasing the network's depth. The layers of a neural network are connected sequentially, so that the output of the previous layer serves as the input to the next layer. The input layer receives the input to the neural network, while the output layer's output becomes the final output. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each processing the input from the layer above.

[0035] It should be understood that machine learning generally includes three phases: training, testing, and application (also known as inference). In the training phase, a given model is trained using a large amount of training data, iteratively updating parameter values ​​until the model can consistently generate inferences that meet the expected goals from the training data. Through training, the model can be considered to have learned the relationship between inputs and outputs (also known as the input-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing phase, test inputs are applied to the trained model to test whether it can provide the correct output, thus determining the model's performance. In the application phase, the model can be used to process actual inputs based on the trained parameter values ​​to determine the corresponding output.

[0036] In one or more embodiments described herein, the term "large model" can refer to a deep learning model with a large number of model parameters, which can include hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. Large models can also be called foundational models or basic models. They are pre-trained using large-scale unlabeled corpora to produce pre-trained models with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability, such as large language models and multimodal pre-trained models. It should be understood that in practical applications, large models only require a small number of samples to fine-tune the pre-trained model before being applied to different tasks. Large models can be widely used in natural language processing, computer vision, and other fields. Specifically, they can be applied to computer vision tasks such as visual question answering, image captioning, and image generation, as well as natural language processing tasks such as text-based sentiment classification, text summarization, and machine translation. Major application scenarios for large models can include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0037] Figure 1 The illustration depicts an application scenario of a document risk identification method according to an embodiment of this application. For example... Figure 1As shown, application scenario 100 according to an embodiment of this application may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables. For example, a user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send information, etc.

[0038] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be electronic devices such as smartphones, wearable devices, personal computers, intelligent voice interaction devices, smart home appliances, intelligent vehicles, in-vehicle terminals, aircraft, unmanned vending terminals, and extended reality devices. Extended reality devices can include virtual reality devices, augmented reality devices, and mixed reality devices. A client application for the target application can be installed and run on the terminal devices. This target application can include, but is not limited to, financial transaction applications, payment applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, and social media platform software (these are just examples). Furthermore, this application embodiment does not limit the form of the target application, and it can include, but is not limited to, applications, mini-programs, etc., installed on the terminal devices, and can also be in the form of web pages.

[0039] Server 105 can be a server providing various services, such as a backend management server supporting websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices. Server 105 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data. Server 105 can be the backend server for the aforementioned target application, used to provide backend services to the clients of the target application.

[0040] It should be noted that the document risk identification method provided in this application embodiment can generally be executed by server 105 and / or terminal devices 101-103. Correspondingly, the document risk identification device provided in this application embodiment can generally be set in server 105 and / or terminal devices 101-103.

[0041] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0042] Figure 2 A flowchart illustrating a document risk identification method according to an embodiment of this application is shown schematically. Figure 2 As shown, a document risk identification method 200 according to an embodiment of this application may include steps S210 to S230.

[0043] The embodiments of this application do not limit the executing entity of a document risk identification method. Optionally, a document risk identification method can be applied to any electronic device or software application, specifically to a server, terminal, verification terminal, or risk identification application.

[0044] In step S210, the target multimodal data contained in the target document is determined; the target multimodal data includes N target unimodal data; N is a positive integer.

[0045] In step S220, based on the first risk identification model, the single-modal risk identification result is determined for different target single-modal data in N target single-modal data respectively.

[0046] In step S230, if there is a situation where the similarity between different single-modal risk identification results is less than a preset similarity threshold among the determined single-modal risk identification results, the multimodal risk identification result of the target document is determined based on the second risk identification model and the different target single-modal data in the N target single-modal data.

[0047] The above method can perform document risk identification based on the multimodal data contained in the document, which can improve the accuracy of document risk identification compared to schemes that only use text data to identify document risks.

[0048] The following is a detailed explanation of each step.

[0049] First, in step S210, the target multimodal data contained in the target document is determined; the target multimodal data includes N target single-modal data; N is a positive integer.

[0050] The embodiments of this application are not limited to a target document. Optionally, the target document can be any document, or any document that requires risk identification. In a specific example, the document currently to be transmitted can be determined as the target document, or the received document can be determined as the target document. The risks of the target document are identified, and risk processing operations corresponding to the identified document risk results are performed to improve document security.

[0051] The embodiments of this application are not limited to multimodal data. Optionally, the target multimodal data may be multimodal data contained in a target document, and the multimodal data may contain data of at least two different modalities. Among them, data modalities include, for example, video modalities, text modalities, audio modalities, image modalities, etc. Different data modalities may differ in their source format, their expression method, or their extraction method, etc.

[0052] The embodiments of this application do not limit the method of determining the target multimodal data. Optionally, the target multimodal data can be extracted based on the data modality for the target document. Specifically, for example, text modal data, video modal data, image modal data, table modal data, etc., can be extracted from the target document respectively. In a specific example, for an image containing text in a presentation document, text modal data and image modal data can be extracted from the image respectively.

[0053] Alternatively, data filtering can be performed when determining the target multimodal data. Specifically, this involves filtering out the main content or content with an importance exceeding a threshold from the target document, and then determining the target multimodal data based on the filtered document content. This can reduce the amount of target multimodal data and improve the efficiency of document risk identification. The specific data filtering method is not limited. In one concrete example, deduplication can be performed on the target document to remove duplicate content.

[0054] Furthermore, for the N target unimodal data points contained in the target multimodal data, the data modes of different target unimodal data points can be the same or different. For example, the data in the target document can be divided according to data modes to determine different target unimodal data points belonging to different data modes; alternatively, data determined in the target document through different methods or from different sources can be used as different target unimodal data points. For example, for different chapters in the target document, multiple different text modal data points can be extracted as different target unimodal data points, which can belong to the same data mode, all belonging to the text modality. Optionally, the number N of target unimodal data points contained in the target multimodal data can be a positive integer greater than 1, and the number of different data modes contained in the target multimodal data can be a positive integer greater than 1.

[0055] Second, in step S220, based on the first risk identification model, the single-modal risk identification result is determined for different target single-modal data in N target single-modal data respectively.

[0056] The embodiments of this application are not limited to the first risk identification model. Optionally, the first risk identification model can be used to perform document risk identification separately for single-modal data of different data modalities; wherein, the architecture of the first risk identification model can be a large model, specifically a multimodal large model, which can accept single-modal data input of different data modalities.

[0057] Optionally, the first risk identification model may also include multiple unimodal risk identification sub-models. Different unimodal risk identification sub-models can correspond to different data modalities. Any unimodal risk identification sub-model can be used to perform risk identification on unimodal data of the corresponding data modality to obtain a unimodal risk identification result. Accordingly, for unimodal data of any data modality, the unimodal risk identification sub-model corresponding to that data modality in the first risk identification model can be determined, and document risk identification can be performed on that unimodal data to obtain a unimodal risk identification result. Therefore, by using multiple unimodal risk identification sub-models in the first risk identification model, document risk identification can be performed on different target unimodal data.

[0058] Optionally, the first risk identification model may contain N single-modal risk identification sub-models, which can correspond one-to-one with N target single-modal data. Accordingly, for each of the N target single-modal data, the single-modal risk identification result can be determined based on the corresponding single-modal risk identification sub-model. Specifically, N single-modal risk identification results can be determined.

[0059] The embodiments of this application do not limit the training method of the first risk identification model. Optionally, training samples for training the first risk identification model can be constructed based on historical documents and corresponding historical document risk tags, which can be referred to as first training samples. Specifically, each historical unimodal data point in the historical multimodal data of the historical document can be used as a sample feature, and the historical document risk of the same historical document can be used as a sample tag to construct multiple first training samples for training the first risk identification model. For a first risk identification model with a large model architecture, the first training samples can be used for fine-tuning training. For a first risk identification model containing multiple unimodal risk identification sub-models, the first training samples can be divided according to the data modality of the historical unimodal data in the sample features, and the corresponding unimodal risk identification sub-model can be trained using first training samples with the same data modality.

[0060] Understandably, the computational cost of performing document risk identification using each individual monomodal data point within a multimodal dataset is typically less than the computational cost of performing document risk identification using fused multimodal data. Therefore, it is advisable to first perform document risk identification using each individual monomodal data point within the multimodal dataset, and then, based on the results of each monomodal risk identification, determine whether it is necessary to fuse the multimodal data for further document risk identification, thereby improving the accuracy of document risk identification.

[0061] The embodiments of this application do not limit the form of the single-modal risk identification result. Optionally, the single-modal risk identification result may include at least one of the following: risk type, risk level, risk probability distribution, etc. Based on the different forms and contents of the single-modal risk identification results, appropriate similarity determination methods can be used in subsequent steps.

[0062] The embodiments of this application are not limited to the specific method of determining the single-modal risk identification result for different target single-modal data in N target single-modal data. Optionally, document risk identification can be performed separately for each of the N target single-modal data to obtain N single-modal risk identification results; alternatively, a portion of the target single-modal data can be selected from the N target single-modal data, and then document risk identification can be performed separately for each of the selected target single-modal data.

[0063] Understandably, for N target monomodal data sets, data filtering can be performed to remove some target monomodal data, thereby improving the efficiency of document risk identification. This can be done by filtering based on the noise level of the target monomodal data, selecting those with less noise for document risk identification; for example, video or audio data with high noise levels often struggles to accurately identify document risks. Alternatively, filtering can be based on the amount of target monomodal data, selecting those with larger amounts of data for document risk identification; for example, 1 second of audio data or 0.5 seconds of video data often fails to accurately identify document risks.

[0064] For specific data filtering methods, please refer to the explanations in other embodiments. Data filtering can also be performed in subsequent steps. Different embodiments in this application can be combined with or referenced by each other, and the specific combination method is not limited.

[0065] Third, in step S230, if there is a situation where the similarity between different single-modal risk identification results is less than a preset similarity threshold among the determined single-modal risk identification results, the multimodal risk identification result of the target document is determined based on the second risk identification model and the different target single-modal data in the N target single-modal data.

[0066] The embodiments of this application do not limit the second risk identification model. Optionally, the second risk identification model can be used to perform document risk identification based on data from different modalities, achieving multimodal risk identification. Specifically, it can fuse data from different modalities and identify document risks based on the multimodal data fusion result. The input features of the second risk identification model may include data from different modalities. The second risk identification model can specifically be a neural network model or a large model architecture, for example, a multimodal large model. Optionally, the number of parameters in the second risk identification model can be greater than that in the first risk identification model. By using a larger number of model parameters and multimodal data input, the document risk identification accuracy of the second risk identification model can be improved.

[0067] Of course, the relationship between the first risk identification model and the second risk identification model is not limited. Optionally, the first risk identification model and the second risk identification model can be the same model, have the same model architecture, or be different models using different model architectures. The number of parameters between the first risk identification model and the second risk identification model can be the same or different, and the number of parameters in the first risk identification model can also be greater than the number of parameters in the second risk identification model.

[0068] Optionally, based on the second risk identification model, the multimodal risk identification result of the target document is determined according to the different target single-modal data in the N target single-modal data. Specifically, it can be: based on the second risk identification model, the multimodal risk identification result of the target document is determined according to the modality fusion result between the different target single-modal data in the N target single-modal data. The embodiments of this application do not limit the method of fusing different target single-modal data; specifically, it can be using a cross-attention mechanism or fusing the modal data features of different target single-modal data, etc.

[0069] In a specific example, the second risk identification model may include an attention mechanism, specifically a cross-modal fusion attention mechanism. This mechanism fuses different modalities of input data to extract corresponding data features for document risk identification. For instance, text modal data can be used as the query vector, and image or video modal data as the key-value vector, with features extracted based on the cross-attention mechanism. Alternatively, video modal data can be used as the query vector, and text or image modal data as the key-value vector, with features extracted based on the cross-attention mechanism.

[0070] Understandably, if there are significant differences among the identified single-modal risk identification results—for example, the content error risk level identified based on text modal data is high, but the content error risk level identified based on image modal data is low—a second risk identification model can be further utilized to improve the accuracy of document risk identification. This model identifies the risk of the target document by leveraging the correlation between different modal data input.

[0071] The embodiments of this application do not limit the method of measuring the degree of difference between different unimodal risk identification results. Optionally, the similarity between different unimodal risk identification results can be used to measure the degree of difference between them, and the similarity can be negatively correlated with the degree of difference.

[0072] In a specific example, the unimodal risk identification result can include the unimodal risk level. Accordingly, the degree of difference or similarity between different unimodal risk identification results can be determined based on the absolute value of the difference between different unimodal risk levels. The degree of difference can be positively correlated with the absolute value of the difference, while the degree of similarity can be negatively correlated with the absolute value of the difference. Alternatively, the degree of difference or similarity between different unimodal risk identification results can also be determined based on statistical values ​​such as variance or range, which are used to measure volatility, among the determined different unimodal risk levels.

[0073] In another specific example, the single-modal risk identification result may include probability distribution information of single-modal risk types. Accordingly, based on the degree of difference in probability distribution information between different single-modal risk types, the degree of difference or similarity between different single-modal risk identification results can be determined. The degree of difference in risk identification results may be positively correlated with the degree of difference in probability distribution, and the similarity may be negatively correlated with the degree of difference in probability distribution.

[0074] Of course, for other forms or other content of single-modal risk identification results, appropriate similarity determination methods can be adopted to determine different single-modal risk identification results. The embodiments of this application are not limited to this.

[0075] The embodiments of this application do not limit the determination of single-modal risk identification results to cases where the similarity between different single-modal risk identification results is less than a preset similarity threshold. Optionally, in the single-modal risk identification results determined by the first risk identification model, a similarity can be determined for every two single-modal risk identification results. The determined minimum similarity value is then compared with a preset similarity threshold. If the determined minimum similarity value is less than the preset similarity threshold, it can be determined that the similarity between different single-modal risk identification results is less than the preset similarity threshold.

[0076] Optionally, similarity can be determined based on the overall results of single-modal risk identification identified by the first risk identification model. For example, for multiple numerical forms of single-modal risk identification results (e.g., single-modal risk levels) identified by the first risk identification model, similarity can be determined based on variance values ​​and further compared with a preset similarity threshold. Variance values ​​can be negatively correlated with similarity; the larger the variance, the higher the volatility and the lower the similarity.

[0077] It is understandable that when there is a situation where the similarity between different single-modal risk identification results is less than the preset similarity threshold, it can be characterized that there are large differences or conflicts between the document risk identification results identified based on different target single-modal data. Thus, the above step S230 can be executed to identify risks based on the second risk identification model and integrate different modal data.

[0078] Furthermore, in the opposite case, where there is no situation where the similarity between different single-modal risk identification results is less than the preset similarity threshold, it can be determined that the single-modal risk identification results are highly consistent and do not have significant differences or conflicts. Thus, the identification results of each single-modal risk can be directly integrated to determine the final risk identification result, which is the multimodal risk identification result.

[0079] In a specific example, there can be inherent relationships between data of different modalities within the same document. Text modal data can include explanations and descriptions of image modal data, as well as introductions to video modal data. Image modal data can also include image content that provides graphical explanations of portions of text modal data. Therefore, if the identified single-modal risk identification results are consistent across different modalities, with a relatively high degree of consistency and overall similarity, the overall risk identification result of the target document can be directly determined based on the identified single-modal risk identification results, serving as the multimodal risk identification result.

[0080] For example, if the single-modal risk identification results determined based on the text modal data, image modal data, and table modal data of the target document are all of high risk, then it can be determined that the target document itself has high risk, which may be content error risk or data security risk.

[0081] Of course, if the single-modal risk identification results differ significantly between different modalities of data in the same document—for example, if the data security risk of the text modality is lower while that of the image modality is higher—in order to improve document security and the accuracy of risk identification, data from multiple different modalities can be integrated for comprehensive risk identification.

[0082] Therefore, optionally, the above method may further include: if the minimum similarity between different single-modal risk identification results is greater than or equal to a preset similarity threshold, then, based on the determined single-modal risk identification results, determine the multimodal risk identification result of the target document. This embodiment can determine the final document risk identification result based on the identified single-modal risk identification results when the minimum similarity between the identified single-modal risk identification results is higher than the threshold, thereby reducing the computational load of multimodal document risk identification and improving its efficiency.

[0083] The embodiments of this application do not limit the specific method for determining the multimodal risk identification result of the target document based on the identified individual unimodal risk identification results. Optionally, methods such as voting or weighted summation can be used to synthesize the identified individual unimodal risk identification results to obtain the multimodal risk identification result of the target document. For example, the risk type with the highest proportion among the identified individual unimodal risk identification results can be determined as the multimodal risk identification result of the target document, or the weighted comprehensive value (weighted sum or weighted mean) of the risk levels among the identified individual unimodal risk identification results can be determined as the multimodal risk identification result of the target document. Of course, pre-trained machine learning models or other methods can also be used to determine the multimodal risk identification result based on the identified individual unimodal risk identification results.

[0084] In one optional embodiment, considering that both the first and second risk identification models require extracting data features from the modal data, the data features extracted by the first risk identification model can be reused in the second risk identification model to perform document risk identification. This improves the efficiency of the second risk identification model in performing document risk identification and reduces the computational load. Correspondingly, this feature reuse method can also be used during model training to reduce model training costs.

[0085] Therefore, optionally, based on the second risk identification model, the multimodal risk identification result of the target document is determined according to the different target single-modal data in the N target single-modal data. Specifically, this may include: based on the second risk identification model, determining the multimodal risk identification result of the target document according to the modal data features extracted from the different target single-modal data in the N target single-modal data in the first risk identification model. This embodiment can improve the efficiency of the second risk identification model in performing document risk identification and reduce the computational load of the second risk identification model in performing document risk identification by reusing the modal data features extracted from the target single-modal data by the first risk identification model in the second risk identification model.

[0086] Optionally, the modal data features extracted by the first risk identification model for different target single-modal data from N target single-modal data can be input into the second risk identification model to obtain the multimodal risk identification result output by the second risk identification model. Optionally, the target single-modal data input to the first risk identification model can also be used as the input data for the second risk identification model. During the feature extraction process of the second risk identification model, the modal data features extracted by the first risk identification model can be reused, or new modal data features can be extracted for the input data. Optionally, for the target single-modal data input to the second risk identification model, it can be determined whether the first risk identification model has extracted the modal data features for the target single-modal data. The modal data features already extracted by the first risk identification model can be further reused. For modal data features not extracted by the first risk identification model, the second risk identification model can continue to extract modal data features. The feature extraction layer used for extracting modal data features can be the same between the first and second risk identification models.

[0087] The embodiments of this application are not limited to the specific method of determining the multimodal risk identification result of a target document based on different target single-modal data from N target single-modal data. Optionally, document risk identification can be performed based on N target single-modal data to obtain a multimodal risk identification result; alternatively, a portion of the target single-modal data can be selected from the N target single-modal data, and then document risk identification can be performed based on the selected multiple target single-modal data to obtain a multimodal risk identification result.

[0088] Therefore, optionally, based on the second risk identification model, the multimodal risk identification result of the target document is determined according to the different target single-modal data in N target single-modal data. Specifically, this may include: determining M target single-modal data from the N target single-modal data as M second single-modal data; M is a positive integer, M < N; and determining the multimodal risk identification result of the target document based on the M second single-modal data according to the second risk identification model. This embodiment can filter out a portion of the target single-modal data from the N target single-modal data to perform multimodal document risk identification, which can reduce the computational load of the second risk identification model in performing document risk identification and improve the efficiency of the second risk identification model in performing document risk identification.

[0089] Alternatively, based on the second risk identification model, the multimodal risk identification result can be determined from N target single-modal data. Using the full amount of target multimodal data to perform document risk identification can improve the accuracy of document risk identification. Optionally, M can also be equal to N.

[0090] Optionally, the second risk identification model can be input with data from at least two different modalities to achieve multimodal document risk identification. M can be a positive integer greater than 1, and the M second unimodal data can include different second unimodal data belonging to different modalities. The second risk identification model can fuse second unimodal data from different modalities and identify document risks based on the multimodal data fusion result.

[0091] The embodiments of this application do not limit the specific method for determining the second monomodal data, nor do they limit the specific method for selecting M second monomodal data. Optionally, random selection can be performed, selection can be based on a certain attribute of the target monomodal data, or selection can be based on the confidence level of the monomodal risk identification result. Among the target monomodal data corresponding to monomodal risk identification results with a confidence level greater than a preset confidence threshold, M target monomodal data are determined as M second monomodal data.

[0092] Understandably, the value of M can be fixed or dynamically adjusted. Specifically, it can be adjusted based on the number of target single-modal data points that meet the filtering criteria. For example, target single-modal data points that meet preset filtering criteria can be designated as the second single-modal data point, and the number of target single-modal data points that meet the preset filtering criteria can be determined as the current value of M.

[0093] In one alternative embodiment, the target single-modal data can be filtered based on its noise level. Single-modal data with high noise levels often makes risk identification difficult, thus allowing the filtering of target single-modal data with noise levels below a threshold.

[0094] Optionally, M target single-modal data points out of N target single-modal data points are determined as M second single-modal data points. Specifically, this may include: determining the proportion of noise data in each of the N target single-modal data points; and determining the M target single-modal data points whose noise data proportion is less than a preset noise proportion threshold as the M second single-modal data points. This embodiment can filter out target single-modal data points with a noise proportion less than a threshold from the N target single-modal data points to perform multimodal document risk identification, which can reduce the computational load of the second risk identification model in performing document risk identification and improve the accuracy and efficiency of the second risk identification model in performing document risk identification.

[0095] The embodiments of this application do not limit the specific method for determining the proportion of noise data. Optionally, for any target single-modal data, noise data can be identified, and the ratio between the amount of identified noise data and the amount of data in the target single-modal data can be determined as the proportion of noise data. Of course, the proportion of noise data in the target single-modal data can also be predicted using a pre-trained noise proportion prediction model.

[0096] The embodiments of this application do not limit the noise data in each modality of data. Optionally, for speech modality data, non-human voice speech data, such as knocking sounds, collision sounds, etc., can be identified as noise data, or background noise data can be identified as noise data; for video data, background information can be identified as noise data, such as distant scenery information or information other than people in the video; for image data, background information can also be identified as noise data.

[0097] In another alternative embodiment, considering that the amount of data in a single modal data form affects the accuracy of document risk identification, it is often difficult to perform document risk identification on single modal data with a small amount of data. For example, it is difficult to perform risk identification on text data containing only 2 characters, and it is also difficult to identify risks in image data with a size of 3 pixels × 3 pixels. Therefore, single modal data with a data volume exceeding a threshold can be filtered out. Considering that there is a significant difference in data volume between different modalities—for example, the data volume of video modal data is usually much larger than that of text modal data—different data volume standards can be used for filtering different data modalities to improve the accuracy of modal data filtering. Of course, the method of measuring data volume can also be adjusted according to the data modality. For example, for video modal data, the data volume can be determined according to the duration of the video data, thus setting a lower limit of 10 seconds for the data volume of video modal data, thereby filtering out video modal data with a duration greater than 10 seconds for document risk identification. For text modal data, the data volume can be determined according to information such as the number of characters contained in the text or the text length. More examples are not elaborated here. Alternatively, filtering can be performed based on the amount of information contained in the single-modal data.

[0098] Optionally, different data modalities correspond to different lower limits on data volume. M target single-modal data points out of N target single-modal data points are identified as M second single-modal data points. Specifically, this may include: among the N target single-modal data points, identifying M target single-modal data points whose data volume exceeds the lower limit of their respective data modalities as M second single-modal data points. This embodiment can filter out target single-modal data points with data volumes exceeding the lower threshold from the N target single-modal data points, and perform multimodal document risk identification. This can reduce the computational load of the second risk identification model in performing document risk identification and improve its efficiency.

[0099] It is understood that the above-described embodiment for screening target single-modal data can also be incorporated into step S220, where the screened target single-modal data is determined as the first single-modal data, and document risk identification is performed on each of the first single-modal data based on the first risk identification model to obtain the single-modal risk identification result. The specific combination of embodiments is not limited.

[0100] In one optional embodiment, the modal data can be further simplified to reduce the amount of modal data, thereby improving the efficiency of document risk identification. The simplified modal data can then be used as input to either the first risk identification model or the second risk identification model.

[0101] The embodiments of this application do not limit the specific method of simplifying modal data. Optionally, simplification processing can be performed on each modal data, taking into account its own characteristics and the needs of document risk identification. Optionally, the simplification processing may specifically include noise reduction processing to reduce noise in the modal data.

[0102] For example, for speech modal data, the text converted from speech can be used as text modal data for document risk identification, which can improve the efficiency of document risk identification.

[0103] For example, for text modal data, keywords or key content can be extracted for simplification. Specifically, for the text modal data "A certain product's feature is field-level encryption, which can greatly improve data security," key content such as "field-level encryption" and "improves data security" can be extracted for document risk identification. The amount of extracted keywords is less than the original amount of text modal data, thus improving the efficiency of document risk identification.

[0104] For example, key regions for document risk identification can be extracted from image modal data and video modal data.

[0105] Based on the above method embodiments, embodiments of this application also provide a document risk identification device. The following will be combined with... Figure 3 The device is described in detail.

[0106] Figure 3 The diagram illustrates a structural block diagram of a document risk identification device according to an embodiment of this application.

[0107] like Figure 3 As shown, the document risk identification device 300 provided in this embodiment includes: a data determination module 310, a first identification module 320, and a second identification module 330.

[0108] The data determination module 310 is used to determine the target multimodal data contained in the target document; the target multimodal data includes N target unimodal data; N is a positive integer. In one embodiment, the data determination module 310 can be used to execute the steps S210 and related steps described above, which will not be repeated here.

[0109] The first identification module 320 is used to determine the single-modal risk identification result for different target single-modal data in N target single-modal data based on the first risk identification model. In one embodiment, the first identification module 320 can be used to execute the steps S220 and related steps described above, which will not be repeated here.

[0110] The second identification module 330 is used to determine the multimodal risk identification result of the target document based on the second risk identification model and according to the different target single-modal data in N target single-modal data when the similarity between different single-modal risk identification results in the determined single-modal risk identification results is less than a preset similarity threshold. In one embodiment, the second identification module 330 can be used to execute the steps S230 and related steps described above, which will not be repeated here.

[0111] According to an embodiment of this application, the second identification module 330 can be used to: determine the multimodal risk identification result of the target document based on the second risk identification model and the modal data features extracted from different target single-modal data in the first risk identification model for N target single-modal data.

[0112] According to an embodiment of this application, the second identification module 330 can be used to: determine M target single-modal data from N target single-modal data as M second single-modal data; M is a positive integer, M < N; and determine the multimodal risk identification result of the target document based on the second risk identification model and the M second single-modal data.

[0113] According to an embodiment of this application, the second identification module 330 can be used to: determine the proportion of noise data in N target single-modal data respectively; and determine M target single-modal data whose noise data proportion is less than a preset noise proportion threshold as M second single-modal data among the N target single-modal data.

[0114] According to the embodiments of this application, different data modalities can correspond to different data volume lower limits respectively; the second identification module 330 can be used to: among N target single modal data, determine M target single modal data whose data volume is greater than the data volume lower limit corresponding to the data modality as M second single modal data.

[0115] According to an embodiment of this application, the second identification module 330 can also be used to: determine the multimodal risk identification result of the target document based on the determined single-modal risk identification result, provided that the minimum similarity between different single-modal risk identification results is greater than or equal to a preset similarity threshold.

[0116] According to embodiments of this application, any plurality of modules among the data determination module 310, the first identification module 320, and the second identification module 330 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this application, at least one of the data determination module 310, the first identification module 320, and the second identification module 330 can be at least partially implemented as a hardware circuit, such as a field-programmable gate array, a programmable logic array, a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit, or any other reasonable means of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, at least one of the data determination module 310, the first identification module 320, and the second identification module 330 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0117] The explanation of the above device embodiments can be found in the explanation of other embodiments, and any step in the above method embodiments can be executed by the corresponding module in the above device embodiments.

[0118] Figure 4 A block diagram schematically illustrates an electronic device suitable for implementing a document risk identification method according to an embodiment of this application.

[0119] like Figure 4 As shown, an electronic device 1200 according to an embodiment of this application includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory 1202 or a program loaded from a storage portion 1208 into a random access memory 1203. The processor 1201 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a dedicated microprocessor. The processor 1201 may also include onboard memory for caching purposes. The processor 1201 may include a single processing unit or multiple processing units for executing different steps of the method flow according to an embodiment of this application.

[0120] Random access memory 1203 stores various programs and data required for the operation of electronic device 1200. Processor 1201, read-only memory 1202, and random access memory 1203 are interconnected via bus 1204. Processor 1201 executes various steps of the method flow according to embodiments of this application by executing programs in read-only memory 1202 and / or random access memory 1203. It should be noted that the programs may also be stored in one or more memories other than read-only memory 1202 and random access memory 1203. Processor 1201 may also execute various steps of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0121] According to embodiments of this application, the electronic device 1200 may further include an input / output interface 1205, which is also connected to the bus 1204. The electronic device 1200 may also include one or more of the following components connected to the input / output interface 1205: an input section 1206 including a keyboard, mouse, etc.; an output section 1207 including a cathode ray tube, liquid crystal display, etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card, such as a local area network card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the input / output interface 1205 as needed. A removable medium 1211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1210 as needed so that computer programs read from it can be installed into the storage section 1208 as needed.

[0122] Embodiments of this application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0123] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include the read-only memory 1202, and / or random access memory 1203, and / or one or more memories other than read-only memory 1202 and random access memory 1203 described above.

[0124] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0125] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 1209, and / or installed from the removable medium 1211. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0126] In embodiments of this application, the computer program can be downloaded and installed from a network via communication section 1209, and / or installed from removable medium 1211. When the computer program is executed by processor 1201, it performs the functions defined in the system of this application embodiment. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0127] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0128] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0129] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A document risk identification method, characterized in that, The method includes: Identify the target multimodal data contained in the target document; the target multimodal data includes N target unimodal data; where N is a positive integer; Based on the first risk identification model, the single-modal risk identification results are determined for different target single-modal data in the N target single-modal data respectively; If, among the determined single-modal risk identification results, the similarity between different single-modal risk identification results is less than a preset similarity threshold, the multimodal risk identification result of the target document is determined based on the second risk identification model and the different target single-modal data in the N target single-modal data.

2. The method according to claim 1, characterized in that, The step of determining the multimodal risk identification result of the target document based on the second risk identification model and according to the different target single-modal data in the N target single-modal data includes: Based on the second risk identification model, and according to the modal data features extracted from different target single-modal data in the N target single-modal data in the first risk identification model, the multimodal risk identification result of the target document is determined.

3. The method according to claim 1, characterized in that, The step of determining the multimodal risk identification result of the target document based on the second risk identification model and according to the different target single-modal data in the N target single-modal data includes: M target single-modal data points out of the N target single-modal data points are determined as M second single-modal data points; where M is a positive integer, M < N; Based on the second risk identification model, the multimodal risk identification result of the target document is determined according to the M second single-modal data.

4. The method according to claim 3, characterized in that, The step of determining M target single-modal data points from the N target single-modal data points as M second single-modal data points includes: For each of the N target single-modal data sets, the proportion of noise data in the target single-modal data set is determined. Among the N target single-mode data, the M target single-mode data with a noise data ratio less than a preset noise ratio threshold are determined as M second single-mode data.

5. The method according to claim 3, characterized in that, Different data modalities correspond to different lower limits of data volume; The step of determining M target single-modal data points from the N target single-modal data points as M second single-modal data points includes: Among the N target single-modal data, the M target single-modal data whose data volume is greater than the lower limit of the corresponding data volume of their respective data modes are identified as the M second single-modal data.

6. The method according to claim 1, characterized in that, The method further includes: If the minimum similarity between different single-modal risk identification results is greater than or equal to the preset similarity threshold among the determined single-modal risk identification results, the multimodal risk identification result of the target document is determined based on the determined single-modal risk identification results.

7. A document risk identification device, characterized in that, The device includes: A data determination module is used to determine the target multimodal data contained in the target document; the target multimodal data includes N target unimodal data; where N is a positive integer; The first identification module is used to determine the single-modal risk identification result for different target single-modal data in the N target single-modal data based on the first risk identification model; The second identification module is used to determine the multimodal risk identification result of the target document based on the second risk identification model and according to the different target single-modal data in the N target single-modal data when there is a similarity between different single-modal risk identification results in the determined single-modal risk identification results.

8. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 6.