Multi-mode network security monitoring method and early warning device

Abstract

The invention relates to the technical field of network security, and discloses a multi-modal network security monitoring method and an early warning device, and the method comprises the steps: obtaining historical multi-modal data, generating a security knowledge graph, generating a security monitoring model, operating the security monitoring model, and generating a real-time monitoring result. The early warning device corresponds to the method. According to the application, distributed data acquisition is utilized, an information source is provided for comprehensively solving a network security condition, a security knowledge graph is generated based on historical multi-modal data, entity equipment and an association relationship in network security are represented, a network security situation is better understood by utilizing association, and a structured knowledge framework is provided for monitoring. A multi-head attention mechanism is combined to generate a safety monitoring model, attention calculation is carried out on multi-modal data, the safety monitoring model focuses on key information, the monitoring accuracy is improved, the diversity and relevance of the multi-modal data are fully utilized, efficient multi-modal network safety monitoring is achieved, and a tool and a method are provided for network safety monitoring.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and specifically to a multimodal network security monitoring method and early warning device. Background Art

[0002] Multimodal data of network security is characterized by diversity and correlation. Research on its diversity and correlation can improve the accuracy of network security monitoring.

[0003] Chinese patent number CN118862078A discloses a model training method, device and apparatus, which obtains a training sample set, wherein the training sample set includes multiple sample programs and multiple multimodal samples corresponding to each of the multiple sample programs; iteratively trains the malicious program detection model to be trained based on the training sample set to obtain a trained malicious program detection model, wherein in each round of iterative training, the following operations are performed: for each sample program in the training sample set, the following operations are performed respectively: based on the first similarity between the multiple multimodal samples corresponding to the sample program and the second similarity between each of the multiple multimodal samples corresponding to the sample program and other multimodal samples in the training sample set, the alignment loss value of the sample program is determined; based on the alignment loss value of each sample program in the training sample set, the malicious program detection model is adjusted. However, the invention places too much emphasis on the similarity between multimodal samples and ignores the correlation between multimodal data when applied, resulting in the inability to fully utilize the diversity and correlation of multimodal data.

[0004] In summary, there is an urgent need for a new technical solution for multimodal network security monitoring to fully utilize the diversity and correlation of multimodal data. Summary of the invention

[0005] The purpose of this application is to provide a multimodal network security monitoring method and early warning device to solve the technical problems raised in the above background technology.

[0006] To achieve the above objectives, this application discloses the following technical solutions:

[0007] In a first aspect, the present application discloses a multimodal network security monitoring method, the method comprising:

[0008] S1: Acquire historical multimodal data; wherein the multimodal data at least includes text, images, and data corresponding to network traffic for network security monitoring;

[0009] S2: Generate a security knowledge graph using the historical modal data; wherein the security knowledge graph is used to characterize physical devices and corresponding association relationships in network security;

[0010] S3: Generate a security monitoring model using the historical multimodal data and the security knowledge graph; wherein a multi-head attention mechanism is preset in the security monitoring model, and the multi-head attention mechanism is used to perform attention calculation on the multimodal data. The security monitoring model is used to call the security knowledge graph based on the multimodal data, and perform network security monitoring in combination with the multi-head attention mechanism, and generate corresponding monitoring results;

[0011] S4: Acquire real-time multimodal data and run the security monitoring model to generate real-time monitoring results.

[0012] Preferably, the process of acquiring the multimodal data includes:

[0013] The multimodal data is collected using a preset distributed data collection tool to obtain the multimodal data; wherein the distributed data collection tool is set on different nodes of the network, and corresponding collection tools are set based on the type of multimodal data, and the collection tools at least include a text collection tool, an image collection device and a network traffic monitoring device.

[0014] Preferably, the generation process of the security knowledge graph includes:

[0015] Acquiring the historical multimodal data;

[0016] Performing entity recognition on the historical multimodal data, the entity recognition is used to obtain the corresponding entity device based on the multimodal data recognition;

[0017] Performing relationship extraction based on the result of the entity recognition to obtain the association relationship; wherein the relationship extraction is used to extract the association relationship of the corresponding entity device based on the historical multimodal data;

[0018] The security knowledge graph is generated by utilizing the association relationship.

[0019] Preferably, the operation process of the multi-head attention mechanism includes:

[0020] A1: performing linear transformation on the historical multimodal data to obtain different groups of query vectors, key vectors, and value vectors;

[0021] A2: Different attention results are calculated using different sets of query vectors, key vectors, and value vectors;

[0022] A3: Calculate different attentions to obtain corresponding multimodal fusion attention features, and the multimodal fusion attention features are used to guide the security monitoring model to match the data that needs to be focused on in the multimodal data and call the corresponding security knowledge graph.

[0023] Preferably, the generation process of the safety monitoring model includes:

[0024] Acquire the historical multimodal data and the security knowledge graph;

[0025] Using the historical multimodal data and the security knowledge graph to train the security monitoring model, based on the multimodal data, calling the security knowledge graph, combining the multi-head attention mechanism to perform network security monitoring, and generate corresponding monitoring results;

[0026] The operation process of the safety monitoring model:

[0027] B1: Acquire the multimodal data;

[0028] B2: Obtain the multimodal fusion attention feature;

[0029] B3: using the multimodal fusion attention mechanism to match the data that needs to be focused on in the multimodal data;

[0030] B4: Use the data that needs to be focused on obtained in step B4 to match the corresponding association relationship in the security knowledge graph, output the association relationship, and increase the frequency of collecting the multimodal data of the corresponding physical device based on the association relationship.

[0031] In a second aspect, the present application discloses a multimodal network security monitoring and early warning device, which is applicable to the multimodal network security monitoring method as described above, and the early warning device includes:

[0032] A data acquisition module, wherein the data acquisition module is configured to: acquire historical multimodal data; wherein the multimodal data at least includes text, images and data corresponding to network traffic for network security monitoring;

[0033] A security knowledge graph generation module, wherein the security knowledge graph generation module is configured to: generate a security knowledge graph using the historical modal data; wherein the security knowledge graph is used to characterize physical devices and corresponding association relationships in network security;

[0034] A security monitoring model generation module, wherein the security monitoring model generation module is configured to: generate a security monitoring model using the historical multimodal data and the security knowledge graph; wherein a multi-head attention mechanism is preset in the security monitoring model, and the multi-head attention mechanism is used to perform attention calculation on the multimodal data; the security monitoring model is used to call the security knowledge graph based on the multimodal data, and perform network security monitoring in combination with the multi-head attention mechanism, and generate corresponding monitoring results;

[0035] The safety monitoring model operation module is configured to: obtain real-time multimodal data and operate the safety monitoring model to generate real-time monitoring results.

[0036] Preferably, the process of acquiring the multimodal data includes:

[0037] The multimodal data is collected using a preset distributed data collection tool to obtain the multimodal data; wherein the distributed data collection tool is set on different nodes of the network, and corresponding collection tools are set based on the type of multimodal data, and the collection tools at least include a text collection tool, an image collection device and a network traffic monitoring device.

[0038] Preferably, the generation process of the security knowledge graph includes:

[0039] Acquiring the historical multimodal data;

[0040] Performing entity recognition on the historical multimodal data, the entity recognition is used to obtain the corresponding entity device based on the multimodal data recognition;

[0041] Performing relationship extraction based on the result of the entity recognition to obtain the association relationship; wherein the relationship extraction is used to extract the association relationship of the corresponding entity device based on the historical multimodal data;

[0042] The security knowledge graph is generated by utilizing the association relationship.

[0043] Preferably, the operation process of the multi-head attention mechanism includes:

[0044] A1: performing linear transformation on the historical multimodal data to obtain different groups of query vectors, key vectors, and value vectors;

[0045] A2: Different attention results are calculated using different sets of query vectors, key vectors, and value vectors;

[0046] A3: Calculate different attentions to obtain corresponding multimodal fusion attention features, and the multimodal fusion attention features are used to guide the security monitoring model to match the data that needs to be focused on in the multimodal data and call the corresponding security knowledge graph.

[0047] Preferably, the generation process of the safety monitoring model includes:

[0048] Acquire the historical multimodal data and the security knowledge graph;

[0049] Using the historical multimodal data and the security knowledge graph to train the security monitoring model, based on the multimodal data, calling the security knowledge graph, combining the multi-head attention mechanism to perform network security monitoring, and generate corresponding monitoring results;

[0050] The operation process of the safety monitoring model:

[0051] B1: Acquire the multimodal data;

[0052] B2: Obtain the multimodal fusion attention feature;

[0053] B3: using the multimodal fusion attention mechanism to match the data that needs to be focused on in the multimodal data;

[0054] B4: Use the data that needs to be focused on obtained in step B4 to match the corresponding association relationship in the security knowledge graph, output the association relationship, and increase the frequency of collecting the multimodal data of the corresponding physical device based on the association relationship.

[0055] Beneficial effects: The multimodal network security monitoring method and early warning device of the present application utilize distributed data collection to provide a rich source of information for a comprehensive understanding of the network security status, generate a security knowledge graph based on historical multimodal data, clearly characterize the physical devices and associations in network security, and use the association to better understand the network security situation, providing a structured knowledge framework for monitoring, combining a multi-head attention mechanism to generate a security monitoring model, and performing attention calculations on multimodal data, so that the security monitoring model can focus on key information and improve monitoring accuracy, thereby making full use of the diversity and correlation of multimodal data, realizing efficient multimodal network security monitoring, and providing more powerful tools and methods for network security monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 A flowchart of a multimodal network security monitoring method provided in an embodiment of the present application;

[0058] Figure 2 This is a structural block diagram of the multimodal network security monitoring and early warning device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present application are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0060] In this article, the term "comprising" is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "comprising..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0061] The first aspect of this embodiment discloses Figure 1 A multimodal network security monitoring method is shown, the method comprising:

[0062] S1: Acquire historical multimodal data; wherein the multimodal data at least includes text, images, and data corresponding to network traffic for network security monitoring;

[0063] S2: Generate a security knowledge graph using historical modal data; the security knowledge graph is used to represent physical devices and corresponding associations in network security;

[0064] S3: Generate a security monitoring model using historical multimodal data and security knowledge graph; wherein a multi-head attention mechanism is preset in the security monitoring model, and the multi-head attention mechanism is used to perform attention calculation on multimodal data. The security monitoring model is used to call the security knowledge graph based on multimodal data, and combine the multi-head attention mechanism to perform network security monitoring and generate corresponding monitoring results;

[0065] S4: Obtain real-time multimodal data and run the security monitoring model to generate real-time monitoring results.

[0066] In a specific application, the text data of this embodiment may be record information in a network system log, the image data may be an image of a status indicator light of a network device, and the network traffic data may be statistics of data transmission volume between different network nodes.

[0067] Based on the above, this embodiment uses distributed data collection to provide a rich source of information for a comprehensive understanding of network security status, generates a security knowledge graph based on historical multimodal data, clearly characterizes physical devices and associations in network security, and uses correlation to better understand the network security situation, providing a structured knowledge framework for monitoring. It combines a multi-head attention mechanism to generate a security monitoring model, performs attention calculations on multimodal data, and enables the security monitoring model to focus on key information and improve monitoring accuracy, thereby making full use of the diversity and correlation of multimodal data, achieving efficient multimodal network security monitoring, and providing more powerful tools and methods for network security monitoring.

[0068] Specifically, the process of acquiring multimodal data includes:

[0069] Use a preset distributed data collection tool to collect multimodal data and obtain multimodal data; wherein the distributed data collection tool is set on different nodes of the network, and corresponding collection tools are set based on the multimodal data type, and the collection tools at least include text collection tools, image collection equipment and network traffic monitoring equipment.

[0070] In a specific application, the text collection tool of this embodiment can be a special log analysis software, which is deployed on each server node to collect text information in the system operation log; the image collection device can be a camera installed in the network room, which is used to capture images of device status indicator lights; the network traffic monitoring device can be a traffic analyzer deployed at key network nodes to monitor network traffic data in real time.

[0071] Based on the above, this embodiment utilizes distributed collection tools to set specific collection devices for different modal data, thereby achieving efficient and comprehensive acquisition of multimodal data, giving full play to the diversity advantages of multimodal data, and providing a richer source of information for network security monitoring.

[0072] Specifically, the generation process of the security knowledge graph includes:

[0073] Acquire historical multimodal data;

[0074] Perform entity recognition on historical multimodal data, where entity recognition is used to obtain corresponding entity devices based on multimodal data recognition;

[0075] Relationship extraction is performed based on the results of entity recognition to obtain association relationships; wherein the relationship extraction is used to extract the association relationships of corresponding entity devices based on historical multimodal data;

[0076] Use association relationships to generate a security knowledge graph.

[0077] In a specific application, this embodiment identifies physical devices such as servers and routers from historical multimodal data by analyzing text; then extracts association relationships such as connection relationships and communication relationships between these physical devices based on network traffic data and image information; and uses these association relationships to build a security knowledge graph.

[0078] Based on the above, this embodiment uses multimodal data analysis to generate a security knowledge graph through entity recognition and relationship extraction, realizes the structured representation of physical devices and association relationships in network security, and uses the correlation of multimodal data to provide knowledge support for network security monitoring.

[0079] Specifically, the operation process of the multi-head attention mechanism includes:

[0080] A1: Perform linear transformation on historical multimodal data to obtain different groups of query vectors, key vectors and value vectors. In a simple example, this embodiment performs linear transformation on historical text data to obtain a group of query vectors, key vectors and value vectors, and performs similar processing on image data and network traffic data to obtain different groups of vectors. In this regard, this embodiment uses linear transformation to process historical multimodal data to provide a basic vector representation for the calculation of the multi-head attention mechanism, which helps to better capture the features of different modal data and utilize the diversity of multimodal data.

[0081] A2: Different attention results are calculated using different groups of query vectors, key vectors, and value vectors; in a simple example, this embodiment calculates the attention results of text data, the attention results of image data, and the attention results of network traffic data based on vectors of different modal data; in this regard, this embodiment uses vector calculation to process different groups of vectors to obtain different attention results, provide intermediate results for the generation of multimodal fusion attention features, and use the correlation of multimodal data for comprehensive analysis;

[0082] A3: Calculate different attentions to obtain corresponding multimodal fusion attention features. The multimodal fusion attention features are used to guide the security monitoring model to match the data that needs to be focused on in the multimodal data and call the corresponding security knowledge graph. In a simple example, this embodiment obtains the multimodal fusion attention features by further processing the results of different attentions. For example, when the network traffic is abnormal and a specific error appears in the system log, and the device image display is abnormal, this feature will guide the security monitoring model to focus on these data and call the relevant association relationships in the security knowledge graph.

[0083] Based on the above, this embodiment uses a multi-head attention mechanism to generate multimodal fusion attention features, realizes comprehensive analysis of multimodal data, provides guidance for the security monitoring model, and makes full use of the correlation of multimodal data for targeted monitoring.

[0084] Specifically, the generation process of the security monitoring model includes:

[0085] Acquire historical multimodal data and security knowledge graphs;

[0086] Use historical multimodal data and security knowledge graphs to train security monitoring models based on multimodal data, call security knowledge graphs, combine multi-head attention mechanisms to perform network security monitoring, and generate corresponding monitoring results;

[0087] The operation process of the safety monitoring model:

[0088] B1: Acquire multimodal data;

[0089] B2: Obtain multimodal fusion attention features;

[0090] B3: Use the multimodal fusion attention mechanism to match the data that needs to be focused on in the multimodal data;

[0091] B4: Use the data that needs to be focused on obtained in step B4 to match the corresponding association relationship in the security knowledge graph, output the association relationship, and increase the frequency of collecting multimodal data of the corresponding physical device based on the association relationship.

[0092] In a specific application, this embodiment obtains historical multimodal data and a constructed security knowledge graph, and uses these data to train a security monitoring model, so that it can call the security knowledge graph based on the multimodal data and combine the multi-head attention mechanism to perform network security monitoring, and generate accurate monitoring results; during operation, first obtain real-time multimodal data, and then match the data that needs to be focused on in the data according to the multimodal fusion attention features, and search for associations in the security knowledge graph to increase the frequency of collecting multimodal data of related entity devices.

[0093] It should be noted that the monitoring result of this embodiment may be a judgment result on whether the network security is normal or abnormal.

[0094] Based on the above, this embodiment uses data and knowledge graphs to train security monitoring models and combines multimodal fusion attention features and security knowledge graphs during operation, thereby achieving efficient and accurate network security monitoring, making full use of the diversity and correlation of multimodal data, and improving the pertinence and timeliness of monitoring.

[0095] The second aspect of this embodiment discloses Figure 2A multimodal network security monitoring and early warning device is shown, and the early warning device is applicable to the above multimodal network security monitoring method, and the early warning device includes:

[0096] A data acquisition module, the data acquisition module is configured to: acquire historical multimodal data; wherein the multimodal data at least includes text, images and data corresponding to network traffic for network security monitoring;

[0097] A security knowledge graph generation module, wherein the security knowledge graph generation module is configured to: generate a security knowledge graph using historical modal data; wherein the security knowledge graph is used to represent physical devices and corresponding associations in network security;

[0098] A security monitoring model generation module, the security monitoring model generation module is configured to: generate a security monitoring model using historical multimodal data and a security knowledge graph; wherein a multi-head attention mechanism is preset in the security monitoring model, and the multi-head attention mechanism is used to perform attention calculation on the multimodal data; the security monitoring model is used to call the security knowledge graph based on the multimodal data, and combine the multi-head attention mechanism to perform network security monitoring and generate corresponding monitoring results;

[0099] The security monitoring model operation module is configured to obtain real-time multimodal data and run the security monitoring model to generate real-time monitoring results.

[0100] It should be noted that the multimodal network security monitoring and early warning device of the present embodiment corresponds to the aforementioned multimodal network security monitoring method. Therefore, the contents not specifically described in the multimodal network security monitoring and early warning device of the present embodiment may include but are not limited to functional definitions, working principles, and technical effects, etc., and may refer to the description of the aforementioned multimodal network security monitoring method, and this text will not elaborate on them here.

[0101] Specifically, the process of acquiring multimodal data includes:

[0102] Use a preset distributed data collection tool to collect multimodal data and obtain multimodal data; wherein the distributed data collection tool is set on different nodes of the network, and corresponding collection tools are set based on the multimodal data type, and the collection tools at least include text collection tools, image collection equipment and network traffic monitoring equipment.

[0103] Specifically, the generation process of the security knowledge graph includes:

[0104] Acquire historical multimodal data;

[0105] Perform entity recognition on historical multimodal data, where entity recognition is used to obtain corresponding entity devices based on multimodal data recognition;

[0106] Relationship extraction is performed based on the results of entity recognition to obtain association relationships; wherein the relationship extraction is used to extract the association relationships of corresponding entity devices based on historical multimodal data;

[0107] Use association relationships to generate a security knowledge graph.

[0108] Specifically, the operation process of the multi-head attention mechanism includes:

[0109] A1: Perform linear transformation on historical multimodal data to obtain different groups of query vectors, key vectors, and value vectors;

[0110] A2: Different attention results are calculated using different sets of query vectors, key vectors, and value vectors;

[0111] A3: Calculate different attentions and obtain corresponding multimodal fusion attention features. The multimodal fusion attention features are used to guide the security monitoring model to match the data that needs to be focused on in the multimodal data and call the corresponding security knowledge graph.

[0112] Specifically, the generation process of the security monitoring model includes:

[0113] Acquire historical multimodal data and security knowledge graphs;

[0114] Use historical multimodal data and security knowledge graphs to train security monitoring models based on multimodal data, call security knowledge graphs, combine multi-head attention mechanisms to perform network security monitoring, and generate corresponding monitoring results;

[0115] The operation process of the safety monitoring model:

[0116] B1: Acquire multimodal data;

[0117] B2: Obtain multimodal fusion attention features;

[0118] B3: Use the multimodal fusion attention mechanism to match the data that needs to be focused on in the multimodal data;

[0119] B4: Use the data that needs to be focused on obtained in step B4 to match the corresponding association relationship in the security knowledge graph, output the association relationship, and increase the frequency of collecting multimodal data of the corresponding physical device based on the association relationship.

[0120] In summary, the multimodal network security monitoring method and early warning device of this embodiment utilize distributed data collection to provide a rich source of information for a comprehensive understanding of the network security status, generate a security knowledge graph based on historical multimodal data, clearly characterize the physical devices and associations in network security, and use the correlation to better understand the network security situation, providing a structured knowledge framework for monitoring, combining the multi-head attention mechanism to generate a security monitoring model, and performing attention calculation on multimodal data, so that the security monitoring model can focus on key information and improve monitoring accuracy, thereby making full use of the diversity and correlation of multimodal data, realizing efficient multimodal network security monitoring, and providing more powerful tools and methods for network security monitoring.

[0121] In the embodiments provided in the present application, it should be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, code or any appropriate combination thereof. For hardware implementation, the processor can be implemented in one or more of the following units: application specific integrated circuit (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), field programmable gate array (FPGA), processor, controller, microcontroller, microprocessor, other electronic units designed to implement the functions described herein or their combination. For software implementation, part or all of the flow of the embodiment can be completed by instructing the relevant hardware through a computer program. When implemented, the above program can be stored in a computer-readable storage medium or transmitted as one or more instructions or codes on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein the communication medium includes any medium that is convenient for transmitting a computer program from one place to another. The storage medium can be any available medium that a computer can access. The computer-readable storage medium can include but is not limited to RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer.

[0122] Finally, it should be noted that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A multimodal network security monitoring method, characterized in that: The method includes: S1: Acquire historical multimodal data; wherein the multimodal data at least includes text, images, and data corresponding to network traffic for network security monitoring; S2: Generate a security knowledge graph using the historical modal data; wherein the security knowledge graph is used to characterize physical devices and corresponding association relationships in network security; S3: Generate a security monitoring model using the historical multimodal data and the security knowledge graph; wherein a multi-head attention mechanism is preset in the security monitoring model, and the multi-head attention mechanism is used to perform attention calculation on the multimodal data. The security monitoring model is used to call the security knowledge graph based on the multimodal data, and perform network security monitoring in combination with the multi-head attention mechanism, and generate corresponding monitoring results; S4: Acquire real-time multimodal data and run the security monitoring model to generate real-time monitoring results.

2. The multimodal network security monitoring method according to claim 1, characterized in that: The process of acquiring the multimodal data includes: The multimodal data is collected by using a preset distributed data collection tool to obtain the multimodal data; wherein the distributed data collection tool is set on different nodes of the network, and corresponding collection tools are set based on the type of multimodal data, and the collection tools at least include text collection tools, image collection equipment and network traffic monitoring equipment.

3. The multimodal network security monitoring method according to claim 1, characterized in that: The generation process of the security knowledge graph includes: Acquiring the historical multimodal data; Performing entity recognition on the historical multimodal data, the entity recognition is used to obtain the corresponding entity device based on the multimodal data recognition; Performing relationship extraction based on the result of the entity recognition to obtain the association relationship; wherein the relationship extraction is used to extract the association relationship of the corresponding entity device based on the historical multimodal data; The security knowledge graph is generated by utilizing the association relationship.

4. The multimodal network security monitoring method according to claim 1, characterized in that: The operation process of the multi-head attention mechanism includes: A1: performing linear transformation on the historical multimodal data to obtain different groups of query vectors, key vectors, and value vectors; A2: Different attention results are calculated using different sets of query vectors, key vectors, and value vectors; A3: Calculate different attentions to obtain corresponding multimodal fusion attention features, and the multimodal fusion attention features are used to guide the security monitoring model to match the data that needs to be focused on in the multimodal data and call the corresponding security knowledge graph.

5. The multimodal network security monitoring method according to claim 4, characterized in that: The generation process of the safety monitoring model includes: Acquire the historical multimodal data and the security knowledge graph; Using the historical multimodal data and the security knowledge graph to train the security monitoring model, based on the multimodal data, calling the security knowledge graph, combining the multi-head attention mechanism to perform network security monitoring, and generate corresponding monitoring results; The operation process of the safety monitoring model: B1: Acquire the multimodal data; B2: Obtain the multimodal fusion attention feature; B3: using the multimodal fusion attention mechanism to match the data that needs to be focused on in the multimodal data; B4: Use the data that needs to be focused on obtained in step B4 to match the corresponding association relationship in the security knowledge graph, output the association relationship, and increase the frequency of collecting the multimodal data of the corresponding physical device based on the association relationship.

6. A multimodal network security monitoring and early warning device, the early warning device is applicable to the multimodal network security monitoring method according to any one of claims 1 to 5, characterized in that: The early warning device includes: A data acquisition module, wherein the data acquisition module is configured to: acquire historical multimodal data; wherein the multimodal data at least includes text, images and data corresponding to network traffic for network security monitoring; A security knowledge graph generation module, wherein the security knowledge graph generation module is configured to: generate a security knowledge graph using the historical modal data; wherein the security knowledge graph is used to characterize physical devices and corresponding association relationships in network security; A security monitoring model generation module, wherein the security monitoring model generation module is configured to: generate a security monitoring model using the historical multimodal data and the security knowledge graph; wherein a multi-head attention mechanism is preset in the security monitoring model, and the multi-head attention mechanism is used to perform attention calculation on the multimodal data; the security monitoring model is used to call the security knowledge graph based on the multimodal data, and perform network security monitoring in combination with the multi-head attention mechanism, and generate corresponding monitoring results; The safety monitoring model operation module is configured to: obtain real-time multimodal data and operate the safety monitoring model to generate real-time monitoring results.

7. The multi-modal network security monitoring and early warning device according to claim 6 is characterized in that: The process of acquiring the multimodal data includes: The multimodal data is collected by using a preset distributed data collection tool to obtain the multimodal data; wherein the distributed data collection tool is set on different nodes of the network, and corresponding collection tools are set based on the type of multimodal data, and the collection tools at least include text collection tools, image collection equipment and network traffic monitoring equipment.

8. The multi-modal network security monitoring and early warning device according to claim 6 is characterized in that: The generation process of the security knowledge graph includes: Acquiring the historical multimodal data; Performing entity recognition on the historical multimodal data, the entity recognition is used to obtain the corresponding entity device based on the multimodal data recognition; Performing relationship extraction based on the result of the entity recognition to obtain the association relationship; wherein the relationship extraction is used to extract the association relationship of the corresponding entity device based on the historical multimodal data; The security knowledge graph is generated by utilizing the association relationship.

9. The multi-modal network security monitoring and early warning device according to claim 6, characterized in that: The operation process of the multi-head attention mechanism includes: A1: performing linear transformation on the historical multimodal data to obtain different groups of query vectors, key vectors, and value vectors; A2: Different attention results are calculated using different sets of query vectors, key vectors, and value vectors; A3: Calculate different attentions to obtain corresponding multimodal fusion attention features, and the multimodal fusion attention features are used to guide the security monitoring model to match the data that needs to be focused on in the multimodal data and call the corresponding security knowledge graph.

10. The multi-modal network security monitoring and early warning device according to claim 9, characterized in that: The generation process of the safety monitoring model includes: Acquire the historical multimodal data and the security knowledge graph; Using the historical multimodal data and the security knowledge graph to train the security monitoring model, based on the multimodal data, calling the security knowledge graph, combining the multi-head attention mechanism to perform network security monitoring, and generate corresponding monitoring results; The operation process of the safety monitoring model: B1: Acquire the multimodal data; B2: Obtain the multimodal fusion attention feature; B3: using the multimodal fusion attention mechanism to match the data that needs to be focused on in the multimodal data; B4: Use the data that needs to be focused on obtained in step B4 to match the corresponding association relationship in the security knowledge graph, output the association relationship, and increase the frequency of collecting the multimodal data of the corresponding physical device based on the association relationship.

Images