Network information security monitoring method and system based on artificial intelligence

Abstract

The invention relates to the technical field of network information security, in particular to a network information security monitoring method based on artificial intelligence, which comprises the following steps: step 1, interface information acquisition: monitoring a user interface in real time and capturing all information on the interface, including a desktop, an application window and document content, capturing a snapshot of the user interface at high frequency through a screen capture tool, extracting a video stream from the user interface and separating an audio stream, and extracting text information from an image by using an optical character recognition technology; according to the scheme, the character, image and sound information of the user interface is collected in real time, advanced artificial intelligence algorithms such as a convolutional neural network and potential semantic analysis are used for deep risk analysis, prohibited information grades are accurately divided, flexible and diverse processing modes such as pop-up window shielding are adopted according to the grades, and the prohibited information classification efficiency is improved. The network information security is comprehensively, accurately and effectively monitored, the defects of the traditional technology are overcome, and a powerful guarantee is provided for the health and stability of the network environment.

Description

Technical Field

[0001] The present invention relates to the field of network information security technology, and in particular to a network information security monitoring method and system based on artificial intelligence. Background Art

[0002] With the rapid development of information technology, the Internet has been deeply integrated into people's lives and work. Various activities such as online office, distance education, social entertainment, etc. are widely carried out on the Internet platform. At the same time, the problem of network information security has become increasingly prominent. Various prohibited information is spread wantonly in cyberspace, causing serious harm to individuals, enterprises and even society.

[0003] Traditional network information security monitoring methods have many limitations. On the one hand, most of them can only detect a single type of information, such as focusing only on keyword filtering of text content, while ignoring prohibited information that may be contained in images and audio. This allows criminals to spread prohibited content through pictures, videos, voice, etc., and evade detection by traditional monitoring methods. On the other hand, traditional technology lacks the ability to deeply understand information when analyzing information, and it is difficult to accurately identify content with ambiguous semantics and implicit prohibited meanings. For example, some suggestive or metaphorical text expressions, as well as images that hide prohibited elements through image processing methods, are often not effectively identified by traditional monitoring methods.

[0004] In addition, existing monitoring technologies lack flexibility and accuracy when processing prohibited information. Once possible prohibited content is detected, simple and crude shielding or filtering methods are often adopted, such as directly banning the entire webpage or account, which may not only misjudge normal information and affect the user's normal use experience, but may also fail to effectively prevent the spread of prohibited information, because criminals can republish by changing the dissemination channel or modifying the form of information. In this context, there is an urgent need for a network information security monitoring method and system that can comprehensively collect information, accurately analyze risks, and flexibly and effectively handle prohibited content. Based on the development of artificial intelligence technology, the present invention arises at the historic moment, aiming to fill the deficiencies of the prior art and provide more reliable protection for network information security.

[0005] Therefore, the present invention proposes an artificial intelligence-based network information security monitoring method and system, which collects text, image and sound information of the user interface in real time, uses advanced artificial intelligence algorithms such as convolutional neural networks and latent semantic analysis to conduct in-depth risk analysis, accurately divides the level of prohibited information, and adopts flexible and diverse pop-up window shielding and other processing methods according to the level, so as to comprehensively, accurately and effectively monitor network information security, make up for the shortcomings of traditional technologies, and provide strong guarantees for the health and stability of the network environment. Summary of the invention

[0006] Technical problems to be solved: Only a single type of information can be detected, and it is difficult to accurately identify content with ambiguous semantics or implicit prohibited meanings.

[0007] In view of the deficiencies in the prior art, the present invention provides a network information security monitoring method and system based on artificial intelligence, thereby solving the technical problems mentioned in the background technology.

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] A network information security monitoring method based on artificial intelligence comprises the following steps:

[0010] Step 1: Interface information collection: monitor the user interface in real time and capture all information on the interface, including the desktop, application windows, and document content. Use screen capture tools to capture snapshots of the user interface at a high frequency, extract video streams from the user interface and separate audio streams, and use optical character recognition technology to extract text information from images.

[0011] Step 2: Risk analysis: Conduct risk analysis on the collected information, including image information analysis and text and audio information analysis. Image information analysis uses image recognition technology to identify prohibited content in images through convolutional neural networks. Text and audio information analysis uses trained machine learning models to identify prohibited words and phrases through latent semantic analysis technology.

[0012] Step 3: Classification of prohibited information: Classify the prohibited information analyzed into levels, including level one and level two. Level one requires local feature recognition, partial marking and processing, while level two requires overall feature judgment, overall marking and processing. Context analysis and multimodal analysis are combined in the analysis process. Context analysis considers the contextual semantic relationship among text, image and audio information, while multimodal analysis integrates text, image and sound information for processing and collaborative analysis.

[0013] Step 4, pop-up window shielding: When prohibited information is detected, corresponding pop-up window shielding processing is performed according to the level of prohibited information. For level one prohibited text, local mosaic pop-up window technology is used, for level two prohibited text, overall mosaic pop-up window technology is used, for level one prohibited images, dynamic mosaic pop-up window technology is used, for level two prohibited images, full-screen mosaic pop-up window technology is used, for level one prohibited sounds, local silencing pop-up window technology is used, for level two prohibited sounds, overall silencing pop-up window technology is used.

[0014] In one possible implementation, the screen capture tool can handle user interfaces of different resolutions and screen sizes to ensure comprehensiveness and accuracy of information collection, the audio capture technology can accurately separate the audio stream from the video content containing audio, and the optical character recognition technology can recognize text in different fonts, sizes, and colors and support multiple languages.

[0015] In a possible implementation, the convolutional neural network includes a convolutional layer, a pooling layer and a fully connected layer, the convolutional layer f(x) = σ(W*x+b), the pooling layer The fully connected layer y=W*h+b, the loss function is The latent semantic analysis technology extracts the theme or concept of the text through singular value decomposition, and its algorithm formula is A=URVT, which is used for prohibited word recognition, context analysis and prohibited content classification.

[0016] In a possible implementation, the local mosaic pop-up window is positioned and masked and followed by a target positioning and tracking algorithm. The target positioning and tracking algorithm formula is as follows:

[0017] Feature extraction formula: O = R (S / K), where S represents image data, K represents text data, R is the feature extraction function, and O is the extracted feature vector;

[0018] Target matching formula: G = P (D, L), where L is the predefined prohibited target feature template, P is the matching function, and G is the matching score, which indicates the similarity between the current content and the prohibited target;

[0019] Position determination formula: l = d(m, i / t), where i / t is the size information of the image or text, d is a function for determining the position of the prohibited object based on the matching score and size, and l is the position information of the prohibited object in the image or text;

[0020] Tracking update formula: n = u (l, m, e), where m is the target motion model, e is the data of the new image frame or text update part, u is the function that updates the prohibited target position based on the motion model and new data, and n is the updated prohibited target position.

[0021] In a possible implementation, an artificial intelligence-based network information security monitoring system executed in the artificial intelligence-based network information security monitoring method described above includes the following modules:

[0022] Information acquisition module: used to monitor the user interface in real time and capture all information on the interface, including a screen capture unit, an audio extraction unit, and a text recognition unit. The screen capture unit is responsible for capturing snapshots of the user interface at a high frequency, the audio extraction unit is responsible for extracting the video stream from the user interface and separating the audio stream, and the text recognition unit is responsible for extracting text information from the image;

[0023] Risk analysis module: used to perform risk analysis on the collected information, including an image recognition unit and a text and audio analysis unit. The image recognition unit uses a convolutional neural network to establish a deep learning model to identify prohibited content in images, and the text and audio analysis unit uses a trained machine learning model based on latent semantic analysis to identify prohibited words and phrases;

[0024] Prohibited information level classification module: used to classify the analyzed prohibited information, including a prohibited information analysis unit and a prohibited level classification unit. The prohibited information analysis unit performs context analysis and multimodal analysis, and the prohibited level classification unit determines the prohibited level according to the analysis result and performs corresponding processing;

[0025] Pop-up window shielding module: used to perform corresponding pop-up window shielding processing according to the level of prohibited information after prohibited information is detected, including text content shielding unit, image content shielding unit and sound content shielding unit. The text content shielding unit adopts corresponding mosaic pop-up window technology for different levels of prohibited texts, the image content shielding unit adopts corresponding mosaic pop-up window technology for different levels of prohibited images, and the sound content shielding unit adopts corresponding silencing pop-up window technology for different levels of prohibited sounds.

[0026] In a possible implementation, the screen capture unit can adapt to user interfaces of different resolutions and screen sizes, the audio extraction unit can accurately separate the audio stream, and the text recognition unit can recognize multiple types of text and support multiple languages.

[0027] Beneficial effects compared with the prior art:

[0028] 1. In this solution, by integrating screen capture, audio extraction and text recognition technology, we can achieve the synchronous collection of multiple types of information and fully cover the user interface content. Compared with existing technologies, it can more efficiently process information of different resolutions, multiple languages ​​and complex formats, ensure the integrity of information collection, provide a rich data foundation for accurate analysis, and effectively avoid monitoring loopholes caused by missing information.

[0029] 2. In this solution, by using advanced deep learning and machine learning models, such as convolutional neural networks and latent semantic analysis, we can deeply mine prohibited features in images, texts and audios, accurately identify potential risks, and use unique contextual analysis and multimodal collaborative analysis, combined with a sophisticated classification of prohibited levels, to deal with prohibited content of varying degrees in a targeted manner. Compared with traditional methods, this greatly improves recognition accuracy, reduces misjudgments, and better adapts to complex and changing prohibited forms.

[0030] 3. In this solution, the corresponding pop-up shielding mechanism is automatically triggered according to the prohibited level. According to the different prohibited situations of text, images and sounds, partial or overall mosaic and silencing technology are used respectively to effectively block the spread of prohibited content without affecting the normal information display. The existing technology often lacks such a flexible and targeted processing method. This method and system can better balance information security and user experience, and ensure that users are not disturbed by prohibited information. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings.

[0032] Figure 1 It is a flowchart of the steps of a network information security monitoring method based on artificial intelligence;

[0033] Figure 2 This is a system framework diagram of a network information security monitoring system based on artificial intelligence. DETAILED DESCRIPTION

[0034] Preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention can also be implemented in various forms, so the present invention is not limited to the embodiments described below.

[0035] The technical solution in the embodiment of the present application is to solve the problems of the above-mentioned background technology, and the overall idea is as follows:

[0036] Embodiment 1:

[0037] Please refer to Figure 1 As shown, this embodiment introduces a network information security monitoring method based on artificial intelligence, including the following steps:

[0038] Step 1: Interface information collection: real-time monitoring of the user interface and capturing all information on the interface, including the desktop, application windows, and document content;

[0039] Screen capture technology: Develop or integrate screen capture tools that can capture snapshots of the user interface at a high frequency (e.g., once per second). These snapshots include the web pages that users are browsing, the documents that they are editing, the application interfaces that they are running, etc. The screen capture tools need to be able to handle different resolutions and screen sizes to ensure the comprehensiveness and accuracy of the information;

[0040] Audio capture technology: extract the video stream from the user interface, which may include a movie being played, the interface of a video conferencing software, an online course, or any other video content containing audio, separate the audio stream from the extracted video stream, and perform content extraction on the extracted audio stream through speech recognition;

[0041] Text capture technology: For text information on the interface, optical character recognition technology is used to extract text from images, text in different fonts, sizes and colors, and can recognize multiple languages.

[0042] Step 2: Risk Analysis,

[0043] 2.1: Image information analysis, using image recognition technology and building deep learning models to identify prohibited content in images. These models can identify prohibited items, scenes or people in images;

[0044] Specifically, we use convolutional neural networks (CNN) to identify prohibited content in images. CNN learns features in images to identify objects, scenes, or people. The following is a basic architecture formula and meaning of CNN:

[0045] Convolutional Layer:

[0046] f(x) = σ(W*x+b);

[0047] Principle: The input image x is convolved with the convolution kernel (filter) W to extract the local features of the image. The convolution kernel slides on the image, calculates the weighted sum of the corresponding positions, and adds the bias term b. Then, the activation function σ is used to introduce nonlinear factors to enhance the expression ability of the model. Different convolution kernels can learn different image features, such as edges, textures, etc.

[0048] Example: When identifying whether an image contains prohibited items (such as guns), the convolution kernel can learn features such as the shape and outline of the gun. For example, a specific convolution kernel may be sensitive to the line features of the barrel of a gun. By sliding the convolution kernel on the image, it can detect areas in the image that are similar to the lines of the gun barrel, thereby helping to determine whether there is a gun in the image.

[0049] Pooling layer:

[0050]

[0051] Principle: The pooling layer is mainly used to reduce the dimension of the feature map, reduce the amount of calculation, and extract important features. In the maximum pooling, the input feature map x is divided into several pooling windows (size is n), and the maximum value in each window is selected as the output. This can retain the most significant feature information in the image, have a certain invariance to the translation and rotation of the image, and improve the generalization ability of the model;

[0052] Example: Assume that in a feature map of an image, a certain area contains some features of a gun, but also contains some noise information. Through the maximum pooling operation, the most significant feature value (such as a larger value related to the gun feature) is selected as the output in the pooling window of this area, which suppresses the noise information and highlights the key features of the gun, so that the subsequent network layers can focus more on the important features for learning and judgment;

[0053] Fully connected layer:

[0054] y=W*h+b;

[0055] Principle: The fully connected layer flattens the feature map after convolution and pooling, multiplies it with the weight matrix W, and adds the bias term b to obtain the final output y. The function of the fully connected layer is to integrate and classify the extracted features and map them to a specific category label space;

[0056] Example: In a gun recognition task, after the previous convolutional layer and pooling layer extract the features related to guns in the image, the fully connected layer combines these features and determines whether the image belongs to the "gun" category or the "non-gun" category based on the weights and biases learned during training. For example, if the feature is multiplied by the weight corresponding to the gun category to obtain a higher score, and it exceeds the set threshold, the image is judged to be a prohibited image containing guns;

[0057] Loss function:

[0058]

[0059] Principle: The loss function is used to measure the difference between the model prediction result and the true label. In the image classification task, y i is the true label (e.g., whether the image actually contains a gun), It is the probability that the model predicts a certain category. By minimizing the loss function and using optimization algorithms such as gradient descent to adjust the parameters (convolution kernel, weight, bias, etc.) in the convolutional neural network, the model's prediction results can be made more accurate.

[0060] Example: When training a gun recognition model, if an image containing a gun is mistakenly predicted by the model as not containing a gun, the loss value calculated by the loss function will be large. Through the back-propagation algorithm, the gradient of the parameters of each layer is calculated according to the loss value, and then the parameters are adjusted so that the model can gradually improve the recognition accuracy of gun images and reduce the loss value in subsequent training. For example, through multiple iterative training, the model can more accurately identify the gun features in the image, thereby more accurately determining whether the image is an image of a prohibited gun.

[0061] 2.2: Text and audio information analysis: using machine learning models to identify prohibited words and phrases, such as sentiment analysis, entity recognition, semantic analysis, etc., to conduct in-depth analysis of text information collected from audio and text. These technologies can help identify potential prohibited content, such as pornography, violence, discrimination, etc.

[0062] The above training machine learning model can be processed by latent semantic analysis (LSA). The specific algorithm formula of LSA is as follows:

[0063] A = URVT;

[0064] Principle: LSA uses singular value decomposition (SVD) to extract the topic or concept of the text. A is the document-term matrix, U and V are the left singular vector and the right singular vector respectively, and R is the diagonal matrix of singular values, which together represent the potential relationship between documents and terms.

[0065] Specifically for the identification of prohibited content, LSA can be applied to text information analysis in the following ways:

[0066] Prohibited Word Identification: By constructing a document-word matrix and applying SVD, LSA is able to identify words related to prohibited content, label them, and classify them in the latent semantic space;

[0067] Contextual analysis: LSA can understand the contextual meaning of banned words by analyzing word co-occurrence patterns, thereby more accurately identifying banned content;

[0068] Classification of prohibited content: Using the latent semantic features extracted by LSA, documents containing prohibited content can be classified for further processing and monitoring.

[0069] Example:

[0070] Suppose we have a simple corpus containing three sentences: 1. I like apples, 2. I love apples and oranges, 3. I eat oranges; First, we build a word-document matrix using binary frequency (i.e., 1 if the word appears in the document and 0 if it does not appear);

[0071] document I like apple like and orange eat 1 1 1 1 0 0 0 0 2 1 0 1 1 1 1 0 3 1 0 0 0 0 1 1

[0072] Next, we perform SVD decomposition on this matrix. For simplicity, we assume that two hidden topics are identified in some way (maybe manually or algorithmically): Topic A is related to "preferences" and Topic B is related to "food". We set the weights so that "like" and "love" have higher weights with Topic A, and "apple" and "orange" have higher weights with Topic B. In this way, we can get a simplified representation of the latent semantic space;

[0073] In this way, LSA helps us extract hidden semantic structures from text, enabling more accurate text comparison and information retrieval. For example, in prohibited content identification, LSA can identify words and documents related to prohibited topics, even if these words do not directly violate regulations literally, but through contextual analysis, their potential association with prohibited content is identified.

[0074] Step 3: Classification of prohibited information

[0075] 3.1: Analysis of Prohibited Information

[0076] 3.1.1: Contextual Analysis

[0077] When analyzing text information, in addition to considering whether the word itself is prohibited, the semantic relationship between the context should also be considered. For example, the word "violence" may be just an ordinary word when viewed alone, but in a sentence such as "committing violent acts", it can be determined as prohibited content when combined with the context. For image information, attention should also be paid to the scene in which the image is located and its relationship with other elements. For example, if a picture depicting a war scene contains some specific, symbolic and non-compliant signs or behaviors, it may constitute prohibited content. In terms of audio, contextual factors such as the tone of voice, speed of speech, and coordination with other sound elements will also affect the determination of prohibited content. For example, a voice expression with a threatening tone and containing prohibited words needs to be paid special attention to.

[0078] Through the use of technologies such as Latent Semantic Analysis (LSA), we can deeply explore the potential semantic relationship of text information, better understand the meaning of words in different contexts, and thus more accurately judge whether it is prohibited. For images, we use the features learned by models such as Convolutional Neural Network (CNN) and the relationship between different image elements to comprehensively judge whether the image as a whole or in part constitutes prohibited content. In audio analysis, after converting the audio into text with the help of speech recognition technology, we use relevant text analysis technology combined with other features of the speech to conduct a comprehensive analysis of prohibited content;

[0079] 3.1.2: Multimodal Analysis

[0080] Data fusion: effectively integrate the text, image and sound information collected from the interface. For example, for a multimedia file containing a video, first use screen capture technology to obtain the video screen (image information), extract the audio stream (sound information) in the video, and then use optical character recognition technology (OCR) to extract text information (text information) from the text part that may appear in the video screen. Then convert and align the information of these three modalities into a unified data format for subsequent comprehensive analysis;

[0081] Collaborative analysis: Establish a multimodal analysis model so that text, image and sound information can complement and verify each other. For example, when judging whether an online video course contains prohibited content, image recognition technology can detect whether there are prohibited items or scenes in the video. The audio content extracted by speech recognition technology can be analyzed to determine whether there are prohibited words or inappropriate expressions. At the same time, text information such as text prompts appearing in the video can be analyzed. The results of these three aspects can be combined to more accurately determine whether there is prohibited content and the degree of prohibition.

[0082] 3.2: Classification of prohibited items

[0083] 3.2.1: Level 1 Prohibition

[0084] Local feature recognition: For text content, it is necessary not only to identify specific banned words, but also to determine the specific locations of these words in sentences or paragraphs, so as to carry out accurate local mosaic processing later. For image content, it is necessary to accurately identify the banned parts in the image, such as accurately finding the hidden banned signs or scenes in a complex landscape picture, and clarifying their boundaries and location information, so as to provide an accurate basis for the application of dynamic mosaic technology. For sound content, it is necessary to accurately identify the starting and ending positions of banned sentences, as well as the locations of key banned words in the sentences, so as to carry out effective local silencing processing;

[0085] Partial marking and processing: After identifying the first-level prohibited content, the system will clearly mark the prohibited local part. For example, in a document, the location of the prohibited words is marked with special colors or symbols; on an image, the location of the prohibited part is highlighted in some way (such as a flashing border, etc.); on the sound playback interface, the time interval of the prohibited sentence is displayed, etc. Then, according to different content types, the corresponding local processing technology is applied, such as local mosaic technology for text, dynamic mosaic technology for images, local silencing technology for sound, etc., to effectively block the prohibited part while minimizing the impact on other normal content;

[0086] 3.2.2: Second level prohibited

[0087] Overall feature judgment: For text content, it is necessary to be able to judge whether the overall content is prohibited through comprehensive analysis of the entire document or text paragraph. This may involve analysis of the theme, emotional tendency, semantic coherence, etc. of the text. For image content, it is necessary to be able to judge whether an image is completely non-compliant with the regulations, such as a picture full of prohibited scenes or objects. For audio content, it is necessary to be able to judge whether the entire audio clip is full of prohibited expressions or bad information;

[0088] Overall marking and processing: Once it is determined to be Level 2 prohibited content, the system will uniformly mark the entire document, image or sound, such as marking the words "Level 2 prohibited" at the beginning of the document, covering the image with a specific identification layer to indicate its prohibited status, and giving clear prompt information on the sound playback interface. Then, the overall mosaic technology is used to block the text and image, and the overall silencing technology is used to eliminate the sound, so as to completely block the prohibited content.

[0089] Step 4: Pop-up window blocking. When prohibited information is detected, the corresponding pop-up window blocking process is performed according to the prohibited information level to effectively prevent the presentation and dissemination of prohibited content to users;

[0090] 4.1: Text content blocking

[0091] 4.1.1: Local mosaic pop-up technology (for first-level prohibited text)

[0092] Once the first-level prohibited text content is identified, the system quickly activates the local mosaic pop-up mechanism; first, determine the specific location of the prohibited words in the text, including detailed information such as the row, column, and character interval;

[0093] Next, a pop-up window adapted to the text display area is generated. The pop-up window presents a mosaic effect at the location of the banned words to block the banned words. When designing the pop-up window, its size and shape must be matched with the banned word area, and the visual attributes of the pop-up window, such as color and transparency, are carefully adjusted to effectively block the banned words without causing too much interference to the reading of the surrounding normal text, ensuring that users can clearly read other non-banned text content. For example, if the banned word "violence" is detected in an article, the system will pop up a rectangular mosaic pop-up window at the location of the word that just covers the word "violence". The background color of the pop-up window can be set to a light gray color similar to the text background, and the transparency is set to about 70%. This can not only block the banned words, but also vaguely show the outline of the text below, so that the overall coherence and readability of the article can be maintained.

[0094] 4.1.2: Overall mosaic pop-up technology (for secondary prohibited text)

[0095] For text content that is judged as Level 2 prohibited, such as when the entire document or the entire interface text contains prohibited information, the system will enable a mosaic pop-up window for the entire document;

[0096] The pop-up window completely covers all text content with a mosaic pattern. The pop-up window style can adopt evenly distributed small square mosaics or blurred mosaic effects, so that the text content cannot be recognized by users. At the same time, the pop-up window can display corresponding prompt information, such as "This content contains prohibited information and has been blocked", to inform users of the blocking reason. For example, if all the text on a web page interface is prohibited promotional content, the system will pop up a full-screen mosaic pop-up window to cover the entire web page text area. Users can only see the mosaic interface with prompt information and cannot obtain any prohibited text information.

[0097] 4.2: Image content shielding

[0098] 4.2.1: Dynamic mosaic pop-up technology (for first-level prohibited images)

[0099] When the system detects first-level prohibited image content, that is, only part of the image contains prohibited elements, the dynamic mosaic pop-up technology starts to work, accurately identifying and tracking the position, shape and size changes of the prohibited image part on the screen in real time;

[0100] As the prohibited image moves or changes, the system dynamically generates a matching mosaic pop-up window. The shape, size and position of the pop-up window are always consistent with the prohibited image part, and it blocks it with a mosaic effect. For example, during the playback of a video, if a prohibited item (such as a prohibited weapon) appears in the picture, the system will quickly generate a dynamic mosaic pop-up window in the area where the weapon is located. No matter how the video screen switches or how the camera moves, the pop-up window can accurately follow the position changes of the prohibited weapon and continue to block it, ensuring that users cannot see the details of the prohibited image, while other normal image parts of the video can still be played normally without affecting the overall viewing experience;

[0101] 4.2.2: Full-screen mosaic pop-up technology (for secondary prohibited images)

[0102] For level 2 prohibited images, i.e., when the entire image is prohibited content, the full-screen mosaic pop-up window technology is triggered to generate a full-screen mosaic pop-up window covering the entire image display area;

[0103] The pop-up window may adopt a uniform mosaic style, such as dense pixelated mosaic or blurred color mosaic effect, to completely cover the entire image and prevent users from seeing any prohibited image information; at the same time, a prompt message such as "This image contains prohibited content and has been blocked" may be displayed on the pop-up window. For example, if a picture is full of prohibited bloody and violent scenes, the system will pop up a full-screen mosaic pop-up window, and users can only see the mosaic-covered screen and prompt information, and cannot access the prohibited image content;

[0104] As mentioned above, for the first-level prohibited text and images, positioning masking and following masking are performed through target positioning and tracking algorithms, which are as follows:

[0105] (1) Algorithm formula

[0106] Feature extraction formula: O = R (S / K), where S represents image data, K represents text data, R is the feature extraction function, and O is the extracted feature vector; for images, the color, texture, shape and other features of the image are extracted through convolutional neural networks and other technologies; for text, the word vectors, grammatical structures and other features of the vocabulary are extracted, for example, the shape features of prohibited knives are extracted from images, and the semantic features of prohibited words are extracted from texts;

[0107] Target matching formula: G = P (D, L), where L is the predefined prohibited target feature template (for text, it is the feature of prohibited words, for images, it is the feature of prohibited items or scenes), P is the matching function, and G is the matching score, which indicates the similarity between the current content and the prohibited target; the extracted features are matched with the predefined prohibited target feature template, and the similarity is calculated. The higher the score, the more likely it is a prohibited target. For example, the shape features extracted from the image are compared and matched with the shape template features of prohibited knives, and the vocabulary features in the text are matched with the prohibited vocabulary template features;

[0108] Position determination formula: l = d(m, i / t), where i / t is the size information of the image or text, d is a function for determining the position of the prohibited object based on the matching score and the size, and l is the position information of the prohibited object in the image or text (for an image, it may be a coordinate range, and for a text, it may be a character interval); based on the matching score and the size of the image or text, the specific position of the prohibited object is determined in the image, the coordinate range of the prohibited item is determined in the image, and the character interval where the prohibited words are located is determined in the text;

[0109] Tracking update formula: n = u (l, m, e), where m is the target motion model (for targets moving in the image, such as prohibited items in the video; for text, it may be a model of operations such as scrolling), e is the data of the new image frame or the updated part of the text, u is a function that updates the position of the prohibited target based on the motion model and the new data, and n is the updated position of the prohibited target; when the image is a video or the text has operations such as scrolling, the position of the prohibited target is updated according to the target motion model and new data (new video frame or new text part), for example, in the video, the position of the knife in the picture is updated according to the motion trend (motion model) of the prohibited item and the new video frame content; in the scrolling text, the position of the prohibited words is updated according to the scrolling operation model and the newly displayed text content;

[0110] 4.3: Audio content blocking

[0111] 4.3.1: Local silencing pop-up window technology (for first-level prohibited sounds)

[0112] When first-level prohibited audio content is detected, that is, only part of the sentence or fragment in the audio is prohibited information, the local silencing pop-up window technology comes into play. Through the voice endpoint detection technology, the starting point and end point of the prohibited audio sentence and its position information in the entire audio stream are accurately identified;

[0113] Then, an audio muting pop-up window is generated. During the audio playback process, the pop-up window mutes the time period where the prohibited sound sentences are located, so that this part of the sound cannot be played, while other normal sound parts are not affected and continue to play normally. For example, in an audio file containing multiple sound elements (such as background music, dialogues and narrations), if one of the narrations contains prohibited words, the system will start a local muting pop-up window when the narration is played, muting the prohibited narration, while the background music and other normal dialogues can still be heard by the user normally, maintaining the overall coherence and audibility of the audio;

[0114] 4.3.2: Overall silencing pop-up window technology (for level 2 prohibited sounds)

[0115] For audio content that is judged as Level 2 prohibited, that is, when the entire audio clip is prohibited information, the overall mute pop-up window technology is activated. The system analyzes the overall characteristics of the audio and identifies the prohibited content. After determining the overall prohibited nature of the audio, a mute pop-up window covering the entire audio playback area is generated;

[0116] When the audio is playing, this pop-up window completely blocks the output of sound, making the entire audio silent and the user unable to hear any audio content. At the same time, the pop-up window may display a prompt message such as "This audio contains prohibited content and has been blocked." For example, if an audio file is full of prohibited violent threatening voices, the system will pop up a whole mute pop-up window, and the user will not hear any audio sound but can only see the prompt message, effectively preventing the spread of prohibited sounds.

[0117] Embodiment 2:

[0118] Please refer to Figure 2 As shown, this embodiment introduces a network information security monitoring system based on artificial intelligence, which is used to implement the network information security monitoring method in embodiment 1. The system includes the following modules:

[0119] 1. Information collection module

[0120] 1.1 Screen Capture Unit

[0121] Responsible for real-time monitoring of the user interface, capturing snapshots of the user interface at a high frequency (once per second), ensuring that different types of image information such as the web pages being browsed, the documents being edited, and the application interfaces being run can be obtained, and can adapt to different resolutions and screen sizes to ensure the comprehensiveness and accuracy of the information;

[0122] 1.2 Audio Extraction Unit

[0123] Extract video streams from user interfaces, such as movies being played, video conferencing software interfaces, online courses, and other video content containing audio, and then separate the audio stream from the extracted video stream for subsequent analysis and processing of the audio information;

[0124] 1.3 Text Recognition Unit

[0125] Use optical character recognition technology (OCR) to extract text information from images. It can recognize texts of different fonts, sizes and colors, and supports multiple languages. This unit mainly collects text information on the interface to provide data sources for subsequent text information analysis.

[0126] 2. Risk Analysis Module

[0127] 2.1 Image Recognition Unit

[0128] Use convolutional neural networks (CNNs) to build deep learning models to identify prohibited content such as prohibited items, scenes, or people by learning features in images;

[0129] Responsible for inputting the collected images into the constructed CNN model, analyzing the images through the features learned by the model and the associations between different image elements, and determining whether the image as a whole or in part constitutes prohibited content;

[0130] 2.2 Text and Audio Analysis Unit

[0131] Building and training machine learning models based on latent semantic analysis (LSA). LSA uses singular value decomposition (SVD) to extract themes or concepts of text. This model is used to identify prohibited words and phrases in text information, and to perform sentiment analysis, entity recognition, semantic analysis, and other in-depth processing on text information collected in audio and text, helping to identify potential prohibited content, such as pornography, violence, discrimination, etc.

[0132] Use the built and trained machine learning model to analyze text and audio information, including banned word identification (by building a document and word matrix and applying SVD, identifying words related to banned content, and marking and classifying them in the latent semantic space), context analysis (by analyzing word co-occurrence patterns, understanding the contextual meaning of banned words, and thus more accurately identifying banned content), and banned content classification (using the latent semantic features extracted by LSA to classify documents containing banned content for further processing and monitoring);

[0133] 3. Prohibited information classification module

[0134] 3.1. Prohibited Information Analysis Unit

[0135] When analyzing text information, we use latent semantic analysis (LSA) and other technologies to deeply explore the potential semantic relationship of text information, understand the meaning of words in different contexts, and accurately determine whether it is prohibited. For image information, we use the features learned by models such as convolutional neural networks (CNN) and the associations between image elements to make a comprehensive judgment. In audio analysis, after converting the audio into text with the help of speech recognition technology, we use related text analysis technology combined with other features of the speech to conduct a comprehensive analysis of prohibited content, while paying attention to the impact of contextual factors such as the tone of voice, speech speed, and coordination with other sound elements on the determination of prohibited content.

[0136] Data fusion part: effectively integrate the text, image and sound information collected from the interface. For example, for multimedia files, first obtain the video screen (image information), extract the audio stream (sound information), and then extract the text information (text information) from the video screen, and then convert and align the information of these three modes into a unified data format for subsequent comprehensive analysis.

[0137] Collaborative analysis: Establish a multimodal analysis model so that text, image and sound information can complement and verify each other. For example, when judging whether an online video course contains prohibited content, the analysis results of the video screen using image recognition technology, the audio content using voice recognition technology, and the text information such as text prompts in the video screen can be used to more accurately determine whether there is prohibited content and the extent of the prohibition.

[0138] 3.2. Prohibition Level Division Unit

[0139] First level prohibited treatment:

[0140] For text content, it can determine the specific location of banned words in a sentence or paragraph; for image content, it can accurately identify the banned parts in the image and their boundaries and location information; for sound content, it can accurately identify the starting and ending positions of banned sentences and the locations of key banned words.

[0141] When the first-level prohibited content is identified, the prohibited parts are clearly marked. For example, the prohibited words are marked with special colors or symbols in the document, the prohibited parts are highlighted with flashing borders on the image, and the time interval of the prohibited sentences is displayed on the sound playback interface. Then, according to different content types, corresponding local processing technologies are applied, such as local mosaic technology for text, dynamic mosaic technology for images, local silencing technology for sound, etc., to block the prohibited parts without affecting other normal content.

[0142] Second level prohibited treatment:

[0143] For text content, a comprehensive analysis of the entire document or text paragraph is conducted to determine whether the overall content is prohibited, involving analysis of the text's theme, emotional tendency, semantic coherence, and other aspects; for image content, the overall assessment is to determine whether the image is completely non-compliant with regulations; in terms of sound content, the assessment is to determine whether the entire sound clip is filled with prohibited expressions or bad information.

[0144] Once it is determined to be Level 2 prohibited content, the entire document, image or sound will be uniformly marked. For example, the words "Level 2 prohibited" will be marked at the beginning of the document, a specific identification layer will be covered on the image, and clear prompt information will be given on the sound playback interface. Then, the overall mosaic technology will be used to block the text and image, and the overall silencing technology will be used to eliminate the sound, completely shielding the prohibited content.

[0145] 4. Pop-up window shielding module

[0146] 4.1 Text content shielding unit

[0147] When first-level prohibited text content is identified, the position of the prohibited words in the text is determined, and a pop-up window adapted to the text display area is generated. A mosaic effect is presented at the position of the prohibited words to block them. The size and shape of the pop-up window match the prohibited word area, and the color, transparency and other visual attributes are adjusted to cover the prohibited words without affecting the normal reading of the surrounding text.

[0148] For the second-level prohibited text content, after scanning and judging the entire text area, a large pop-up window covering the entire text display area is generated, and all text content is completely obscured by a mosaic pattern. A uniformly distributed small square mosaic or a blurred mosaic effect can be used, and a prompt message is displayed on the pop-up window to inform the user of the reason for blocking.

[0149] 4.2 Image Content Shielding Unit

[0150] When first-level prohibited image content is detected, the position, shape and size changes of the prohibited image part on the screen are identified and tracked in real time, and a matching mosaic pop-up window is dynamically generated to cover it, ensuring that other normal image parts of the video are played normally.

[0151] For second-level prohibited images, after determining the prohibited nature through overall feature analysis of the image, a full-screen mosaic pop-up window covering the entire image display area is generated, using a unified mosaic style (such as dense pixelated mosaics or color-blurred mosaic effects) to cover the image and display a prompt message.

[0152] 4.3. Audio content shielding unit

[0153] When first-level prohibited sound content is detected, voice endpoint detection technology is used to identify the starting point, end point and location information of the prohibited sound sentence, generate an audio mute pop-up window, and mute the sound during the time period where the prohibited sound sentence is located to ensure that other normal sounds are played normally.

[0154] For the second-level prohibited sound content, after determining the overall prohibition through overall feature analysis of the audio, a mute pop-up window covering the entire audio playback area is generated, completely blocking the sound output, making the audio silent, and displaying a prompt message.

[0155] The specific implementation is as follows:

[0156] 1. Text content blocking

[0157] Level 1 prohibited text: When the system detects level 1 prohibited text, for example, the prohibited word "drug trade" is found in a chat window of an instant messaging software, the system immediately activates the local mosaic pop-up mechanism, determines the position of the four words "drug trade" in the chat window through the text positioning algorithm, and then generates a rectangular pop-up window that just covers the four words. The background color of the pop-up window is set to a gray color that is similar to the background of the chat window but slightly darker, and the transparency is adjusted to about 60%, which can effectively block the prohibited words and allow users to vaguely see the outline of the text below without affecting the reading of other chat content. The pop-up window pops up instantly when the prohibited words appear and continues to be displayed until the chat history is scrolled or the window is closed;

[0158] Level 2 prohibited text: If the text content of the entire news page of a website is judged as level 2 prohibited (such as propaganda of reactionary ideas), the system triggers a mosaic pop-up window. The system quickly scans and determines the text area of ​​the entire news page. After confirming that it is prohibited, it generates a full-screen pop-up window covering the entire text display area. The pop-up window uses a mosaic effect of evenly distributed small squares in gray, making the text completely unrecognizable. At the same time, a striking prompt message "This page contains prohibited information and has been blocked" is displayed in the center of the pop-up window. Users can only see this prompt message and the page covered by the mosaic, and cannot obtain any prohibited text content;

[0159] 2. Image content blocking

[0160] Level 1 prohibited images: When playing a surveillance video, if a person holding a prohibited knife (level 1 prohibited image) suddenly appears in the picture, the system uses a combination of deep learning-based target detection algorithm and Kalman filter algorithm to identify and track the position, shape and size changes of the knife in the picture in real time. As the knife moves, the system dynamically generates a matching mosaic pop-up window. The shape and size of the pop-up window are always consistent with the knife area, and the color is black mosaic effect, which effectively blocks the knife part, while other normal scenes of the video (such as background environment, other people, etc.) continue to play normally, without affecting the viewing and analysis of the overall surveillance video;

[0161] Level 2 prohibited images: When a picture full of pornographic content (level 2 prohibited images) is detected, the system determines its prohibited nature through overall feature analysis of the image (such as color distribution, image element features, etc.) and prohibited content determination algorithm, and then generates a full-screen mosaic pop-up window covering the entire image display area. The pop-up window uses a color-blurred mosaic effect to make the image completely invisible. At the same time, the pop-up window displays a prompt message "This image contains prohibited content and has been blocked" to prevent users from being exposed to prohibited image content.

[0162] 3. Audio content blocking

[0163] Level 1 prohibited sound: When listening to an audio podcast, if one of the narrations contains the prohibited word "scam method" (level 1 prohibited sound), the system uses voice endpoint detection technology and audio feature extraction algorithms to accurately identify the start time (such as 2:10) and end time (such as 2:20) of the prohibited sentence. When the audio plays to 2:10, the system generates an audio mute pop-up window and mutes the audio during this time period so that the audience cannot hear the prohibited narration, while other normal audio content of the podcast (such as background music, other normal narration, etc.) continues to play normally to maintain the continuity and audibility of the audio;

[0164] Level 2 prohibited sound: If an audio file is entirely composed of voice content advocating violent crimes (level 2 prohibited sound), the system will determine its overall prohibited nature through overall feature analysis of the audio (such as voice intonation, speech speed, vocabulary frequency, etc.) and prohibited content recognition algorithm, and then generate a mute pop-up window covering the entire audio playback area. When you try to play the audio, the mute pop-up window will take effect, the audio will be completely silent, and the user will not be able to hear any audio content. At the same time, the prompt message "This audio contains prohibited content and has been blocked" will be displayed on the audio playback interface, effectively preventing the spread of prohibited sounds.

[0165] Finally, it should be noted that: Obviously, the above embodiments are only examples for clearly explaining the present invention, and are not intended to limit the implementation methods. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from this are still within the scope of protection of the present invention.

Claims

1. A network information security monitoring method based on artificial intelligence, characterized in that: The following steps are involved: Step 1: Interface information collection: monitor the user interface in real time and capture all information on the interface, including the desktop, application windows, and document content. Use screen capture tools to capture snapshots of the user interface at a high frequency, extract video streams from the user interface and separate audio streams, and use optical character recognition technology to extract text information from images. Step 2: Risk analysis: Conduct risk analysis on the collected information, including image information analysis and text and audio information analysis. Image information analysis uses image recognition technology to identify prohibited content in images through convolutional neural networks. Text and audio information analysis uses trained machine learning models to identify prohibited words and phrases through latent semantic analysis technology. Step 3: Classification of prohibited information: Classify the prohibited information analyzed into levels, including level one and level two. Level one requires local feature recognition, partial marking and processing, while level two requires overall feature judgment, overall marking and processing. Context analysis and multimodal analysis are combined in the analysis process. Context analysis considers the contextual semantic relationship among text, image and audio information, while multimodal analysis integrates text, image and sound information for processing and collaborative analysis. Step 4, pop-up window shielding: When prohibited information is detected, corresponding pop-up window shielding processing is performed according to the level of prohibited information. For level one prohibited text, local mosaic pop-up window technology is used, for level two prohibited text, overall mosaic pop-up window technology is used, for level one prohibited images, dynamic mosaic pop-up window technology is used, for level two prohibited images, full-screen mosaic pop-up window technology is used, for level one prohibited sounds, local silencing pop-up window technology is used, for level two prohibited sounds, overall silencing pop-up window technology is used.

2. The network information security monitoring method based on artificial intelligence according to claim 1, characterized in that: The screen capture tool can handle user interfaces of different resolutions and screen sizes to ensure comprehensiveness and accuracy of information collection. The audio capture technology can accurately separate the audio stream from the video content containing audio. The optical character recognition technology can recognize texts of different fonts, sizes, colors and supports multiple languages.

3. The network information security monitoring method based on artificial intelligence as claimed in claim 1, characterized in that: The convolutional neural network includes a convolutional layer, a pooling layer and a fully connected layer, the convolutional layer f(x) = σ(W*x+b), the pooling layer The fully connected layer y=W*h+b, the loss function is The latent semantic analysis technology extracts the theme or concept of the text through singular value decomposition, and its algorithm formula is A=URVT, which is used for prohibited word recognition, context analysis and prohibited content classification.

4. The network information security monitoring method based on artificial intelligence as claimed in claim 1, characterized in that: The local mosaic pop-up window is positioned and masked and followed by a target positioning and tracking algorithm. The target positioning and tracking algorithm formula is as follows: Feature extraction formula: O = R (S / K), where S represents image data, K represents text data, R is the feature extraction function, and O is the extracted feature vector; Target matching formula: G = P (D, L), where L is the predefined prohibited target feature template, P is the matching function, and G is the matching score, which indicates the similarity between the current content and the prohibited target; Position determination formula: l = d(m, i / t), where i / t is the size information of the image or text, d is a function for determining the position of the prohibited object based on the matching score and size, and l is the position information of the prohibited object in the image or text; Tracking update formula: n = u (l, m, e), where m is the target motion model, e is the data of the new image frame or text update part, u is the function that updates the prohibited target position based on the motion model and new data, and n is the updated prohibited target position.

5. An artificial intelligence-based network information security monitoring system that implements the artificial intelligence-based network information security monitoring method described in any one of claims 1 to 4, characterized in that: The system includes the following modules: Information acquisition module: used to monitor the user interface in real time and capture all information on the interface, including a screen capture unit, an audio extraction unit, and a text recognition unit. The screen capture unit is responsible for capturing snapshots of the user interface at a high frequency, the audio extraction unit is responsible for extracting the video stream from the user interface and separating the audio stream, and the text recognition unit is responsible for extracting text information from the image; Risk analysis module: used to perform risk analysis on the collected information, including an image recognition unit and a text and audio analysis unit. The image recognition unit uses a convolutional neural network to establish a deep learning model to identify prohibited content in images, and the text and audio analysis unit uses a trained machine learning model based on latent semantic analysis to identify prohibited words and phrases; Prohibited information level classification module: used to classify the analyzed prohibited information, including a prohibited information analysis unit and a prohibited level classification unit. The prohibited information analysis unit performs context analysis and multimodal analysis, and the prohibited level classification unit determines the prohibited level according to the analysis result and performs corresponding processing; Pop-up window shielding module: used to perform corresponding pop-up window shielding processing according to the level of prohibited information after prohibited information is detected, including text content shielding unit, image content shielding unit and sound content shielding unit. The text content shielding unit adopts corresponding mosaic pop-up window technology for different levels of prohibited texts, the image content shielding unit adopts corresponding mosaic pop-up window technology for different levels of prohibited images, and the sound content shielding unit adopts corresponding silencing pop-up window technology for different levels of prohibited sounds.

6. The network information security monitoring system based on artificial intelligence as claimed in claim 5, characterized in that: The screen capture unit can adapt to user interfaces of different resolutions and screen sizes, the audio extraction unit can accurately separate the audio stream, and the text recognition unit can recognize multiple types of text and support multiple languages.

Images