An artificial intelligence-based network information security monitoring method and system

Abstract

The present application relates to the technical field of network information security, and particularly relates to a network information security monitoring method based on artificial intelligence, comprising the following steps: step 1, interface information collection step: real-time monitoring of the user interface and capturing all information on the interface, including the desktop, application program window and document content, capturing the snapshot of the user interface at a high frequency through a screen capture tool, extracting the video stream from the user interface and separating the audio stream, and extracting the text information from the image by using the optical character recognition technology; in the present scheme, the text, image and sound information of the user interface are collected in real time, advanced artificial intelligence algorithms such as convolutional neural network and latent semantic analysis are used for deep risk analysis, the level of prohibited information is accurately divided, and flexible and diverse pop-up shielding and other processing methods are used according to the level, so that the network information security is comprehensively, accurately and effectively monitored, the defects of traditional technologies are made up, and strong guarantee is provided for the healthy and stable network environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network information security, in particular to a network information security monitoring method and system based on artificial intelligence. BACKGROUND

[0002] With the rapid development of information technology, the network has been deeply integrated into people's life and work, such as online office, distance education, social entertainment and other activities are widely carried out on the network platform. At the same time, network information security problems are increasingly prominent, and various illegal information spreads freely in the network space, causing serious harm to individuals, enterprises and even society.

[0003] Traditional network information security monitoring methods have many limitations. On the one hand, they can only detect a single type of information, such as focusing only on keyword filtering of text content, while ignoring the illegal information that may be contained in images and audio. This allows illegal elements to spread illegal content through pictures, videos, and audio, evading detection by traditional monitoring methods. On the other hand, traditional techniques lack depth understanding when analyzing information, making it difficult to accurately identify content with ambiguous semantics or implied illegal meanings. For example, some text expressions with suggestive or metaphorical meanings, as well as images that hide illegal elements through image processing, are often not effectively identified by traditional monitoring methods.

[0004] In addition, existing monitoring techniques lack flexibility and precision when dealing with illegal information. Once possible illegal content is detected, simple and crude methods such as directly blocking entire web pages or accounts are often used, which not only may misjudge normal information and affect user experience, but also may not effectively prevent the spread of illegal information, as illegal elements can republish by changing the dissemination channel or modifying the information form. In this context, there is an urgent need for a network information security monitoring method and system that can comprehensively collect information, accurately analyze risks, and flexibly and effectively handle illegal content. Based on the development of artificial intelligence technology, the present application is born to fill the gaps in existing technology and provide more reliable protection for network information security.

[0005] Therefore, the present application proposes a network information security monitoring method and system based on artificial intelligence, which collects text, image and sound information of the user interface in real time, uses advanced artificial intelligence algorithms such as convolutional neural networks and latent semantic analysis for deep risk analysis, accurately classifies illegal information levels, and uses flexible and diverse pop-up shielding and other processing methods according to the level, to comprehensively, accurately and effectively monitor network information security, and to make up for the shortcomings of traditional technology, providing strong protection for the healthy and stable development of network environment. SUMMARY

[0006] The technical problem solved by the present application is that the existing technology can only detect a single type of information and cannot accurately identify content with ambiguous semantics or implied prohibited meanings.

[0007] To overcome the deficiencies of the prior art, the present application provides a network information security monitoring method and system based on artificial intelligence, thereby solving the technical problems mentioned in the background art.

[0008] To achieve the above object, the present application is implemented by the following technical solutions:

[0009] A network information security monitoring method based on artificial intelligence, comprising the following steps:

[0010] Step 1, interface information collection: real-time monitoring of the user interface and capturing all information on the interface, including the desktop, application window and document content, capturing snapshots of the user interface at a high frequency through a screen capture tool, extracting a video stream from the user interface and separating the audio stream, and extracting text information from the image using optical character recognition technology;

[0011] Step 2, risk analysis: risk analysis of the collected information, including image information analysis and text and audio information analysis, image information analysis using image recognition technology, identifying prohibited content in the image through a convolutional neural network, and text and audio information analysis using a trained machine learning model, identifying prohibited words and phrases through latent semantic analysis technology;

[0012] Step 3, prohibited information classification: classifying the analyzed prohibited information, including first-level prohibited and second-level prohibited, first-level prohibited requiring local feature recognition and partial marking and processing, and second-level prohibited requiring overall feature judgment and overall marking and processing, combining context analysis and multi-modal analysis during the analysis process, considering the context semantic relationship in the text, image and audio information, and fusing and cooperatively analyzing the text, image and sound information;

[0013] Step 4, pop-up window shielding: when prohibited information is detected, corresponding pop-up window shielding processing is performed according to the prohibited information level, local mosaic pop-up window technology is used for first-level prohibited text, overall mosaic pop-up window technology is used for second-level prohibited text, dynamic mosaic pop-up window technology is used for first-level prohibited images, full-screen mosaic pop-up window technology is used for second-level prohibited images, local sound-off pop-up window technology is used for first-level prohibited sound, and overall sound-off pop-up window technology is used for second-level prohibited sound.

[0014] In a possible implementation, the screen capture tool can process user interfaces of different resolutions and screen sizes to ensure the comprehensiveness and accuracy of information collection, the audio capture technology can accurately separate the audio stream from the video content containing the audio, and the optical character recognition technology can recognize text of different fonts, sizes, and colors and support multiple languages.

[0015] In a possible implementation, the convolutional neural network includes a convolutional layer, a pooling layer, and a fully connected layer, the convolutional layer f(x) = σ(W*x+b), the pooling layer The fully connected layer y = W*h+b, and the loss function is The latent semantic analysis technology extracts the theme or concept of the text through singular value decomposition, and the algorithm formula is A = URVT, which is used for forbidden word identification, context analysis, and forbidden content classification.

[0016] In a possible implementation, the local mosaic popup window is positioned and shielded and followed by a target positioning and tracking algorithm, and the formula of the target positioning and tracking algorithm is as follows:

[0017] The feature extraction formula is O = R(S / K), where S represents image data, K represents text data, R is a feature extraction function, and O is a feature vector extracted;

[0018] The target matching formula is G = P(D, L), where L is a predefined forbidden target feature template, P is a matching function, G is a matching score, and represents the similarity between the current content and the forbidden target;

[0019] The position determination formula is l = d(m, i / t), where i / t is the size information of the image or text, d is a function of determining the position of the forbidden target according to the matching score and the size, and l is the position information of the forbidden target in the image or text;

[0020] The tracking update formula is n = u(l, m, e), where m is a target motion model, e is the data of a new image frame or a text update part, u is a function of updating the position of the forbidden target according to the motion model and the new data, and n is the updated position of the forbidden target.

[0021] In a possible implementation, an artificial intelligence-based network information security monitoring system is executed on the above-mentioned artificial intelligence-based network information security monitoring method, and the system includes the following modules:

[0022] The information collection module is used for real-time monitoring of the user interface and capturing all information on the interface, including a screen capture unit, an audio extraction unit and a text recognition unit. The screen capture unit is responsible for capturing snapshots of the user interface at a high frequency. The audio extraction unit is responsible for extracting a video stream from the user interface and separating the audio stream. The text recognition unit is responsible for extracting text information from images.

[0023] The risk analysis module is used for risk analysis of the collected information, including an image recognition unit and a text / audio analysis unit. The image recognition unit uses a convolutional neural network to establish a deep learning model to identify prohibited content in images. The text / audio analysis unit uses a trained machine learning model based on latent semantic analysis to identify prohibited words and phrases.

[0024] The prohibited information level division module is used for level division of the analyzed prohibited information, including a prohibited information analysis unit and a prohibited level division unit. The prohibited information analysis unit performs context analysis and multi-modal analysis. The prohibited level division unit determines the prohibited level according to the analysis results and performs corresponding processing.

[0025] The pop-up shielding module is used for shielding pop-ups according to the level of prohibited information when prohibited information is detected, including a text content shielding unit, an image content shielding unit and a sound content shielding unit. The text content shielding unit uses corresponding mosaic pop-up technology for different levels of prohibited text. The image content shielding unit uses corresponding mosaic pop-up technology for different levels of prohibited images. The sound content shielding unit uses corresponding sound reduction pop-up technology for different levels of prohibited sounds.

[0026] In one possible implementation, the screen capture unit can adapt to user interfaces of different resolutions and screen sizes, the audio extraction unit can accurately separate the audio stream, and the text recognition unit can recognize multiple types of text and support multiple languages.

[0027] Compared with the prior art, the beneficial effects are as follows:

[0028] 1. In this scheme, by integrating screen capture, audio extraction and text recognition technologies, synchronous collection of multiple types of information is realized, and the content of the user interface is fully covered. Compared with the prior art, different resolutions, multiple languages and complex formats of information can be processed more efficiently, ensuring the integrity of information collection, providing a rich data basis for accurate analysis, and effectively avoiding monitoring loopholes caused by information loss.

[0029] 2. In this scheme, by using advanced deep learning and machine learning models such as convolutional neural networks and latent semantic analysis, deep mining of prohibited features in images, texts and audios, accurate identification of potential risks, unique context analysis and multi-modal collaborative analysis, combined with fine prohibited level division, can deal with different degrees of prohibited content, compared with traditional methods, greatly improve the identification accuracy, reduce misjudgment, and better adapt to complex and variable prohibited forms.

[0030] 3. In this scheme, by automatically triggering the corresponding pop-up shielding mechanism according to the prohibited level, for different prohibited conditions of text, image and sound, local or whole mosaic, sound elimination technology is used respectively, which can effectively block the spread of prohibited content without affecting the normal information display. The existing technology often lacks such flexible and targeted processing method, and the method and system can better balance information security and user experience, and ensure that users are not disturbed by prohibited information. BRIEF DESCRIPTION OF DRAWINGS

[0031] The above description is only a summary of the technical scheme of the present application. In order to more clearly understand the technical means of the present application and can be implemented according to the content of the specification, the following will be described in detail with the preferred embodiments of the present application and with the help of the accompanying drawings.

[0032] Figure 1 It is a step flow chart of a network information security monitoring method based on artificial intelligence;

[0033] Figure 2 It is a system framework diagram of a network information security monitoring system based on artificial intelligence. DETAILED DESCRIPTION

[0034] The preferred embodiments of the present application will be described in detail with reference to the accompanying drawings However, the present application can be realized in various different forms, and therefore the present application is not limited to the embodiments described below;

[0035] The technical scheme in the embodiments of the present application is to solve the problems in the above background art, and the general idea is as follows:

[0036] Embodiment 1:

[0037] Please refer to Figure 1 The present embodiment introduces a network information security monitoring method based on artificial intelligence, which includes the following steps:

[0038] Step 1: Interface information collection, real-time monitoring of user interface and capturing all information on the interface, including desktop, application window and document content;

[0039] Screen capture technology: Develop or integrate screen capture tools that can capture snapshots of the user interface at a high frequency (e.g., once per second), including web pages being viewed, documents being edited, application interfaces being run, etc. The screen capture tools need to be able to handle different resolutions and screen sizes to ensure the completeness and accuracy of the information.

[0040] Audio capture technology: Extract video streams from the user interface, which may include movies being played, interfaces of video conferencing software, online courses, or any other video content containing audio. Then separate the audio stream from the extracted video stream and perform content extraction on the extracted audio stream using speech-to-text.

[0041] Text capture technology: For text information on the interface, use optical character recognition technology to extract text from images, including different fonts, sizes, and colors, and be able to recognize multiple languages.

[0042] Step 2: Risk analysis,

[0043] 2.1: Image information analysis, use image recognition technology, build deep learning models to identify prohibited content in images, these models can identify prohibited items, scenes or characters in images;

[0044] Specifically, we use Convolutional Neural Networks (CNN) to identify prohibited content in images. CNN identifies items, scenes or characters by learning features in images. Here is a basic architecture formula and meaning of CNN:

[0045] Convolutional layer:

[0046] f(x) = σ(W * x + b);

[0047] Principle: The input image x is convolved with the convolution kernel (filter) W to extract local features of the image. The convolution kernel slides over the image, calculates the weighted sum of the corresponding position, and adds the bias term b. Then the activation function σ is introduced to introduce nonlinearity, enhancing the expression ability of the model. Different convolution kernels can learn different image features such as edges and textures.

[0048] Example: When identifying whether an image contains prohibited items (such as guns), the convolution kernel can learn the shape and outline of the gun. For example, a specific convolution kernel may be sensitive to the line features of the gun barrel. By sliding the convolution kernel over the image, it can detect areas similar to the gun barrel line in the image, helping to determine whether the image contains a gun.

[0049] Pooling layer:

[0050]

[0051] Principle: Pooling layers are mainly used to reduce the dimensionality of feature maps, reduce computational complexity, and extract important features. In max pooling, the input feature map x is divided into several pooling windows (size n), and the maximum value in each window is selected as the output. This way, the most significant feature information in the image is preserved, and the image translation, rotation, etc. have certain invariance, improving the generalization ability of the model.

[0052] Example: Suppose in a feature map of an image, a certain region contains part of the features of a gun, but there are also some noise information. Through the max pooling operation, in the pooling window of this region, the most significant feature value (such as the larger value related to the gun feature) is selected as the output, suppressing the noise information and highlighting the key features of the gun, so that the subsequent network layers can focus more on important features for learning and judgment.

[0053] Fully connected layer:

[0054] y = W * h + b

[0055] Principle: The fully connected layer flattens the feature map after convolution and pooling, then multiplies it with the weight matrix W, and adds the bias term b to get the final output y. The role of the fully connected layer is to integrate and classify the extracted features, mapping them to a specific class label space.

[0056] Example: In the gun recognition task, after extracting the features related to the gun in the image through the previous convolution and pooling layers, the fully connected layer integrates these features and, according to the learned weights and biases in the training process, judges whether the image belongs to the "gun" category or the "non-gun" category. For example, if the weight multiplied by the feature corresponding to the gun category gets a high score and exceeds the set threshold, the image is judged as a prohibited image containing a gun.

[0057] Loss function:

[0058]

[0059] Principle: The loss function is used to measure the difference between the model's prediction and the true label. In image classification tasks, y i is the true label (such as the true situation of whether the image contains a gun), is the probability of the model predicting a certain category, and by minimizing the loss function, using gradient descent and other optimization algorithms to adjust the parameters (convolution kernel, weight, bias, etc.) in the convolutional neural network, the model's prediction result becomes more accurate.

[0060] Example: When training a gun recognition model, if an image containing a gun is incorrectly predicted by the model as not containing a gun, the loss value calculated by the loss function will be large. Through the backpropagation algorithm, the gradient of each layer parameter is calculated according to the loss value, and then the parameters are adjusted so that the model gradually improves the recognition accuracy of gun images in subsequent training, reducing the loss value. For example, through multiple iterations of training, the model can more accurately identify the features of the gun in the image, thus more accurately determining whether the image is a prohibited gun image.

[0061] 2.2: Text and audio information analysis, using trained machine learning models to identify prohibited words and phrases, such as sentiment analysis, entity recognition, semantic analysis, etc. In-depth analysis of the text information collected in the audio and text can help identify potential prohibited content, such as pornography, violence, discrimination, etc.

[0062] The above-mentioned training of machine learning models can be specifically processed by latent semantic analysis (LSA), and the specific algorithm formula of LSA is as follows:

[0063] A = URVT;

[0064] Principle: LSA uses singular value decomposition (SVD) to extract the theme or concept of the text, A is the document-term matrix, U and V are the left and right singular vectors respectively, and R is the diagonal matrix of singular values, which collectively represent the latent relationship between documents and terms.

[0065] Specifically, for the identification of prohibited content, LSA can be applied to text information analysis in the following ways:

[0066] Prohibited word identification: By constructing a document-word matrix and applying SVD, LSA can identify words related to prohibited content and label and classify them in the latent semantic space;

[0067] Context analysis: LSA can understand the contextual meaning of prohibited words by analyzing co-occurrence patterns, thus more accurately identifying prohibited content;

[0068] Prohibited content classification: Using the latent semantic features extracted by LSA, documents containing prohibited content can be classified for further processing and monitoring.

[0069] Example:

[0070] Suppose we have a simple corpus containing three sentences: 1. I like apples, 2. I love apples and oranges, 3. I eat oranges; First, we construct a word-document matrix using binary frequency (i.e. 1 if the word appears in the document, 0 otherwise);

[0071] Document I like apples love and oranges eat 1 1 1 1 0 0 0 0 2 1 0 1 1 1 1 0 3 1 0 0 0 0 1 1

[0072] Next, we perform SVD decomposition on this matrix. For simplicity, let's assume that we have identified two hidden topics by some means (possibly manually or algorithmically): Topic A is related to "likes" and Topic B is related to "food". We set the weights such that "like" and "love" have higher weights with Topic A, and "apple" and "orange" have higher weights with Topic B. In this way, we can obtain a simplified latent semantic space representation;

[0073] In this way, LSA helps us extract hidden semantic structures from text, enabling more accurate text comparison and information retrieval. For example, in the context of prohibited content identification, LSA can identify words and documents related to prohibited topics, even if these words do not directly violate the rules on their own, but through contextual analysis, their potential association with prohibited content is identified.

[0074] Step 3: Prohibited information classification

[0075] 3.1: Prohibited information analysis

[0076] 3.1.1: Contextual analysis

[0077] In analyzing textual information, in addition to considering whether the words themselves are prohibited, the semantic relationship between the words and their context must also be considered. For example, the word "violence" may be a normal word on its own, but in the context of "committing violent acts", it can be determined to be prohibited content. For image information, the scene in which the image is located and its association with other elements must also be considered. For example, a picture depicting a war scene may contain some specific, symbolic, and non-compliant signs or behaviors that constitute prohibited content. In terms of audio, factors such as the tone, speed, and coordination with other sound elements of the voice also affect the determination of prohibited content. For example, a voice expression with a threatening tone and containing prohibited words needs to be paid special attention to;

[0078] Through techniques such as latent semantic analysis (LSA), the potential semantic relationships of textual information are deeply explored to better understand the meanings of words in different contexts, thus more accurately determining whether they are prohibited. For images, features learned by models such as convolutional neural networks (CNN) and the associations between different image elements are used to comprehensively determine whether the image as a whole or part constitutes prohibited content. In audio analysis, after converting audio to text using speech recognition technology, relevant text analysis techniques are used in combination with other features of the voice to conduct a comprehensive analysis of prohibited content;

[0079] 3.1.2: Multimodal analysis

[0080] Data fusion: effectively fuse the text, image and sound information collected from the interface. For example, for a multimedia file containing a video, first use screen capture technology to obtain the video screen (image information), and extract the audio stream (sound information) in the video, and then use optical character recognition technology (OCR) to extract text information (text information) from the possible text part in the video screen. Then unify the data format conversion and alignment of the three modalities for subsequent comprehensive analysis;

[0081] Collaborative analysis: establish a multi-modal analysis model to enable text, image and sound information to complement and verify each other. For example, when judging whether an online video course contains prohibited content, image recognition technology can detect whether there are prohibited items or scenes in the video screen, audio content extracted by speech recognition technology can be analyzed to determine whether there are prohibited words or inappropriate expressions, and text information such as text prompts appearing in the video screen is analyzed, and the results of the three aspects are combined to more accurately determine whether there is prohibited content and the degree of violation;

[0082] 3.2: Prohibited level classification

[0083] 3.2.1: First-level violation

[0084] Local feature recognition: For text content, not only specific prohibited words are identified, but also the specific location of these words in the sentence or paragraph is determined to facilitate subsequent precise local mosaic processing. For image content, the prohibited part in the image is accurately identified, such as finding the hidden prohibited sign or scene in a complex landscape picture and clearly specifying its boundary and location information to provide accurate basis for the application of dynamic mosaic technology. For sound content, the start and end positions of prohibited sentences and the positions of key prohibited words in the sentences are accurately identified to facilitate effective local sound elimination processing;

[0085] Partial marking and processing: After identifying the first-level prohibited content, the system clearly marks the local part of the violation, such as marking the location of prohibited words with special colors or symbols in a document, highlighting the location of prohibited parts on the image in a certain way (such as flashing frame, etc.), and displaying the time interval of prohibited sentences in the sound playback interface. Then, according to different content types, apply corresponding local processing techniques such as local mosaic technology for text, dynamic mosaic technology for images, and local sound elimination technology for sound to process and effectively shield the prohibited part without affecting other normal content as much as possible;

[0086] 3.2.2: Second-level violation

[0087] Overall feature judgment: For text content, it can be determined whether the overall content is prohibited through comprehensive analysis of the entire document or text paragraph, which may involve analysis of the theme, emotional tendency, semantic coherence, etc. For image content, it can be determined whether a picture is completely inconsistent with the regulations, such as a picture full of prohibited scenes or objects. For sound content, it can be determined whether the entire sound segment is full of prohibited expressions or bad information.

[0088] Overall marking and processing: Once it is determined that it is a secondary prohibited content, the system uniformly marks the entire document, image or sound, such as marking "secondary prohibited" at the beginning of the document, covering a specific identification layer on the image to indicate its prohibited state, and giving clear prompt information in the sound playing interface. Then, the overall mosaic technology is applied to shield the text and image, and the overall sound elimination technology is applied to eliminate the sound, so as to completely shield the prohibited content.

[0089] Step 4: Pop-up shielding, when detecting prohibited information, corresponding pop-up shielding processing is carried out according to the level of prohibited information, so as to effectively block the presentation and spread of prohibited content to users.

[0090] 4.1: Text content shielding

[0091] 4.1.1: Local mosaic pop-up technology (for first-level prohibited text)

[0092] Once the first-level prohibited text content is identified, the system quickly starts the local mosaic pop-up mechanism; first, determine the specific position of the prohibited vocabulary in the text, including its line, column and character interval, etc. Detailed information;

[0093] Then, generate a pop-up window that fits the text display area, which presents a mosaic effect at the position of the prohibited vocabulary, shielding the prohibited vocabulary; when designing the pop-up window, ensure that its size, shape and prohibited vocabulary area match, and the visual attributes such as color and transparency of the pop-up window are carefully adjusted, which can effectively shield the prohibited vocabulary and will not cause too much interference to the reading of surrounding normal text, ensuring that users can clearly read other non-prohibited text content; for example, if the prohibited vocabulary "violence" is detected in an article, a rectangular mosaic pop-up window that covers the "violence" words will be popped up at the position of the vocabulary, the background color of the pop-up window can be set to a light gray color similar to the text background, and the transparency is set to about 70%, which can shield the prohibited words and can vaguely show the outline of the underlying text, so that the overall coherence and readability of the article can be maintained;

[0094] 4.1.2: Overall mosaic pop-up technology (for secondary prohibited text)

[0095] For the text content judged as secondary prohibited, such as the whole document or the entire interface text contains prohibited information, the system enables the overall mosaic pop-up window;

[0096] The pop-up window completely covers all the text content with a mosaic pattern. The style of the pop-up window can adopt uniformly distributed small square mosaic or blurred mosaic effect, so that the text content cannot be recognized by the user. At the same time, the pop-up window can display corresponding prompt information, such as "the content contains prohibited information and has been shielded", to inform the user of the shielding reason. For example, if all the text on a web page interface is prohibited propaganda content, the system will pop up a full-screen mosaic pop-up window to cover the entire web page text area. The user can only see the mosaic interface with prompt information and cannot obtain any prohibited text information.

[0097] 4.2: Image content shielding

[0098] 4.2.1: Dynamic mosaic pop-up window technology (for first-level prohibited images)

[0099] When the system detects first-level prohibited image content, that is, only part of the image area contains prohibited elements, the dynamic mosaic pop-up window technology starts to work, accurately identifies and tracks the position, shape and size changes of the prohibited image part on the screen in real time.

[0100] As the prohibited image part moves or changes, the system dynamically generates a mosaic pop-up window that matches it. The shape, size and position of the pop-up window always match the prohibited image part, and it is shielded with a mosaic effect. For example, during the playback of a video, if a prohibited item (such as a prohibited weapon) appears in the picture, the system will quickly generate a dynamic mosaic pop-up window in the area where the weapon is located. Regardless of how the video picture switches or the lens moves, the pop-up window can accurately follow the position changes of the prohibited weapon and continuously shield it, ensuring that the user cannot see the prohibited image details, while the other normal image parts of the video can still be played normally, without affecting the overall viewing experience.

[0101] 4.2.2: Full-screen mosaic pop-up window technology (for second-level prohibited images)

[0102] For second-level prohibited images, that is, the entire image is prohibited content, the full-screen mosaic pop-up window technology is triggered to generate a full-screen mosaic pop-up window that covers the entire image display area.

[0103] The pop-up window can adopt a unified mosaic style, such as a dense pixelated mosaic or a color blurred mosaic effect, to completely mask the entire image and prevent the user from seeing any prohibited image information; at the same time, the pop-up window can display prompt information such as "this image contains prohibited content and has been shielded"; for example, if an image is full of prohibited violent and bloody scenes, the system will pop up a full-screen mosaic pop-up window, and the user can only see the screen full of mosaics and prompt information, and cannot access the prohibited image content;

[0104] For the above-mentioned first-level prohibited text and images, positioning and shielding and following shielding are performed through the target positioning and tracking algorithm, which is as follows:

[0105] (1) Algorithm formula

[0106] Feature extraction formula: O = R(S / K), where S represents image data, K represents text data, R is a feature extraction function, and O is the extracted feature vector; for images, color, texture, shape and other features of the image are extracted through convolutional neural network and other technologies; for text, word vectors, syntax structures and other features of words are extracted, for example, shape features of prohibited knives are extracted in images, and semantic features of prohibited words are extracted in text;

[0107] Target matching formula: G = P(D, L), where L is a predefined prohibited target feature template (for text, it is a feature of prohibited words, and for images, it is a feature of prohibited articles or scenes), P is a matching function, and G is a matching score, which represents the similarity between the current content and the prohibited target; the extracted features are matched with the predefined prohibited target feature template to calculate the similarity. The higher the score, the more likely it is a prohibited target, for example, the shape features extracted from the image are compared and matched with the shape template features of the prohibited knife, and the word features in the text are matched with the prohibited word template features;

[0108] Position determination formula: l = d(m, i / t), where i / t is the size information of the image or text, d is a function of determining the position of the prohibited target according to the matching score and the size, and l is the position information of the prohibited target in the image or text (for images, it may be a coordinate range, and for text, it may be a character interval); according to the matching score and the size of the image or text, the specific position of the prohibited target in it is determined, the coordinate range of the prohibited article in the image is determined, and the character interval of the prohibited word in the text is determined;

[0109] Tracking update formula: n = u(l, m, e), where m is the target motion model (for moving targets in the image, such as contraband in video; for text, it can be a model of scrolling or other operations), e is the data of the new image frame or text update part, u is a function of updating the position of the contraband target according to the motion model and new data, and n is the updated position of the contraband target; when the image is a video or the text has scrolling or other operations, the position of the contraband target is updated according to the target motion model and the new data (new video frame or new text part), for example, in a video, the position of the knife in the picture is updated according to the motion trend (motion model) of the contraband and the content of the new video frame; in scrolling text, the position of the contraband vocabulary is updated according to the scrolling operation model and the newly displayed text content;

[0110] 4.3: Sound content shielding

[0111] 4.3.1: Local muting pop-up technology (for first-level contraband sound)

[0112] When a first-level contraband sound content is detected, that is, only part of the sentence or segment in the sound is contraband information, the local muting pop-up technology works, which accurately identifies the start point, end point and position information of the contraband sound sentence in the entire audio stream through voice endpoint detection technology;

[0113] Then, an audio muting pop-up is generated, which performs muting processing on the time period where the contraband sound sentence is located during audio playback, so that this part of the sound cannot be played out, while other normal sound parts are not affected and continue to be played normally; for example, in an audio file containing multiple sound elements (such as containing background music, dialogue and narration), if a sentence of narration contains contraband vocabulary, the system will start the local muting pop-up when playing the sentence of narration, and mute the contraband narration, while the background music and other normal dialogue can still be heard by the user, maintaining the coherence and audibility of the entire audio;

[0114] 4.3.2: Whole muting pop-up technology (for second-level contraband sound)

[0115] For sound content judged as second-level contraband, that is, the entire audio segment is contraband information, the whole muting pop-up technology is started, and the system generates a muting pop-up that covers the entire audio playback area after determining the overall contraband nature of the audio through overall feature analysis and contraband content identification;

[0116] The pop-up window completely blocks the output of the sound when the audio is playing, making the entire audio silent, and the user cannot hear any audio content. At the same time, the pop-up window can display prompt information such as "the audio contains prohibited content and has been shielded". For example, if an audio file is entirely prohibited violent threat speech, the system will pop up an overall sound elimination pop-up window, and the user will not hear any audio sound, and can only see the prompt information, effectively preventing the spread of prohibited sound.

[0117] Embodiment 2:

[0118] Please refer to Figure 2 The present embodiment introduces a network information security monitoring system based on artificial intelligence, which is used to realize the network information security monitoring method in embodiment one. The system includes the following modules:

[0119] 1. Information collection module

[0120] 1.1 Screen capture unit

[0121] Responsible for real-time monitoring of the user interface, capturing snapshots of the user interface at a high frequency (once per second), ensuring that different types of image information such as the web page being browsed, the document being edited, and the application interface being run can be obtained, and can adapt to different resolutions and screen sizes, ensuring the comprehensiveness and accuracy of the information;

[0122] 1.2 Audio extraction unit

[0123] Extracts video streams from the user interface, such as movies being played, video conference software interfaces, online courses, and other audio-containing video content, and separates the audio stream from the extracted video stream for subsequent analysis and processing of audio information;

[0124] 1.3 Text recognition unit

[0125] Uses optical character recognition technology (OCR) to extract text information from images, can recognize text of different fonts, sizes, and colors, and supports multiple languages. This unit mainly collects text information on the interface to provide data sources for subsequent text information analysis;

[0126] 2. Risk analysis module

[0127] 2.1 Image recognition unit

[0128] Uses a convolutional neural network (CNN) to establish a deep learning model to identify prohibited content such as prohibited items, scenes, or characters by learning features in images;

[0129] Responsible for inputting the collected images into the built CNN model, analyzing the images through the features learned by the model and the associations between different image elements, and determining whether the overall or part of the image constitutes prohibited content.

[0130] 2.2, text audio analysis unit

[0131] Based on latent semantic analysis (LSA), a trained machine learning model is built, which uses singular value decomposition (SVD) to extract the theme or concept of the text. Through this model, prohibited words and phrases are identified in the text information, and in-depth processing such as sentiment analysis, entity recognition, and semantic analysis is performed on the collected text information in the audio and text, helping to identify potential prohibited content such as pornography, violence, and discrimination.

[0132] Using the trained machine learning model, analyze the text and audio information, including prohibited word identification (identify prohibited content-related words by building a document and word matrix and applying SVD, and label and classify them in the latent semantic space), context analysis (understand the context meaning of prohibited words by analyzing co-occurrence patterns to more accurately identify prohibited content), and prohibited content classification (use the latent semantic features extracted by LSA to classify documents containing prohibited content for further processing and monitoring).

[0133] 3, prohibited information classification module

[0134] 3.1, prohibited information analysis unit

[0135] In analyzing text information, combine techniques such as latent semantic analysis (LSA) to deeply mine the latent semantic relationships of text information and understand the meaning of words in different contexts to accurately determine whether it is prohibited. For image information, use features learned by models such as convolutional neural networks (CNN) and associations between image elements for comprehensive judgment. In audio analysis, after converting audio to text using speech recognition technology, use related text analysis techniques combined with other features of the speech for comprehensive prohibited content analysis, while paying attention to the influence of factors such as the tone, speed, and coordination with other sound elements of the speech on prohibited content determination.

[0136] Data fusion part: effectively fuse the text, image, and sound information collected from the interface. For example, for multimedia files, first obtain the video screen (image information), extract the audio stream (sound information), then extract the text information (text information) from the video screen, and then convert and align the information of the three modalities into a unified data format for subsequent comprehensive analysis.

[0137] Cooperative analysis part: Establish a multi-modal analysis model to make text, image and sound information complement and verify each other. For example, when judging whether the online video course contains prohibited content, the analysis results of image recognition technology on video pictures, speech recognition technology on audio content and text information such as video picture text prompts can more accurately determine whether there is prohibited content and the degree of violation.

[0138] 3.2, Prohibited level division unit

[0139] First level prohibited processing:

[0140] For text content, the specific position of prohibited words in the sentence or paragraph can be determined; for image content, the prohibited part and its boundary and position information in the image can be accurately identified; in terms of sound content, the start and end positions of prohibited sentences and the positions of key prohibited words can be accurately identified.

[0141] When the first level prohibited content is identified, the local part of the prohibited content is marked. For example, the position of the prohibited word in the document is marked with special color or symbol, the position of the prohibited part in the image is highlighted with flashing frame, and the time interval of the prohibited sentence in the sound playing interface is displayed. Then, according to different content types, apply corresponding local processing technology, such as local mosaic technology for text, dynamic mosaic technology for image, and local mute technology for sound, to shield the prohibited part without affecting other normal content.

[0142] Second level prohibited processing:

[0143] For text content, the overall content is judged to be prohibited or not through comprehensive analysis of the entire document or text paragraph, involving analysis of text theme, emotional tendency, semantic coherence and other aspects; for image content, it is judged whether the image is completely inconsistent with the regulation; in terms of sound content, it is judged whether the entire sound segment is full of prohibited expressions or bad information.

[0144] Once it is determined to be a second level prohibited content, the entire document, image or sound is uniformly marked. For example, the word "second level prohibited" is marked at the beginning of the document, a specific identification layer is overlaid on the image, and clear prompt information is given in the sound playing interface. Then, apply the overall mosaic technology to shield the text and image, and apply the overall mute technology to eliminate the sound, so as to completely shield the prohibited content.

[0145] 4, Pop-up shielding module

[0146] 4.1, Text content shielding unit

[0147] When the first level of prohibited text content is identified, the position of the prohibited vocabulary in the text is determined, a pop-up window that fits the text display area is generated, a mosaic effect is presented at the position of the prohibited vocabulary to block it, the size and shape of the pop-up window match the area of the prohibited vocabulary, and the visual properties such as color and transparency are adjusted to block the prohibited vocabulary without affecting the reading of normal text around it.

[0148] For the second level of prohibited text content, after scanning the entire text area, a large pop-up window covering the entire text display area is generated to completely block all text content with a mosaic pattern. Uniformly distributed small square mosaic or blurred mosaic effect can be used, and prompt information is displayed on the pop-up window to inform the user of the reason for the shielding.

[0149] 4.2, image content shielding unit

[0150] When the first level of prohibited image content is detected, the position, shape and size changes of the prohibited image part on the screen are identified and tracked in real time, and a mosaic pop-up window that matches the changes is dynamically generated to block it, ensuring that other normal image parts of the video are played normally.

[0151] For the second level of prohibited image, after determining the prohibited nature through overall feature analysis of the image, a full-screen mosaic pop-up window covering the entire image display area is generated, which uses a uniform mosaic style (such as dense pixelated mosaic or color blurred mosaic effect) to block the image and display prompt information.

[0152] 4.3, sound content shielding unit

[0153] When the first level of prohibited sound content is detected, the start point, end point and position information of the prohibited sound sentence are identified using voice endpoint detection technology, an audio mute pop-up window is generated, and mute processing is performed in the time period where the prohibited sound sentence is located to ensure that other normal sounds are played normally.

[0154] For the second level of prohibited sound content, after determining the overall prohibited nature through overall feature analysis of the audio, a mute pop-up window covering the entire audio playback area is generated to completely block the sound output, making the audio in a silent state, and displaying prompt information.

[0155] The specific implementation is as follows:

[0156] 1. Text content shielding

[0157] Level 1 forbidden words: When the system detects level 1 forbidden words, such as finding the forbidden word "drug transaction" in the chat window of an instant messaging software, the system immediately starts the local mosaic pop-up mechanism. The system determines the position of the four characters "drug transaction" in the chat window through text positioning algorithm, and then generates a rectangular pop-up window that covers the four characters. The background color of the pop-up window is set to a gray color similar to but slightly darker than the background of the chat window, with a transparency of about 60%. This effectively blocks the forbidden words while allowing the user to vaguely see the outline of the underlying text, without affecting the reading of other chat content. The pop-up window appears instantly when the forbidden words appear and remains displayed until the chat record is scrolled or the window is closed.

[0158] Level 2 forbidden words: If the entire text content of a news page on a website is determined to be level 2 forbidden (such as promoting reactionary ideas), the system triggers the overall mosaic pop-up. The system quickly scans and determines the text area of the entire news page, and generates a full-screen pop-up window that covers the entire text display area after confirming the violation. The pop-up window uses a uniform distribution of small square mosaic effect with a gray color, making the text completely illegible. At the same time, it displays a prominent prompt message "This page contains forbidden information and has been blocked" in the center of the pop-up window. Users can only see this prompt message and the page covered by the mosaic, and cannot access any forbidden text content.

[0159] 2. Image content shielding

[0160] Level 1 forbidden images: When playing a surveillance video, if a person holding a forbidden knife (level 1 forbidden image) suddenly appears in the picture, the system uses a combination of deep learning-based target detection algorithm and Kalman filter algorithm to identify and track the position, shape and size changes of the knife in the picture in real time. As the knife moves, the system dynamically generates a mosaic pop-up window that matches the knife area. The shape and size of the pop-up window are always consistent with the knife area, and the color is black mosaic effect, effectively blocking the knife part. While the other normal pictures of the video (such as background environment, other characters, etc.) continue to play normally, the overall surveillance video can be watched and analyzed without any impact.

[0161] Level 2 forbidden images: When a picture full of pornographic content (level 2 forbidden image) is detected, the system determines its forbidden nature through overall feature analysis of the image (such as color distribution, image element features, etc.) and forbidden content determination algorithm, and generates a full-screen mosaic pop-up window that covers the entire image display area. The pop-up window uses a color-fuzzy mosaic effect to make the image completely invisible, while displaying a prompt message "This image contains forbidden content and has been blocked" on the pop-up window, preventing users from accessing the forbidden image content.

[0162] 3. Sound content shielding

[0163] Primary forbidden sound: When listening to an audio podcast, if one of the voiceovers contains the forbidden word "fraudulent method" (primary forbidden sound), the system uses speech endpoint detection technology and audio feature extraction algorithms to accurately identify the start time (e.g. 2:10) and end time (e.g. 2:20) of the forbidden sentence. When the audio reaches 2:10, the system generates an audio mute pop-up, which mutes the audio during this time period, so that the listener cannot hear the forbidden voiceover, while the other normal audio content of the podcast (such as background music, other normal voiceovers, etc.) continues to play normally, maintaining the coherence and audibility of the audio.

[0164] Secondary forbidden sound: If an audio file is entirely composed of speech content promoting violent crime (secondary forbidden sound), the system determines its overall forbidden nature through overall feature analysis (such as speech tone, speech speed, word frequency, etc.) and forbidden content recognition algorithms, and generates a mute pop-up that covers the entire audio playback area. When attempting to play the audio, the mute pop-up takes effect, and the audio is completely silent, so the user cannot hear any audio content, while the audio playback interface displays the prompt "This audio contains forbidden content and has been blocked", effectively preventing the spread of forbidden sound.

[0165] Finally, it should be noted that the above embodiments are merely examples for the purpose of clearly illustrating the present application, and are not intended to limit the implementation. For those skilled in the art, based on the above description, other different forms of changes or variations can also be made. Here, it is not necessary or possible to exhaust all the implementations. The obvious changes or variations derived therefrom are still within the scope of protection of the present application.

Claims

1. An artificial intelligence-based network information security monitoring method, characterized in that, The method comprises the following steps: Step 1, interface information collection: real-time monitoring of the user interface and capturing all information on the interface, including the desktop, application windows and document content, capturing snapshots of the user interface at a high frequency through a screen capture tool, extracting video streams from the user interface and separating out audio streams, and extracting text information from images using optical character recognition technology; Step 2, risk analysis: risk analysis of the collected information, including image information analysis and text and audio information analysis, image information analysis using image recognition technology, identifying prohibited content in images through convolutional neural networks, and text and audio information analysis using trained machine learning models, identifying prohibited words and phrases through latent semantic analysis technology; Step 3, prohibited information classification: classification of the analyzed prohibited information, including first-level prohibited and second-level prohibited, first-level prohibited requiring local feature recognition and partial marking and processing, and second-level prohibited requiring overall feature judgment and overall marking and processing, combining context analysis and multi-modal analysis during the analysis process, context analysis considering the context semantic relationship in text, image and audio information, and multi-modal analysis fusing and cooperatively analyzing text, image and sound information; Step 4, pop-up window shielding: when prohibited information is detected, corresponding pop-up window shielding processing is performed according to the prohibited information level, local mosaic pop-up window technology is used for first-level prohibited text, overall mosaic pop-up window technology is used for second-level prohibited text, dynamic mosaic pop-up window technology is used for first-level prohibited images, full-screen mosaic pop-up window technology is used for second-level prohibited images, local sound-off pop-up window technology is used for first-level prohibited sound, and overall sound-off pop-up window technology is used for second-level prohibited sound.

2. The network information security monitoring method based on artificial intelligence according to claim 1, characterized in that, The screen capture tool can process user interfaces of different resolutions and screen sizes to ensure the comprehensiveness and accuracy of information collection, the video stream extraction and audio stream separation are achieved through audio capture technology, and the optical character recognition technology can recognize text of different fonts, sizes and colors and supports multiple languages.

3. The network information security monitoring method based on artificial intelligence of claim 1, wherein, The convolutional neural network comprises a convolutional layer, a pooling layer and a fully connected layer, the convolutional layer f(x)=σ(W*x+b), the pooling layer The fully connected layer y=W*h+b, and the loss function is The latent semantic analysis technology extracts the theme or concept of the text through singular value decomposition, and an algorithm formula thereof is A=URVT, which is used for forbidden word identification, context analysis and forbidden content classification.

4. The network information security monitoring method based on artificial intelligence of claim 1, wherein, The local mosaic pop-up window is positioned and shielded by a target positioning and tracking algorithm, and the target positioning and tracking algorithm formula is as follows: Feature extraction formula: O = R(S / K), where S represents image data, K represents text data, R is a feature extraction function, and O is the extracted feature vector; Target matching formula: G = P(D, L), where L is a predefined prohibited target feature template, P is a matching function, G is a matching score, and represents the similarity between the current content and the prohibited target; Position determination formula: l = d(m, i / t), where i / t is the size information of the image or text, d is a function of determining the position of the prohibited target according to the matching score and the size, and l is the position information of the prohibited target in the image or text; Tracking update formula: n = u(l, m, e), where m is a target motion model, e is the data of the new image frame or text update part, u is a function of updating the position of the prohibited target according to the motion model and the new data, and n is the updated position of the prohibited target.

5. An artificial intelligence-based network information security monitoring system for performing the artificial intelligence-based network information security monitoring method of any one of claims 1-4, characterized in that, The system comprises the following modules: The information collection module is used for real-time monitoring of the user interface and capturing all information on the interface, including a screen capture unit, an audio extraction unit and a text recognition unit. The screen capture unit is responsible for capturing snapshots of the user interface at a high frequency. The audio extraction unit is responsible for extracting a video stream from the user interface and separating the audio stream. The text recognition unit is responsible for extracting text information from the image. The risk analysis module is used for risk analysis of the collected information, including an image recognition unit and a text / audio analysis unit. The image recognition unit uses a convolutional neural network to establish a deep learning model to identify prohibited content in the image. The text / audio analysis unit uses a trained machine learning model based on latent semantic analysis to identify prohibited words and phrases. The prohibited information level division module is used for level division of the analyzed prohibited information, including a prohibited information analysis unit and a prohibited level division unit. The prohibited information analysis unit performs context analysis and multi-modal analysis. The prohibited level division unit determines the prohibited level according to the analysis result and performs corresponding processing. The pop-up shielding module is used for pop-up shielding processing according to the prohibited information level when prohibited information is detected, including a text content shielding unit, an image content shielding unit and a sound content shielding unit. The text content shielding unit uses corresponding mosaic pop-up technology for different levels of prohibited text. The image content shielding unit uses corresponding mosaic pop-up technology for different levels of prohibited images. The sound content shielding unit uses corresponding sound reduction pop-up technology for different levels of prohibited sounds.

6. The network information security monitoring system based on artificial intelligence of claim 5, wherein, The screen capture unit can adapt to user interfaces of different resolutions and screen sizes. The audio extraction unit can accurately separate the audio stream. The text recognition unit can recognize multiple types of text and support multiple languages.

Images