Monitoring device and monitoring method for public screen content security supervision

Abstract

The invention discloses a monitoring device and a monitoring method for public screen content security supervision, and belongs to the technical field of screen monitoring, and the method specifically comprises the following steps: compressing and temporarily storing a video published on a public screen, preprocessing the compressed and temporarily stored video, classifying and identifying the preprocessed video, and storing the classified and identified video into a database; different classification labels of video frame images are recognized, text information in the video frame images with the labels being sensitive content is extracted, sensitive detection is carried out, real-time decision making is carried out in combination with image content recognition and text sensitive detection results, and when illegal content is detected, alarm information is sent through a background management platform. Immediately shielding the sensitive video, and detecting the next video; monitoring of bad information and control of large-screen content display are achieved, and the safety of large-screen published content is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of screen monitoring, and in particular to a monitoring device and a monitoring method for security supervision of public screen content. Background Art

[0002] With the continuous development of technology and the reduction of costs, LED / OLED large screens (hereinafter referred to as large screens) are increasingly being used in various public information display and interactive platforms. Since the sources of information displayed on large screens are scattered and different, it is difficult to form reliable management for information release. Once the user makes an error in operation or the release system is attacked and hijacked, it will lead to information release accidents. Currently, video matrix devices on the market can control the content displayed on large screens, but the number of large screens that can be controlled by one device is limited, and the existing equipment needs to be modified and connected, which is difficult to implement and the cost is relatively high. At the same time, it is also difficult to solve the problem of the content release system being attacked and hijacked, and it is difficult to achieve unified and scalable large-screen release content monitoring and control.

[0003] For example, a Chinese patent with authorization announcement number CN116841424B discloses a screenshot monitoring method and system, which relates to the field of screenshot monitoring. The specific disclosed technical solution is to collect application windows of the screen interface, scan and analyze the content displayed in the application windows, extract window content information, logically combine window keywords belonging to the same window, generate keyword proofreading groups, perform sensitivity judgment on the keyword proofreading groups, obtain a first sensitivity evaluation, determine a second sensitivity evaluation of the screenshot based on the determined spatial logical relationship between the image feature factors, determine a comprehensive sensitivity evaluation of the screenshot based on the first sensitivity evaluation and the second sensitivity evaluation, comprehensively analyze the screenshot action features and the comprehensive sensitivity evaluation of the screenshot, determine whether to block and warn the screenshot action, and realize customized judgment of whether to block or warn the screenshot action based on the window content information.

[0004] For example, a Chinese patent with authorization announcement number CN107423187B discloses a screen monitoring method, device and system to solve the problem that the bitmap obtained by the client cannot be enlarged or reduced in proportion. The method is: the first terminal obtains the screen resolution of the second terminal; the first terminal sends an instruction of the user requesting to monitor the preset page through the screen of the second terminal to the server corresponding to the preset page, and the instruction carries the screen resolution of the second terminal; the first terminal receives the description information of the preset screen monitoring area in the preset page replied by the server, and the description information is used to enable the first terminal to capture the preset screen monitoring area in the preset page presented on the screen of the first terminal, and obtain a bitmap with the same aspect ratio as the screen resolution of the second terminal; the first terminal captures the preset screen monitoring area in the preset page presented on the screen of the first terminal at a regular interval according to the description information, and sends the capture result to the second terminal.

[0005] The defects of the above patents are: they all capture and analyze the content displayed on the screen, and are unable to provide analysis and warning before playback, posing a security risk. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes a monitoring device and a monitoring method for security supervision of public screen content. By adding an intermediary device in the last link of large-screen content release, real-time monitoring of large-screen content release and control of large-screen displayed content are achieved. At the same time, large-model AI technology is introduced to realize intelligent recognition and processing of content, greatly improving work efficiency. Regardless of the release method and device of the original content, the security monitoring of large-screen content release can be transparently realized to ensure the security of large-screen content release.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A monitoring method for public screen content security supervision, comprising:

[0009] Compress and temporarily store the video to be published on the public screen, and pre-process the compressed and temporarily stored video;

[0010] Classify and identify the preprocessed video and identify different classification labels of the video frame images;

[0011] Extract text information from video frame images labeled as sensitive content and perform sensitivity detection;

[0012] Combine the results of image content recognition and text sensitivity detection to make real-time decisions;

[0013] When illegal content is detected, an alarm message is sent through the background management platform, sensitive videos are immediately blocked, and the next video is tested.

[0014] Specifically, the pre-processed video is classified and identified to identify different classification labels of the video frame images, including:

[0015] Select a machine learning algorithm to build an image classification model, use an annotated image dataset to train the image classification model, set the model training goal to minimize the loss function, and the specific formula for minimizing the loss function is:

[0016] ,

[0017] Among them, L represents the minimized loss function of the image classification model, C represents the total number of image categories, and P c represents the actual category of the annotated image, represents the predicted category of the cth labeled image by the image classification model, represents the loss function;

[0018] The parameters of the image classification model are updated through back propagation to minimize the loss function. The specific formula for the update is:

[0019] ,

[0020] in, represents the parameters of the image classification model, represents the learning rate, It represents the partial derivative function;

[0021] Train the image classification model until the minimization loss function of the model's training objective converges and remains unchanged, then stop training.

[0022] Specifically, the classifying and identifying the preprocessed video to identify different classification labels of the video frame images also includes:

[0023] Set the video frame image at time t as I(t), and use the convolutional neural network to extract features from the video frame image at time t. The specific formula is:

[0024] ,

[0025] Among them, F(t) represents the feature map of the video frame image at time t, J n represents the nth layer of convolution operation, H n represents the nth layer of pooling operation, and n represents the total number of layers of the convolutional neural network;

[0026] The extracted feature map is mapped to the classification space through the fully connected layer. The specific formula is:

[0027] ,

[0028] Where V(t) represents the feature map of the video frame image after mapping at time t, W represents the weight matrix of the fully connected layer, and b represents the bias term;

[0029] The mapped result is transformed nonlinearly through the activation function, input into the trained image classification model, and output the classification of the video frame image at time t. The specific formula is:

[0030] ,

[0031] Among them, P(t) cl represents the probability that the video frame image category is cl at time t, e represents the exponential function, V(t) cl Represents the feature of category cl, V(t) i Represents the features of category i.

[0032] Specifically, extracting text information from video frame images labeled as sensitive content and performing sensitivity detection includes:

[0033] Use the OCR algorithm to extract the text information T(t) in the video frame image at time t;

[0034] Use multimodal embedding to analyze the text information T(t), embed the video frame image and the text information extracted by OCR into a high-dimensional vector, and set the video frame image embedding as I embed (t), calculate the multimodal vector of the text, the specific formula is:

[0035] ,

[0036] Among them, ME(t) represents the multimodal vector of the text in the video frame image at time t, W q Represents the text embedding matrix, which is used to convert text information into word vectors. Indicates a connection operation;

[0037] The multimodal vector of the text is input into the neural network for semantic analysis, the embedded features of sensitive words are transferred to the high-dimensional classification space, and the classification probability of sensitivity is obtained through the attention mechanism. The specific formula is:

[0038] ,

[0039] Among them, P class (t) represents the text classification probability vector, that is, the probability that the text belongs to different sensitive categories, W c represents the sensitive classification weight matrix, b c represents the bias vector, and softmax() represents the classification function;

[0040] A sensitivity score is calculated for the classification probability result of each frame of text. The specific formula for the sensitivity score is:

[0041] ,

[0042] Where S(t) represents the sensitivity score of the text in the A-frame video frame image, A represents the total number of video frame images, and P class (t) a Represents the text classification probability vector of the a-th video frame image, Represents the weight coefficient.

[0043] Specifically, the extracting of text information from the video frame image labeled as sensitive content and performing sensitivity detection also includes:

[0044] Set the text information sequence to T(1), T(2), ..., T(A), use the sequence model to analyze the text information sequence, combine the context of the current frame to detect sensitive content, and calculate the context score of the text information sequence. The specific formula is:

[0045] ,

[0046] Among them, S yj (t) represents the context score of the text in the video frame image at time t, f() represents the sequence model function, ME(t-1) represents the multimodal vector of the text in the video frame image at time t-1, and ME(t+1) represents the multimodal vector of the text in the video frame image at time t+1;

[0047] Calculate the final sensitivity score of the text in the video frame image at time t. The specific formula is:

[0048] ,

[0049] Among them, Final_Score(t) represents the final sensitivity score of the text in the video frame image at time t. and Represents the weight coefficient.

[0050] Specifically, the combination of the results of image content recognition and text sensitivity detection to make real-time decisions includes:

[0051] Set the image safety threshold to T img , the safety threshold of the text is T txt ;

[0052] The decision formula is used to determine whether to trigger an alarm. The specific formula of the decision formula is:

[0053] ,

[0054] in, The decision formula for the video frame image at time t, P(t) mg represents the probability that the video frame image category at time t is sensitive, Indicates that, =1, indicating that the video frame image at time t exceeds the threshold, triggering an alarm. =0, it means that the video frame image at time t does not exceed the threshold.

[0055] Specifically, the preprocessing includes: frame screenshot and denoising;

[0056] Frame screenshot, used to convert compressed and temporarily stored video into frames and convert them into video frame image data;

[0057] De-noising is used to filter noise and adjust brightness of the video frame image data after framing.

[0058] A monitoring device for public screen content security supervision, used to implement the monitoring method for public screen content security supervision, comprising: a video processing module, an image recognition and classification module, a text detection module, a real-time decision module and a shielding detection module;

[0059] The video processing module is used to compress and temporarily store the video published on the public screen, and pre-process the compressed and temporarily stored video;

[0060] The image recognition and classification module is used to classify and recognize the preprocessed video and identify different classification labels of the video frame images;

[0061] The text detection module is used to extract text information in video frame images labeled as sensitive content and perform sensitivity detection;

[0062] The real-time decision module is used to combine the results of image content recognition and text sensitivity detection to make real-time decisions;

[0063] The shielding detection module is used to send an alarm message through the background management platform when illegal content is detected, immediately shield the sensitive video, and detect the next video.

[0064] Specifically, the image recognition and classification module includes: a model training optimization unit, a feature extraction unit and an image recognition and classification unit;

[0065] The model training optimization unit is used to construct an image classification model and perform optimization and training;

[0066] The feature extraction unit is used to extract the features of the video frame image at time t;

[0067] The image recognition and classification unit is used to use the trained image classification model to recognize and classify the video frame image at time t.

[0068] Specifically, the text detection module includes: a sensitivity scoring unit, a context scoring unit and a sensitivity final scoring unit;

[0069] The sensitivity scoring unit is used to extract the multimodal vector of the text in the video frame image and perform sensitivity scoring;

[0070] The context scoring unit is used to analyze the text information sequence using the sequence model, perform sensitive content detection in combination with the context of the current frame, and calculate the context score of the text information sequence;

[0071] The final sensitivity scoring unit is used to calculate the final sensitivity score of the text in the video frame image at time t.

[0072] Compared with the prior art, the present invention has the following beneficial effects:

[0073] 1. The present invention proposes a monitoring method for the security supervision of public screen content. By adding an intermediary device in the last link of large-screen content release, real-time monitoring of large-screen content release is achieved. Regardless of the release method and device of the original content, the security monitoring of large-screen content release can be transparently achieved to ensure the security of large-screen content release.

[0074] 2. The present invention proposes a monitoring method for public screen content security supervision, which can reduce the reliance on manual review, improve the coverage and accuracy of content supervision, and automatically identify bad information, including inappropriate information in text, images and video content, to ensure that illegal content is warned and processed in the first place.

[0075] 3. The present invention proposes a monitoring method for content security supervision of public screens, which supports comprehensive supervision of multiple content formats and utilizes an efficient content security supervision mechanism, which can effectively reduce the spread of harmful content in public places and also provides traceable supervision records. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 A flow chart of a monitoring method for public screen content security supervision provided by the present invention;

[0077] Figure 2 A sensitive detection flow chart provided by the present invention;

[0078] Figure 3 A video transmission processing flow chart provided by the present invention;

[0079] Figure 4An architecture diagram of a monitoring device for public screen content security supervision provided by the present invention. DETAILED DESCRIPTION

[0080] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, in the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate the orientation or position relationship based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "No. 1", "No. 2" and "No. 3" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. The present invention is further explained below in conjunction with specific embodiments.

[0081] Example 1

[0082] See also Figure 1-Figure 3 , an embodiment of the present invention provides: a monitoring method for public screen content security supervision, comprising the following specific steps:

[0083] Step S1: compressing and temporarily storing the video published on the public screen, and preprocessing the compressed and temporarily stored video;

[0084] The preprocessing in step S1 includes: frame segmentation and denoising;

[0085] Frame screenshot, used to convert compressed and temporarily stored video into frames and convert them into video frame image data;

[0086] De-noising is used to filter noise and adjust brightness of the video frame image data after framing.

[0087] The LED control device maintains a reliable connection with the unified control system based on the Internet through VPN. After the HDMI video signal is sent from the signal source, it enters the LED control device. The LED control device encodes the HDMI signal into a digital video stream and transmits it to the back-end unified control system in real time when needed according to the configuration. The unified control system processes the received video.

[0088] Step S2: classify and identify the preprocessed video to identify different classification labels of the video frame images;

[0089] The specific steps of step S2 are:

[0090] Step S201: Select a machine learning algorithm to build an image classification model, use an annotated image dataset to train the image classification model, set the model training goal to minimize the loss function, and the specific formula for minimizing the loss function is:

[0091] ,

[0092] Among them, L represents the minimized loss function of the image classification model, C represents the total number of image categories, and P c represents the actual category of the annotated image, represents the predicted category of the cth labeled image by the image classification model, represents the loss function;

[0093] Loss Function The specific formula is:

[0094] ,in, Represents the category weight of the c-th data, which is used to deal with category imbalance. For example, if there are fewer samples in a certain category, it can be given a higher weight.

[0095] Cross entropy loss is a common method to measure the difference between two probability distributions. In classification problems, it evaluates the distance between the predicted distribution of the model output and the true distribution. The weights are introduced to balance the impact of class imbalance. In many practical applications, some categories may have significantly fewer samples than other categories, causing the model to be biased towards the main categories. By giving higher weights to minority category samples, the model can pay more attention to these samples during training, thereby improving its prediction performance. The goal of minimizing this loss function is to adjust the model parameters through iterative optimization (such as gradient descent) so that the prediction results are closer to the true label. This method can not only improve the accuracy of the model, but also enhance the ability to identify minority category faults.

[0096] Step S202: Update the parameters of the image classification model through back propagation to minimize the loss function. The specific formula for updating is:

[0097] ,

[0098] in, represents the parameters of the image classification model, represents the learning rate, It represents the partial derivative function;

[0099] Step S203: training the image classification model until the minimization loss function of the training objective of the model converges and remains unchanged, and then stopping the training;

[0100] Step S204: Set the video frame image at time t as I(t), and use a convolutional neural network to extract features from the video frame image at time t. The specific formula is:

[0101] ,

[0102] Among them, F(t) represents the feature map of the video frame image at time t, J n represents the nth layer of convolution operation, H n represents the nth layer of pooling operation, and n represents the total number of layers of the convolutional neural network;

[0103] Convolution operation: extracts local features through filters, which can effectively identify basic features such as edges and textures in images. Pooling operation: reduces feature dimensions through downsampling, enhances feature robustness, and reduces computational complexity.

[0104] Step S205: Map the extracted feature map to the classification space through the fully connected layer. The specific formula is:

[0105] ,

[0106] Where V(t) represents the feature map of the video frame image after mapping at time t, W represents the weight matrix of the fully connected layer, and b represents the bias term;

[0107] Fully connected layer: maps the extracted features to the final classification space to form a decision boundary;

[0108] Step S206: The mapped result is transformed nonlinearly through an activation function, input into the trained image classification model, and output the classification of the video frame image at time t. The specific formula is:

[0109] ,

[0110] Among them, P(t) cl represents the probability that the video frame image category is cl at time t, e represents the exponential function, V(t) cl Represents the feature of category cl, V(t) i Represents the features of category i.

[0111] In this embodiment, the main function of the above formula is to convert the input feature value V(t) into a probability distribution, exponential operation: for each category feature value V(t) cPerform an exponential operation, which converts all eigenvalues ​​into positive values ​​and increases the influence of larger eigenvalues, making the model more sensitive to high-confidence classifications. Normalization: Normalize the exponential results of all categories, that is, divide the exponential value of each category by the sum of the exponential values ​​of all categories. This ensures that the sum of the output probability values ​​is 1, so that the output of the model can be regarded as a probability distribution.

[0112] The advantages of the above formula are: the nonlinear characteristics enable the model to capture complex decision boundaries; it is suitable for multi-classification tasks, and the most likely category can be easily selected through the output probability distribution.

[0113] Step S3: extracting text information from video frame images labeled as sensitive content and performing sensitivity detection;

[0114] The specific steps of step S3 are:

[0115] Step S301: extracting text information T(t) from the video frame image at time t using an OCR algorithm;

[0116] T(t)=OCR(I(t)), where OCR() represents the OCR algorithm;

[0117] Step S302: Use multimodal embedding to analyze the text information T(t), embed the video frame image and the text information extracted by OCR into a high-dimensional vector, and set the video frame image embedding as I embed (t), calculate the multimodal vector of the text, the specific formula is:

[0118] ,

[0119] Among them, ME(t) represents the multimodal vector of the text in the video frame image at time t, W q Represents the text embedding matrix, which is used to convert text information into word vectors. Indicates a connection operation;

[0120] Step S303: Input the multimodal vector of the text into the neural network for semantic analysis, transfer the embedded features of the sensitive words to the high-dimensional classification space, and obtain the classification probability of the sensitivity through the attention mechanism. The specific formula is:

[0121] ,

[0122] Among them, P class (t) represents the text classification probability vector, that is, the probability that the text belongs to different sensitive categories, W c represents the sensitive classification weight matrix, b c represents the bias vector, and softmax() represents the classification function;

[0123] Using neural networks can enhance the model's ability to parse complex semantics. Combining attention mechanisms or multi-head attention can capture contextual information and give more weight to key words in the sequence.

[0124] Step S304: Perform sensitivity scoring for the classification probability results of each frame of text. The specific formula for sensitivity scoring is:

[0125] ,

[0126] Where S(t) represents the sensitivity score of the text in the A-frame video frame image, A represents the total number of video frame images, and P class (t) a Represents the text classification probability vector of the a-th video frame image, Represents the weight coefficient, which is obtained based on experimental simulation or actual situation;

[0127] The text in the A frame video frame image here is the text information sequence in step S305;

[0128] Step S305: Set the text information sequence to T(1), T(2), ..., T(A), use the sequence model to analyze the text information sequence, perform sensitive content detection in combination with the context of the current frame, and calculate the context score of the text information sequence. The specific formula is:

[0129] ,

[0130] Among them, S yj (t) represents the context score of the text in the video frame image at time t, f() represents the sequence model function, ME(t-1) represents the multimodal vector of the text in the video frame image at time t-1, and ME(t+1) represents the multimodal vector of the text in the video frame image at time t+1;

[0131] The sequence model function f() is used to calculate the contextual association of the current frame and its previous and next frames. This function can be a bidirectional long short-term memory network (BiLSTM) or a multi-head self-attention mechanism, which aims to combine the embedded information of surrounding frames to understand the context of the current frame. For example, some sensitive words may not appear sensitive when they appear alone, but may become sensitive content in a specific context.

[0132] In this embodiment, the core of the formula lies in the context-aware context score calculation. For the text information extracted from the video frame, the context-aware model is used to map it into a high-dimensional vector space to form a multimodal vector. The multimodal vector retains the semantic information of the text, so that the model can understand the meaning of different words in different contexts. The sequence model generates a context score by weighted combination of the embedding vectors of the current frame and adjacent frames. A high score value indicates that the frame is more likely to contain sensitive content in the context.

[0133] Step S306: Calculate the final sensitivity score of the text in the video frame image at time t. The specific formula is:

[0134] ,

[0135] Among them, Final_Score(t) represents the final sensitivity score of the text in the video frame image at time t. and Represents the weight coefficient, which controls the relative influence of the sensitivity score and the context score.

[0136] In this embodiment, the context-aware model can capture the relationship between frames, so that the system not only relies on the detection results of a single frame, but also considers the overall content and contextual relevance of the text. This context-related weighting mechanism effectively improves the detection ability of continuity-sensitive content.

[0137] Step S4: Combining the results of image content recognition and text sensitivity detection, making real-time decisions;

[0138] The specific steps of step S4 are:

[0139] Step S401: Set the image safety threshold to T img , the safety threshold of the text is T txt ;

[0140] Step S402: Determine whether to trigger an alarm by using a decision formula. The specific formula of the decision formula is:

[0141] ,

[0142] in, The decision formula for the video frame image at time t, P(t) mg represents the probability that the video frame image category at time t is sensitive, Indicates that, =1, indicating that the video frame image at time t exceeds the threshold, triggering an alarm. =0, it means that the video frame image at time t does not exceed the threshold.

[0143] Trigger responses based on real-time decision results to ensure that the system can react quickly when sensitive content appears and effectively control the spread of sensitive content.

[0144] Step S5: When illegal content is detected, an alarm message is sent through the background management platform, sensitive videos are immediately blocked, and the next video is tested.

[0145] Step S5 performs the operations of step S2 to step S4 on the next video. If it is determined that the video contains sensitive information, the detection of the next video is continued.

[0146] Figure 3 Note: After the HDMI video signal is sent from the signal source, it enters the LED control device. The LED control device maintains a reliable connection with the unified control system based on the Internet through VPN. The LED control device encodes the HDMI signal into a digital video stream and transmits it to the back-end unified control system in real time when needed according to the configuration. Here, all videos are compressed and transmitted to the unified control system. In the unified control system, each video is screenshoted and identified to determine whether there is sensitive information. The LED control device determines whether to copy the HDMI signal based on real-time warnings and forward it to the LED screen to ensure that the LED screen displays content normally and no sensitive content appears.

[0147] Example 2

[0148] See also Figure 4 , another embodiment provided by the present invention: a monitoring device for public screen content security supervision, comprising: a video processing module, an image recognition and classification module, a text detection module, a real-time decision module and a shielding detection module;

[0149] The video processing module is used to compress and temporarily store the video published on the public screen, and pre-process the compressed and temporarily stored video;

[0150] The image recognition and classification module is used to classify and recognize the preprocessed video and identify different classification labels of the video frame images;

[0151] The text detection module is used to extract text information in video frame images labeled as sensitive content and perform sensitivity detection;

[0152] The real-time decision module is used to combine the results of image content recognition and text sensitivity detection to make real-time decisions;

[0153] The shielding detection module is used to send an alarm message through the background management platform when illegal content is detected, immediately shield the sensitive video, and detect the next video.

[0154] Image recognition and classification module, including: model training optimization unit, feature extraction unit and image recognition and classification unit;

[0155] The model training optimization unit is used to construct an image classification model and perform optimization and training;

[0156] The feature extraction unit is used to extract the features of the video frame image at time t;

[0157] The image recognition and classification unit is used to use the trained image classification model to recognize and classify the video frame image at time t.

[0158] The text detection module includes: a sensitivity scoring unit, a context scoring unit and a sensitivity final scoring unit;

[0159] The sensitivity scoring unit is used to extract the multimodal vector of the text in the video frame image and perform sensitivity scoring;

[0160] The context scoring unit is used to analyze the text information sequence using the sequence model, perform sensitive content detection in combination with the context of the current frame, and calculate the context score of the text information sequence;

[0161] The final sensitivity scoring unit is used to calculate the final sensitivity score of the text in the video frame image at time t.

[0162] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0163] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A monitoring method for public screen content security supervision, characterized in that: include: Compress and temporarily store the video to be published on the public screen, and pre-process the compressed and temporarily stored video; Classify and identify the preprocessed video and identify different classification labels of the video frame images; Extract text information from video frame images labeled as sensitive content and perform sensitivity detection; Combine the results of image content recognition and text sensitivity detection to make real-time decisions; When illegal content is detected, an alarm message is sent through the background management platform, sensitive videos are immediately blocked, and the next video is tested.

2. A monitoring method for public screen content security supervision as claimed in claim 1, characterized in that: The pre-processed video is classified and identified to identify different classification labels of the video frame images, including: Select a machine learning algorithm to build an image classification model, use an annotated image dataset to train the image classification model, set the model training goal to minimize the loss function, and the specific formula for minimizing the loss function is: ， Among them, L represents the minimized loss function of the image classification model, C represents the total number of image categories, and P c represents the actual category of the annotated image, represents the predicted category of the cth labeled image by the image classification model, represents the loss function; The parameters of the image classification model are updated through back propagation to minimize the loss function. The specific formula for the update is: ， in, represents the parameters of the image classification model, represents the learning rate, It represents the partial derivative function; Train the image classification model until the minimization loss function of the model's training objective converges and remains unchanged, then stop training.

3. A monitoring method for public screen content security supervision as claimed in claim 2, characterized in that: The method of classifying and identifying the preprocessed video to identify different classification labels of the video frame images also includes: Set the video frame image at time t as I(t), and use the convolutional neural network to extract features from the video frame image at time t. The specific formula is: ， Among them, F(t) represents the feature map of the video frame image at time t, J n represents the nth layer of convolution operation, H n represents the nth layer of pooling operation, and n represents the total number of layers of the convolutional neural network; The extracted feature map is mapped to the classification space through the fully connected layer. The specific formula is: ， Where V(t) represents the feature map of the video frame image after mapping at time t, W represents the weight matrix of the fully connected layer, and b represents the bias term; The mapped result is transformed nonlinearly through the activation function, input into the trained image classification model, and output the classification of the video frame image at time t. The specific formula is: ， Among them, P(t) cl represents the probability that the video frame image category is cl at time t, e represents the exponential function, V(t) cl Represents the feature of category cl, V(t) i Represents the features of category i.

4. A monitoring method for public screen content security supervision as claimed in claim 1, characterized in that: The extracting of text information from the video frame image labeled as sensitive content and performing sensitivity detection includes: Use the OCR algorithm to extract the text information T(t) in the video frame image at time t; Use multimodal embedding to analyze the text information T(t), embed the video frame image and the text information extracted by OCR into a high-dimensional vector, and set the video frame image embedding as I embed (t), calculate the multimodal vector of the text, the specific formula is: ， Among them, ME(t) represents the multimodal vector of the text in the video frame image at time t, W q Represents the text embedding matrix, which is used to convert text information into word vectors. Indicates a connection operation; The multimodal vector of the text is input into the neural network for semantic analysis, the embedded features of sensitive words are transferred to the high-dimensional classification space, and the classification probability of sensitivity is obtained through the attention mechanism. The specific formula is: ， Among them, P class (t) represents the text classification probability vector, that is, the probability that the text belongs to different sensitive categories, W c represents the sensitive classification weight matrix, b c represents the bias vector, and softmax() represents the classification function; A sensitivity score is calculated for the classification probability result of each frame of text. The specific formula for the sensitivity score is: ， Where S(t) represents the sensitivity score of the text in the A-frame video frame image, A represents the total number of video frame images, and P class (t) a Represents the text classification probability vector of the a-th video frame image, Represents the weight coefficient.

5. A monitoring method for public screen content security supervision as claimed in claim 4, characterized in that: The extracting of text information from the video frame image labeled as sensitive content and performing sensitivity detection also includes: Set the text information sequence to T(1), T(2), ..., T(A), use the sequence model to analyze the text information sequence, combine the context of the current frame to detect sensitive content, and calculate the context score of the text information sequence. The specific formula is: ， Among them, S yj (t) represents the context score of the text in the video frame image at time t, f() represents the sequence model function, ME(t-1) represents the multimodal vector of the text in the video frame image at time t-1, and ME(t+1) represents the multimodal vector of the text in the video frame image at time t+1; Calculate the final sensitivity score of the text in the video frame image at time t. The specific formula is: ， Among them, Final_Score(t) represents the final sensitivity score of the text in the video frame image at time t. and Represents the weight coefficient.

6. A monitoring method for public screen content security supervision as claimed in claim 1, characterized in that: The results of combining image content recognition and text sensitivity detection are used to make real-time decisions, including: Set the image safety threshold to T img , the safety threshold of the text is T txt ; The decision formula is used to determine whether to trigger an alarm. The specific formula of the decision formula is: ， in, The decision formula for the video frame image at time t, P(t) mg represents the probability that the video frame image category at time t is sensitive, Indicates that, =1, indicating that the video frame image at time t exceeds the threshold, triggering an alarm. =0, it means that the video frame image at time t does not exceed the threshold.

7. A monitoring method for public screen content security supervision as claimed in claim 1, characterized in that: The preprocessing includes: frame screenshot and denoising; Frame screenshot, used to convert compressed and temporarily stored video into frames and convert them into video frame image data; De-noising is used to filter noise and adjust brightness of the video frame image data after framing.

8. A monitoring device for public screen content security supervision, used to implement a monitoring method for public screen content security supervision as claimed in any one of claims 1 to 7, characterized in that: include: Video processing module, image recognition and classification module, text detection module, real-time decision module and shielding detection module; The video processing module is used to compress and temporarily store the video published on the public screen, and pre-process the compressed and temporarily stored video; The image recognition and classification module is used to classify and recognize the preprocessed video and identify different classification labels of the video frame images; The text detection module is used to extract text information in video frame images labeled as sensitive content and perform sensitivity detection; The real-time decision module is used to combine the results of image content recognition and text sensitivity detection to make real-time decisions; The shielding detection module is used to send an alarm message through the background management platform when illegal content is detected, immediately shield the sensitive video, and detect the next video.

9. A monitoring device for public screen content security supervision as claimed in claim 8, characterized in that: The image recognition and classification module includes: a model training optimization unit, a feature extraction unit and an image recognition and classification unit; The model training optimization unit is used to construct an image classification model and perform optimization and training; The feature extraction unit is used to extract the features of the video frame image at time t; The image recognition and classification unit is used to use the trained image classification model to recognize and classify the video frame image at time t.

10. A monitoring device for public screen content security supervision as claimed in claim 9, characterized in that: The text detection module includes: a sensitivity scoring unit, a context scoring unit and a sensitivity final scoring unit; The sensitivity scoring unit is used to extract the multimodal vector of the text in the video frame image and perform sensitivity scoring; The context scoring unit is used to analyze the text information sequence using the sequence model, perform sensitive content detection in combination with the context of the current frame, and calculate the context score of the text information sequence; The final sensitivity scoring unit is used to calculate the final sensitivity score of the text in the video frame image at time t.

Images