A real-time video sensitive content recognition method and system

Abstract

The present invention relates to the field of video content processing technology, and specifically to a real-time video sensitive content identification method and system, comprising the following steps: S1: receiving real-time video streams from multiple video sources and splitting them according to frames; S2: performing language recognition on the split audio data in S1 and generating corresponding text data; S3: identifying potential sensitive content and labeling it according to the type of sensitive content; S4: updating the sensitive content definition library in real time according to the latest laws and regulations, social standards and user-defined rules to dynamically adjust the judgment criteria of sensitive content; S5: classifying the sensitive content preliminarily identified in S3 according to the sensitive content definition library updated in real time in S4. The present invention, by adopting deep learning technology and real-time update mechanism, can effectively and quickly identify and process sensitive content in multi-source video streams, ensuring the accuracy of content monitoring and compliance with the latest laws and regulations.

Description

Technical Field

[0001] The present invention relates to the technical field of video content processing, and in particular to a real-time video sensitive content recognition method and system. Background Art

[0002] With the rapid growth of Internet video content, real-time video streaming has become an indispensable part of daily life. A large number of live broadcasts, short videos, online conferences and other content are constantly emerging, bringing users a rich visual experience. However, potential bad information and sensitive content in video content, such as violence, indecent behavior and display of illegal items, are also spreading rapidly and may cause serious social problems. In order to ensure the content security of the platform, government departments and content providers have put forward higher requirements for real-time detection of sensitive content. Traditional manual review methods are not only inefficient and unable to process large amounts of video data in a timely manner, but also prone to subjective misjudgment.

[0003] In the prior art, although there are some methods for identifying sensitive video content, they generally have several difficulties. First, the recognition technology for multiple languages ​​and dialects is not yet perfect, especially when the audio and video content involves complex language environments, the existing multi-language processing models are difficult to deal with effectively. Second, the current sensitive content judgment criteria are usually fixed and single, and cannot be dynamically adjusted according to the latest laws and regulations, social standards, and user-defined rules, resulting in the recognition results being unable to meet the latest requirements in a timely manner. Therefore, it is particularly necessary to develop a real-time video sensitive content recognition method and system. Summary of the invention

[0004] Based on the above objectives, the present invention provides a real-time video sensitive content identification method and system.

[0005] A real-time video sensitive content recognition method comprises the following steps:

[0006] S1: receiving real-time video streams from multiple video sources, and splitting the received real-time video data into frames to obtain synchronized audio data and video image frames;

[0007] S2: Perform language recognition on the audio data split in S1, and use a pre-trained multi-language recognition model to detect the language type in the video; if a dialect or non-mainstream language is detected, call the corresponding dialect recognition model to perform semantic analysis on the audio and generate corresponding text data;

[0008] S3: extract image features from the video image frames split in S1, preliminarily identify potential sensitive content based on text data and image features, and label them according to the sensitive content type;

[0009] S4: By connecting to the legal and regulatory database API, the sensitive content definition library is updated in real time according to the latest laws and regulations, social standards and user-defined rules to dynamically adjust the criteria for determining sensitive content;

[0010] S5: Classify the sensitive content initially identified in S3 according to the sensitive content definition library updated in real time in S4, and divide the classified sensitive content into preset levels, and then trigger corresponding processing strategies according to different levels.

[0011] Optionally, the S1 specifically includes:

[0012] S11: receiving real-time video streams from multiple video sources, the video sources including a webcam, a mobile device camera, and a pre-recorded video file stored locally, while establishing a stable network connection using a streaming media transmission protocol or a real-time messaging transmission protocol;

[0013] S12: for each received video stream, parse the video stream using a frame extraction algorithm, and split the continuous real-time video data into independent video frame sequences according to fixed time intervals;

[0014] S13: extracting audio data corresponding to each frame from the video stream, using audio and video synchronization technology in the process to ensure that the timestamps of the audio data and the video image frame are consistent;

[0015] S14: Synchronously package the audio data and video image frames of each video source according to the timestamps to generate a data stream that can be processed in parallel.

[0016] Optionally, the S2 specifically includes:

[0017] S21: preprocessing the audio data split in S1, specifically including audio data denoising and volume normalization;

[0018] S22: inputting the preprocessed audio data into a pre-trained multi-language recognition model, wherein the multi-language recognition model is trained based on a deep learning Transformer architecture and is used to extract relevant language type information by analyzing speech features in the audio signal, determine the language type in the audio, and output a corresponding language label;

[0019] S23: When a non-mainstream language or dialect is detected, a dedicated recognition model trained for the corresponding language or dialect is automatically called, and the dialect recognition model is also trained based on the Transformer architecture of deep learning;

[0020] S24: Perform semantic analysis on the audio data through the dialect recognition model, identify the semantic information in the audio, and convert the audio signal into structured text data.

[0021] Optionally, the S22 specifically includes:

[0022] S221: Preprocess the input audio data and convert it into a feature vector sequence. The audio signal is divided into several frames of fixed length, where each frame is represented by Mel-frequency cepstral coefficients to generate a feature vector: in, is the feature vector sequence of the entire audio, For frame The feature vector of the audio, is the total number of audio frames;

[0023] S222: Receive feature vector sequence , and processed by a multi-head self-attention mechanism to capture the long-range dependencies between audio frames;

[0024] S223: The audio feature vector sequence processed by the self-attention mechanism is nonlinearly mapped through a feedforward neural network;

[0025] S224: Training is performed using the cross entropy loss function to minimize the difference between the predicted label and the true label during the optimization process;

[0026] S225: After training is completed, the multi-language recognition model receives the audio feature vector sequence in the inference phase , and through the above-mentioned multi-head self-attention mechanism and feedforward neural network processing, based on the temporal characteristics and contextual information of the input data, generate the predicted probability for each language category ; Finally, the language tag is selected based on the maximum probability.

[0027] Optionally, the S3 specifically includes:

[0028] S31: preprocessing each frame of the video image split in S1, specifically including image denoising, brightness adjustment and size standardization, to ensure the clarity and consistency of the image data;

[0029] S32: inputting the preprocessed image frame into a pre-trained convolutional neural network model to extract multi-level features in the image, specifically including feature information of edges, textures, and object shapes;

[0030] S33: After extracting the image features, preliminarily identifying potential sensitive content in the image through a pre-trained convolutional neural network model, wherein the sensitive content includes violence, indecent behavior, and dangerous items;

[0031] S34: Label each frame of the image according to the sensitive content type identified in S33, and the type of label is determined by the sensitive content type, including violence, inappropriate speech, and display of illegal items.

[0032] Optionally, the S32 specifically includes:

[0033] S321: Input the preprocessed image frame into the first layer of the convolutional neural network model, and extract low-level features of the image, including edges and simple shapes, through convolution operations;

[0034] S322: The output of the convolutional layer is processed nonlinearly through the ReLU activation function;

[0035] S323: The output of the convolutional layer passes through the pooling layer, using the maximum pooling operation to compress the size of the feature map while retaining the most important features;

[0036] S324: After multiple layers of convolution and pooling operations, high-level features are extracted, including complex textures and object shapes;

[0037] S325: The output layer of the convolutional neural network will integrate all the extracted image features to generate a multi-level feature vector.

[0038] Optionally, the S33 specifically includes:

[0039] S331: Input the multi-level features of the image extracted in S32 into the classification layer of the pre-trained convolutional neural network model. The classification layer uses a fully connected layer to convert the feature vector extracted by the convolution layer into a vector of fixed length. The calculation formula of the fully connected layer is: ,in, For the classifier The score of the class output, is the weight vector, is the image feature vector, is the bias term;

[0040] S332: The classification layer performs preliminary classification based on the image feature vector to identify potential sensitive content in the image. The specific classification process uses the Softmax function to calculate the probability value of each type of sensitive content. The specific calculation formula is: ,in, Indicates that the image belongs to The probability of sensitive content. is the total number of all categories, For the classifier The score of the class output;

[0041] S333: For each input image frame, calculate the probability values ​​of all categories, and select the category with the highest probability as the preliminary recognition result;

[0042] S334: For each sensitive content, if its probability value is higher than a preset threshold, it is marked as potential sensitive content.

[0043] Optionally, the S4 specifically includes:

[0044] S41: Establish connections with online regulatory databases, social standards libraries, and user-defined rule libraries through APIs to obtain the latest laws, regulations, social standards, and user-defined rules on a regular or real-time basis;

[0045] S42: Compare the items obtained from the outside with the existing sensitive content definition library, and automatically merge or replace the inconsistent content. The user-defined rules have the highest priority and can be used to override the standard rules.

[0046] S43: Automatically adjust the criteria for determining sensitive content based on the updated definition library, including thresholds and category classification rules, based on the weight of the sensitive content category. and identification threshold To adjust the judgment criteria, the formula is: ,in, For Category Sensitive content rating, is the category weight, Features The function representation is: is the total number of features; sensitive content determination is based on the score Whether it exceeds the threshold Make judgments;

[0047] S44: When determining the sensitive content of each frame of the image, use the determination criteria updated in real time to perform the determination, and map each image frame to a corresponding sensitive category in the definition library.

[0048] Optionally, the S5 specifically includes:

[0049] S51: Match the sensitive content preliminarily identified in S3 with the entries in the sensitive content definition library updated in real time in S4, and map the preliminarily identified content to the corresponding sensitive category;

[0050] S52: Calculate the rating of each sensitive content using the following formula: ,in, For Category Sensitive content rating, For Category The weight of the sensitive content score, For Category Sensitive content rating of

[0051] S53: Rating based on the level of each sensitive content The scores are divided into three levels: high, medium and low. The specific classification criteria are determined by the preset scoring threshold. When When When the content is classified as low-level sensitive content, and The preset scoring thresholds for high and medium sensitive content;

[0052] S54: According to the level of sensitive content, the corresponding processing strategy is automatically triggered. Specifically, for high-level sensitive content, immediate processing measures such as blocking, alarming or reporting to management personnel will be taken; for medium-level sensitive content, a warning will be issued, and automatic blocking or delayed playback will be carried out according to user settings.

[0053] A real-time video sensitive content recognition system, used to implement the above-mentioned real-time video sensitive content recognition method, includes the following modules:

[0054] Video stream receiving module: used to receive real-time video streams from multiple video sources, and split the received video streams into frames to obtain synchronized audio data and video image frames;

[0055] Audio processing module: used to pre-process the split audio data, including denoising and volume normalization, and input the processed audio data into the multi-language recognition module for language recognition;

[0056] Multi-language recognition module: connected to the audio processing module, using a pre-trained multi-language recognition model to detect the language type in the audio; when a non-mainstream language or dialect is detected, the dialect recognition model is called to perform semantic analysis and generate corresponding text data;

[0057] Image processing module: connected to the video stream receiving module, used to pre-process the split video image frames, including image denoising, brightness adjustment and size standardization; and input the pre-processed image frames into the pre-trained convolutional neural network model to extract multi-level features in the image, including feature information of edges, textures and object shapes;

[0058] Sensitive content identification module: used to preliminarily identify potential sensitive content based on multi-level feature information in text data and images, such as violence, indecent behavior and dangerous items, and to label each frame of the image with a corresponding sensitive content label;

[0059] Sensitive content definition library update module: through real-time synchronization with the online regulatory database, social standard library and user-defined rule library, it obtains the latest laws, regulations, social standards and user rules, updates the sensitive content definition library in the system, and dynamically adjusts the judgment criteria for sensitive content;

[0060] Classification and processing module: used to classify the initially identified sensitive content according to the real-time updated sensitive content definition library, and divide it into three levels: high, medium and low according to the severity of the sensitive content; according to different levels, corresponding processing strategies will be triggered, including blocking, delayed playback or alarming the management personnel.

[0061] Beneficial effects of the present invention:

[0062] The present invention, by adopting advanced deep learning technologies, including convolutional neural networks and multilingual recognition models based on Transformer architecture, can efficiently identify and process sensitive information in video content. These technologies can accurately identify voice content in multiple language environments and effectively extract key features in video images, such as edges, textures, and object shapes. In addition, through real-time analysis of deep learning models, potential sensitive content can be quickly identified and classified, significantly improving the speed and accuracy of sensitive content processing.

[0063] The present invention, by introducing a dynamic update mechanism, can adjust the definition library and judgment criteria of sensitive content in real time according to the latest laws and regulations, social standards and user-defined rules. This mechanism ensures the timeliness and compliance of the identification process, making the identification of sensitive content not only fast but also in line with current legal and social requirements. Through this dynamic adjustment, corresponding processing strategies can be automatically triggered according to the severity of sensitive content, such as blocking, delayed playback or alarm, which effectively reduces the burden of manual review and improves the efficiency and automation of content supervision. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0065] Figure 1 A schematic diagram of a process flow of a method for identifying sensitive content in a real-time video according to an embodiment of the present invention;

[0066] Figure 2 Schematic diagram of a real-time video sensitive content recognition system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.

[0068] It should be noted that the references to "one embodiment", "embodiment", "exemplary embodiments", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0069] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0070] like Figure 1 As shown, a real-time video sensitive content identification method includes the following steps:

[0071] S1: receiving real-time video streams from multiple video sources, and splitting the received real-time video data into frames to obtain synchronized audio data and video image frames;

[0072] S2: Perform language recognition on the audio data split in S1, and use a pre-trained multi-language recognition model to detect the language type in the video; if a dialect or non-mainstream language is detected, call the corresponding dialect recognition model to perform semantic analysis on the audio and generate corresponding text data;

[0073] S3: extract image features from the video image frames split in S1, preliminarily identify potential sensitive content based on text data and image features, and label them according to the sensitive content type;

[0074] S4: By connecting to the legal and regulatory database API, the sensitive content definition library is updated in real time according to the latest laws and regulations, social standards and user-defined rules to dynamically adjust the criteria for determining sensitive content;

[0075] S5: Classify the sensitive content initially identified in S3 according to the sensitive content definition library updated in real time in S4, and divide the classified sensitive content into preset levels, and then trigger corresponding processing strategies according to different levels.

[0076] The real-time video data in S1 is split into frames, including:

[0077] S11: Receive live video streams from multiple video sources, including webcams, mobile device cameras, and pre-recorded video files stored locally, while using RTSP or RTMP to establish a stable network connection to ensure continuous transmission of video stream data;

[0078] S12: for each received video stream, the video stream is parsed using a frame extraction algorithm to split the continuous real-time video data into independent video frame sequences at fixed time intervals. The frame extraction algorithm is based on an image coding structure, such as H.264 or H.265 format, to ensure efficient splitting of video frames.

[0079] S13: extracting audio data corresponding to each frame from the video stream, using audio and video synchronization technology (AVSync) in the process to ensure that the timestamps of the audio data and the video image frame are consistent, and to ensure synchronous processing of the audio and video frames; the synchronization formula is: ,in, For the Frame video image frame timestamp, For the corresponding The timestamp of the audio data corresponding to the frame; audio and video synchronization technology adjusts the playback speed of the audio or video stream by comparing the timestamps of the video frame and the audio to ensure the consistency of the timestamps of the two;

[0080] S14: Synchronously package the audio data and video image frames of each video source according to the timestamps to generate a data stream that can be processed in parallel; specifically, the following synchronous packaging formula is used:

[0081] ,in, For the Data packets, is the video image frame of frame , is the corresponding audio data, The synchronous packaging processing combines each video frame and the corresponding audio frame at the same time to generate a data stream that can be processed in parallel for subsequent multimodal analysis. The above-mentioned audio and video synchronization technology and synchronous packaging processing ensure the strict temporal correspondence between audio and video frame data, avoiding the problem of audio and video asynchrony. The synchronous packaging mechanism based on timestamp ensures efficient parallel processing of audio and video data streams, thereby improving the processing capability and real-time performance of the system.

[0082] S2 specifically includes:

[0083] S21: preprocessing the audio data split in S1, including audio data denoising and volume normalization, improving the quality of the audio data, and ensuring that the audio data input to the multi-language recognition model has high clarity and consistency;

[0084] S22: input the preprocessed audio data into a pre-trained multi-language recognition model. The multi-language recognition model is trained based on the Transformer architecture of deep learning and has the ability to recognize multiple languages. The model is used to extract relevant language type information by analyzing the speech features in the audio signal, determine the language type in the audio, and output the corresponding language label.

[0085] S23: When a non-mainstream language or dialect is detected, a dedicated recognition model trained for the corresponding language or dialect is automatically called. The dialect recognition model is also trained based on the Transformer architecture of deep learning to adapt to the speech characteristics of a specific dialect;

[0086] S24: The audio data is semantically parsed through the dialect recognition model to identify the semantic information in the audio and convert the audio signal into structured text data. During the parsing process, the model combines the specific pronunciation habits and grammatical rules of the audio to generate accurate text data corresponding to the video content; finally, the generated text data is annotated as a specific language type, packaged together with the corresponding timestamp information, and transmitted to the subsequent multimodal content recognition module for further processing; by introducing multilingual and dialect recognition models, it can efficiently identify multiple language types in the video, accurately parse the semantics of dialects or non-mainstream languages, and generate reliable text data. This solution improves the system's ability to handle complex language environments and ensures the accuracy and comprehensiveness of language recognition.

[0087] The construction of a multilingual recognition model in S22 specifically includes:

[0088] S221: Preprocess the input audio data and convert it into a feature vector sequence. The audio signal is divided into several frames of fixed length, where each frame is represented by Mel Frequency Cepstral Coefficient (MFCC) to generate a feature vector: in, is the feature vector sequence of the entire audio, For the The feature vector of the frame audio, is the total number of audio frames;

[0089] S222: The encoder of the Transformer model receives a sequence of feature vectors , and is processed by a multi-head self-attention mechanism to capture the long-distance dependencies between audio frames; the calculation formula of the self-attention mechanism is: ,in, is the query matrix, is the key matrix, is the value matrix, is the dimension of the key matrix, represents the product of the query and key matrices, The function is used to normalize the attention score, which is a mechanism used to analyze temporal relationships in audio data;

[0090] S223: The audio feature vector sequence processed by the self-attention mechanism is nonlinearly mapped through a feedforward neural network (FFN). The calculation formula of the feedforward neural network is:

[0091] ,in, is the input feature vector of the frame, and are the weight matrices of the network, and is the bias term, It is the ReLU activation function to ensure nonlinear feature expression;

[0092] S224: Training is performed using the cross entropy loss function to minimize the difference between the predicted label and the true label during the optimization process. The cross entropy loss function is defined as: ,in, is the loss value, is the total number of language categories, is the true label of the j-th language, is the probability value of the j-th language predicted by the model;

[0093] S225: After training is completed, the multi-language recognition model receives the audio feature vector sequence in the inference phase , and through the above-mentioned multi-head self-attention mechanism and feedforward neural network processing, based on the temporal characteristics and contextual information of the input data, generate the predicted probability for each language category ; Finally, the language tag is selected based on the maximum probability. The formula for the language tag is: in, is the language tag of the final output, For the model category The predicted probability of It is an index of language categories. Through the multi-language recognition model based on the Transformer architecture, the self-attention mechanism is used to effectively capture the long-distance dependencies between audio frames. The feedforward neural network is combined to enhance the feature expression ability and realize accurate judgment of the language type in the audio signal. The optimization of the cross entropy loss function ensures the high accuracy of the model, and finally generates reliable language labels for subsequent processing.

[0094] S3 specifically includes:

[0095] S31: preprocessing each frame of the video image split in S1, including image denoising, brightness adjustment and size standardization, to ensure the clarity and consistency of the image data, so as to improve the effect of subsequent image feature extraction;

[0096] S32: Input the preprocessed image frame into a pre-trained convolutional neural network (CNN) model, which has been trained using a large-scale image dataset and can automatically extract multi-level features in the image, including feature information of edges, textures, and object shapes;

[0097] S33: After extracting image features, the pre-trained convolutional neural network model is used to preliminarily identify potential sensitive content in the image, including violence, indecent behavior, and dangerous items;

[0098] S34: Label each frame of the image according to the type of sensitive content identified in S33. The type of label is determined by the type of sensitive content, including violence, inappropriate speech, and display of illegal items. Labeling is performed based on specific rules to ensure that the sensitive content of each frame of the image is accurately labeled. Finally, the image frame with the sensitive content label is recorded together with its timestamp for further processing and classification operations in subsequent steps. The label is not only used to mark specific sensitive content, but also provides a basis for subsequent sensitive content classification and processing.

[0099] S32 specifically includes:

[0100] S321: The preprocessed image frame is input into the first layer of the convolutional neural network (CNN) model, which has been pre-trained with a large-scale image dataset. The low-level features of the image, including edges and simple shapes, are first extracted through a convolution operation; the calculation formula of the convolution operation is: ,in, For the The convolution kernels are located at The output at For the The weight matrix of the convolution kernel, is the input image frame, It is the bias of the convolution kernel. The convolution kernel extracts edges and local features by performing dot product with the local area of ​​the input image.

[0101] S322: The output of the convolutional layer is processed nonlinearly through the ReLU (rectified linear unit) activation function. The activation function is used to increase the nonlinear expression ability of the model. Its calculation formula is: ,in, is the output after being processed by the activation function; the ReLU function converts negative values ​​to 0 and keeps positive values ​​unchanged, thereby enhancing the model's capture of low-level features such as edges;

[0102] S323: The output of the convolutional layer passes through the pooling layer, which uses the maximum pooling operation to compress the size of the feature map while retaining the most important features; the formula for the maximum pooling operation is: in, For the pooling layer at position The output at Represents the pooled area. The pooling layer reduces the dimension of the feature map and retains key information in the image, including edges and textures;

[0103] S324: After multiple layers of convolution and pooling operations, high-level features are extracted, including complex textures and object shapes. As the network depth increases, the convolution layer can capture more complex visual patterns and generate high-level feature representations, which are used to represent the shape, contour, and texture of the object.

[0104] S325: The output layer of the convolutional neural network will integrate all extracted image features to generate a multi-level feature vector for use in the subsequent sensitive content identification step; through multi-layer convolution operations, ReLU activation and maximum pooling, the convolutional neural network model can effectively extract multi-level features such as edges, textures and object shapes in the image. The convolution kernel combines local feature extraction with global pooling operations to ensure the accurate extraction and expression of image features, providing a rich visual feature foundation for subsequent sensitive content identification.

[0105] S33 specifically includes:

[0106] S331: The multi-level features of the image extracted in S32 are input into the classification layer of the pre-trained convolutional neural network (CNN) model. The classification layer uses a fully connected layer to convert the feature vector extracted by the convolution layer into a vector of fixed length for subsequent classification operations. The calculation formula of the fully connected layer is: ,in, For the classifier The score of the class output, is the weight vector, is the image feature vector, is the bias term;

[0107] S332: The classification layer performs preliminary classification based on the image feature vector to identify potential sensitive content in the image. The specific classification process uses the Softmax function to calculate the probability value of each type of sensitive content (including violence, indecent behavior and dangerous items). The specific calculation formula is: ,in, Indicates that the image belongs to The probability of sensitive content. is the total number of all categories, For the classifier The score of the class output;

[0108] S333: For each input image frame, the probability values ​​of all categories are calculated, and the category with the highest probability is selected as the preliminary recognition result. The model judges the potential sensitive content such as violence, indecent behavior or dangerous items in the image based on the feature patterns learned during the training process;

[0109] S334: For each sensitive content, if its probability value is higher than the preset threshold, it is marked as potential sensitive content; let the model output category label , then the formula is: in, is the sensitive content category label with the largest probability value, representing the final preliminary identification result; by using the pre-trained convolutional neural network and Softmax classifier, this method can preliminarily identify sensitive content such as violence, indecent behavior and dangerous items in the image. The multi-level features extracted by the convolutional layer effectively improve the classification accuracy, and the Softmax function ensures the accurate probability distribution of different categories of sensitive content, so that the system can identify potential sensitive content in a timely and accurate manner.

[0110] S4 specifically includes:

[0111] S41: The system establishes a connection with the online regulatory database, social standards library and user-defined rule library through API (application programming interface), and obtains the latest laws and regulations, social standards and user-defined rules regularly or in real time; at the same time, data synchronization is performed, and the updated content is determined by the following synchronization formula: in, Indicates the updated sensitive content definition library. is the entry in the currently defined library, New or modified entries obtained from the regulatory database;

[0112] S42: The system compares the entries obtained from the outside with the existing sensitive content definition library and automatically merges or replaces the inconsistent content. The updated definition library is described by the following formula: ,in, Define a library for the resulting sensitive content, User-defined rules have the highest priority and can be used to override standard rules.

[0113] S43: Automatically adjust the criteria for determining sensitive content based on the updated definition library, including thresholds and category classification rules, based on the weight of the sensitive content category. and identification threshold To adjust the judgment criteria, the formula is: ,in, For Category Sensitive content rating, is the category weight, Features The function representation is: is the total number of features; sensitive content determination is based on the score Whether it exceeds the threshold Make judgments;

[0114] S44: When determining the sensitive content of each frame of the image, use the judgment criteria updated in real time to perform the judgment, and combine the laws, regulations, social standards and user-defined rules to map each image frame to the corresponding sensitive category in the definition library; by using the scoring formula and threshold judgment of sensitive content, the judgment criteria of sensitive content can be flexibly adjusted in real time according to the updated laws, regulations, social standards and user-defined rules to ensure compliance with current social norms and user needs. This method not only ensures the compliance of the system, but also enhances the user's customization flexibility.

[0115] S5 specifically includes:

[0116] S51: Match the sensitive content preliminarily identified in S3 with the entries in the sensitive content definition library updated in real time in S4, and classify the sensitive content based on the latest laws and regulations, social standards and user-defined rules. The classification process uses the classification algorithm to map the preliminarily identified content to the corresponding sensitive category based on the label of each sensitive content in the definition library and its corresponding category rules;

[0117] S52: Calculate the rating of each sensitive content using the following formula: ,in, For Category Sensitive content rating, For Category The weight of the sensitive content score, For Category The classification score is calculated by the aforementioned sensitive content scoring formula, and the weight is dynamically adjusted according to laws, social standards and user rules;

[0118] S53: Rating based on the level of each sensitive content The system divides the content into three levels: high, medium, and low. The specific classification criteria are determined by the preset scoring threshold. The system scores each sensitive content. Compare with the preset threshold to determine the risk level of the sensitive content; When the content is classified as high-level sensitive content, such content usually includes serious illegal or socially unacceptable behavior, such as violence, illegal activities, etc., and emergency measures must be taken immediately; when When the system classifies the content as medium-level sensitive content, such content may involve minor violations, such as indecent actions or remarks, and the handling measures are relatively mild, such as delayed playback or warning prompts; when When the content is classified as low-level sensitive content, such content is generally regarded as minor violations or marginal content, and usually only recorded or user prompts are issued; and Preset scoring thresholds for high and medium sensitive content to ensure that the system can accurately classify different types of content according to their severity;

[0119] S54: According to the level of sensitive content, the corresponding processing strategy is automatically triggered. Specifically, for high-level sensitive content (such as violence or illegal activities), immediate processing measures such as blocking, alarming or reporting to management personnel will be taken; for medium-level sensitive content (such as indecent behavior), a warning will be issued, and automatic blocking or delayed playback will be based on user settings; for low-level sensitive content (such as minor violations), it will be recorded or a prompt message will be sent to the user; by classifying according to the sensitive content definition library and grading the content according to the preset grading standards, the system can flexibly respond to different types of sensitive content. Combined with the automatically triggered processing strategy, the system can take appropriate response measures according to the severity of the sensitive content to ensure the security and compliance of the video content.

[0120] like Figure 2 As shown, a real-time video sensitive content recognition system is used to implement the above-mentioned real-time video sensitive content recognition method, including the following modules:

[0121] Video stream receiving module: used to receive real-time video streams from multiple video sources and split the received video streams into frames to obtain synchronized audio data and video image frames. Video sources include network cameras, mobile device cameras, and pre-recorded video files stored locally.

[0122] Audio processing module: used to pre-process the split audio data, including denoising and volume normalization, and input the processed audio data into the multi-language recognition module for language recognition;

[0123] Multi-language recognition module: connected to the audio processing module, using a pre-trained multi-language recognition model to detect the language type in the audio; when a non-mainstream language or dialect is detected, the dialect recognition model is called to perform semantic analysis and generate corresponding text data;

[0124] Image processing module: connected to the video stream receiving module, used to pre-process the split video image frames, including image denoising, brightness adjustment and size standardization; and input the pre-processed image frames into the pre-trained convolutional neural network model to extract multi-level features in the image, including feature information of edges, textures and object shapes;

[0125] Sensitive content identification module: used to preliminarily identify potential sensitive content based on multi-level feature information in text data and images, including violence, indecent behavior and dangerous items, and label each frame of the image with a corresponding sensitive content label;

[0126] Sensitive content definition library update module: through real-time synchronization with the online regulatory database, social standard library and user-defined rule library, it obtains the latest laws, regulations, social standards and user rules, updates the sensitive content definition library in the system, and dynamically adjusts the judgment criteria for sensitive content;

[0127] Classification and processing module: used to classify the initially identified sensitive content according to the real-time updated sensitive content definition library, and divide it into three levels: high, medium and low according to the severity of the sensitive content; according to different levels, the system will trigger corresponding processing strategies, including blocking, delaying playback or alerting management personnel.

[0128] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0129] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A real-time video sensitive content identification method, characterized in that: The following steps are involved: S1: receiving real-time video streams from multiple video sources, and splitting the received real-time video data into frames to obtain synchronized audio data and video image frames; S2: Perform language recognition on the audio data split in S1, and use a pre-trained multi-language recognition model to detect the language type in the video; if a dialect or non-mainstream language is detected, call the corresponding dialect recognition model to perform semantic analysis on the audio and generate corresponding text data; S3: extract image features from the video image frames split in S1, preliminarily identify potential sensitive content based on text data and image features, and label them according to the sensitive content type; S4: By connecting to the legal and regulatory database API, the sensitive content definition library is updated in real time according to the latest laws and regulations, social standards and user-defined rules to dynamically adjust the criteria for determining sensitive content; The S4 specifically includes: S41: Establish connections with online regulatory databases, social standards libraries, and user-defined rule libraries through APIs to obtain the latest laws, regulations, social standards, and user-defined rules in real time; S42: Compare the items obtained from the outside with the existing sensitive content definition library, and automatically merge or replace the inconsistent content. The user-defined rules have the highest priority and can be used to override the standard rules. S43: Automatically adjust the criteria for determining sensitive content based on the updated definition library, including thresholds and category classification rules, based on the weight of the sensitive content category. and identification threshold To adjust the judgment criteria, the formula is: ,in, For Category Sensitive content rating, is the category weight, Features The function representation is: is the total number of features; sensitive content determination is based on the score Whether it exceeds the threshold Make judgments; S44: when determining the sensitive content of each frame of the image, use the determination criteria updated in real time to perform determination, and map each image frame to a corresponding sensitive category in the definition library; S5: Classify the sensitive content initially identified in S3 according to the sensitive content definition library updated in real time in S4, and divide the classified sensitive content into preset levels, and then trigger corresponding processing strategies according to different levels; The S5 specifically includes: S51: Match the sensitive content preliminarily identified in S3 with the entries in the sensitive content definition library updated in real time in S4, and map the preliminarily identified content to the corresponding sensitive category; S52: Calculate the rating of each sensitive content using the following formula: ,in, For Category Sensitive content rating, For Category The weight of the sensitive content score, For Category Sensitive content rating of S53: Rating based on the level of each sensitive content The scores are divided into three levels: high, medium and low. The specific classification criteria are determined by the preset scoring threshold. When When When the content is classified as low-level sensitive content, and The preset scoring thresholds for high and medium sensitive content; S54: According to the level of sensitive content, the corresponding processing strategy is automatically triggered. Specifically, for high-level sensitive content, immediate processing measures such as blocking, alarming or reporting to management personnel will be taken; for medium-level sensitive content, a warning will be issued, and automatic blocking or delayed playback will be carried out according to user settings.

2. A real-time video sensitive content recognition method according to claim 1, characterized in that: The S1 specifically includes: S11: receiving real-time video streams from multiple video sources, the video sources including a webcam, a mobile device camera, and a pre-recorded video file stored locally, while establishing a stable network connection using a streaming media transmission protocol or a real-time messaging transmission protocol; S12: for each received video stream, parse the video stream using a frame extraction algorithm, and split the continuous real-time video data into independent video frame sequences according to fixed time intervals; S13: extracting audio data corresponding to each frame from the video stream, using audio and video synchronization technology in the process to ensure that the timestamps of the audio data and the video image frame are consistent; S14: Synchronously package the audio data and video image frames of each video source according to the timestamps to generate a data stream that can be processed in parallel.

3. A real-time video sensitive content recognition method according to claim 1, characterized in that: The S2 specifically includes: S21: preprocessing the audio data split in S1, specifically including audio data denoising and volume normalization; S22: inputting the preprocessed audio data into a pre-trained multi-language recognition model, wherein the multi-language recognition model is trained based on a deep learning Transformer architecture and is used to extract relevant language type information by analyzing speech features in the audio signal, determine the language type in the audio, and output a corresponding language label; S23: When a non-mainstream language or dialect is detected, a dedicated recognition model trained for the corresponding language or dialect is automatically called, and the dialect recognition model is also trained based on the Transformer architecture of deep learning; S24: Perform semantic analysis on the audio data through the dialect recognition model, identify the semantic information in the audio, and convert the audio signal into structured text data.

4. A real-time video sensitive content recognition method according to claim 3, characterized in that: The S22 specifically includes: S221: Preprocess the input audio data and convert it into a feature vector sequence. The audio signal is divided into several frames of fixed length, where each frame is represented by Mel-frequency cepstral coefficients to generate a feature vector: in, is the feature vector sequence of the entire audio, For the The feature vector of the frame audio, is the total number of audio frames; S222: Receive feature vector sequence , and processed by a multi-head self-attention mechanism to capture the long-range dependencies between audio frames; S223: The audio feature vector sequence processed by the self-attention mechanism is nonlinearly mapped through a feedforward neural network; S224: Training is performed using the cross entropy loss function to minimize the difference between the predicted label and the true label during the optimization process; S225: After training is completed, the multi-language recognition model receives the audio feature vector sequence in the inference phase , and through the above-mentioned multi-head self-attention mechanism and feedforward neural network processing, based on the temporal characteristics and contextual information of the input data, generate the predicted probability for each language category ; Finally, the language tag is selected based on the maximum probability.

5. A real-time video sensitive content recognition method according to claim 1, characterized in that: The S3 specifically includes: S31: preprocessing each frame of the video image split in S1, specifically including image denoising, brightness adjustment and size standardization, to ensure the clarity and consistency of the image data; S32: inputting the preprocessed image frame into a pretrained convolutional neural network model to extract multi-level features in the image, specifically including feature information of edges, textures, and object shapes; S33: After extracting the image features, preliminarily identifying potential sensitive content in the image through a pre-trained convolutional neural network model, wherein the sensitive content includes violence, indecent behavior, and dangerous items; S34: Label each frame of the image according to the sensitive content type identified in S33, and the type of label is determined by the sensitive content type, including violence, inappropriate speech, and display of illegal items.

6. A real-time video sensitive content recognition method according to claim 5, characterized in that: The S32 specifically includes: S321: Input the preprocessed image frame into the first layer of the convolutional neural network model, and extract low-level features of the image, including edges and simple shapes, through convolution operations; S322: The output of the convolutional layer is processed nonlinearly through the ReLU activation function; S323: The output of the convolutional layer passes through the pooling layer, using the maximum pooling operation to compress the size of the feature map while retaining the most important features; S324: After multiple layers of convolution and pooling operations, high-level features are extracted, including complex textures and object shapes; S325: The output layer of the convolutional neural network will integrate all the extracted image features to generate a multi-level feature vector.

7. A real-time video sensitive content identification method according to claim 6, characterized in that: The S33 specifically includes: S331: Input the multi-level features of the image extracted in S32 into the classification layer of the pre-trained convolutional neural network model. The classification layer uses a fully connected layer to convert the feature vector extracted by the convolution layer into a vector of fixed length. The calculation formula of the fully connected layer is: ,in, For the classifier The score of the class output, is the weight vector, is the image feature vector, is the bias term; S332: The classification layer performs preliminary classification based on the image feature vector to identify potential sensitive content in the image. The specific classification process uses the Softmax function to calculate the probability value of each type of sensitive content. The specific calculation formula is: ,in, Indicates that the image belongs to The probability of sensitive content. is the total number of all categories, For the classifier The score of the class output; S333: For each input image frame, calculate the probability values ​​of all categories, and select the category with the highest probability as the preliminary recognition result; S334: For each sensitive content, if its probability value is higher than a preset threshold, it is marked as potential sensitive content.

8. A real-time video sensitive content recognition system, used to implement a real-time video sensitive content recognition method as claimed in any one of claims 1 to 7, characterized in that: Includes the following modules: Video stream receiving module: used to receive real-time video streams from multiple video sources, and split the received video streams into frames to obtain synchronized audio data and video image frames; Audio processing module: used to pre-process the split audio data, including denoising and volume normalization, and input the processed audio data into the multi-language recognition module for language recognition; Multi-language recognition module: connected to the audio processing module, using a pre-trained multi-language recognition model to detect the language type in the audio; when a non-mainstream language or dialect is detected, the dialect recognition model is called to perform semantic analysis and generate corresponding text data; Image processing module: connected to the video stream receiving module, used to pre-process the split video image frames, including image denoising, brightness adjustment and size standardization; The preprocessed image frames are input into a pre-trained convolutional neural network model to extract multi-level features in the image, including feature information of edges, textures, and object shapes; Sensitive content identification module: used to preliminarily identify potential sensitive content based on multi-level feature information in text data and images, such as violence, indecent behavior and dangerous items, and to label each frame of the image with a corresponding sensitive content label; Sensitive content definition library update module: through real-time synchronization with the online regulatory database, social standard library and user-defined rule library, it obtains the latest laws, regulations, social standards and user rules, updates the sensitive content definition library in the system, and dynamically adjusts the judgment criteria for sensitive content; Classification and processing module: used to classify the initially identified sensitive content according to the real-time updated sensitive content definition library, and divide it into three levels: high, medium and low according to the severity of the sensitive content; according to different levels, corresponding processing strategies will be triggered, including blocking, delayed playback or alarming the management personnel.

Images