A content security management method based on multi-modal sequence penalty

Abstract

The application discloses a content security management and control method based on a multi-modal sequence penalty. At least one to-be-detected GOP and corresponding to-be-detected audio are extracted from multiple continuous audio and video frames. A GOP with the largest difference from other GOPs is selected as a target GOP. The P-frame prediction value of the target GOP, the prediction error of the P-frame of the target GOP, and the target audio signal corresponding to the target GOP are input into a Transformer framework to infer a sequence penalty value of the target GOP. The multiple continuous audio and video frames, the sequence penalty value, and random noise are input into a generative adversarial network generator for training to generate audio and video frames with abnormal content. The audio and video frames with abnormal content are input into the trained generative adversarial network, and a probability score of a discrimination result is output by a discriminator in the generative adversarial network. Whether the audio and video frames with abnormal content violate rules is determined based on the tolerance of the probability score.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of display screens, and in particular to a content security management method based on a multi-modal sequence penalty, an electronic device, and a display screen. BACKGROUND

[0002] With the popularity of LED and 4K high-resolution screens, and the rise of technologies such as the Internet of Things, mobile Internet, and big data, high-resolution large screens are increasingly used as display media for content playing and information publishing. In these scenarios, safety accidents occur frequently during content playing, which are mostly caused by improper operation, hacker attacks, and deliberate pranks, highlighting the lack of content management for content played on large screens and information published. In fact, most large screens in public places are in a zero-protection state, which poses a security risk and has low system security. Once the content is played incorrectly, attacked by hackers, or deliberately tampered with, it may have a wide range of adverse effects.

[0003] If the content played on a large screen is managed only based on text recognition detection, there are problems such as strong network dependence, high latency, and large resource consumption. Moreover, in the case of multimedia prevalence, audio, video, and text may not correspond, so it is not possible to manage the risks of audio and video content through text recognition. SUMMARY

[0004] The embodiments of the present application provide a content security management method based on a multi-modal sequence penalty to solve the problem of being unable to manage multimedia playing content through text recognition.

[0005] In a first aspect, the embodiments of the present application provide a content security management method based on a multi-modal sequence penalty, comprising:

[0006] extracting at least one to-be-detected GOP (Group of Pictures) and corresponding to-be-detected audio from a plurality of continuous audio-video frames;

[0007] comparing the to-be-detected GOP with all GOPs in the continuous audio-video frames in terms of HASH value difference, and selecting a GOP with the largest difference in HASH value from other GOPs as a target GOP, wherein each GOP corresponds to a HASH value;

[0008] extracting an I frame and a P frame from the target GOP, decoding the I frame to obtain a P frame prediction value of the target GOP, comparing the P frame prediction value of the target GOP with the P frame of the target GOP, and obtaining a prediction error of the P frame of the target GOP;

[0009] inputting the P-frame prediction value of the target GOP, the prediction error of the P-frame of the target GOP, and a target audio signal corresponding to the target GOP into a Transformer framework to infer a sequence penalty value of the target GOP;

[0010] training the plurality of continuous audio-video frames, the sequence penalty value, and random noise by using a generator of a generative adversarial network to generate an audio-video frame with abnormal content;

[0011] inputting the audio-video frame with abnormal content into the trained generative adversarial network to output a probability score of a discrimination result by using a discriminator in the generative adversarial network;

[0012] judging whether the audio-video frame with abnormal content is in violation based on a tolerance of the probability score.

[0013] Optionally, the inputting the P-frame prediction value of the target GOP, the prediction error of the P-frame of the target GOP, and a target audio signal corresponding to the target GOP into a Transformer framework to infer a sequence penalty value of the target GOP comprises:

[0014] stacking first position information in the form of an embedding matrix to the P-frame prediction value of the target GOP, the prediction error of the P-frame of the target GOP, and a target audio signal corresponding to the target GOP to obtain a first fusion feature matrix with the first position information, and inputting the first fusion feature matrix into an encoder of the Transformer framework to obtain normalized first attention scores;

[0015] Optionally, in the continuous audio-video frames, a second GOP other than the target GOP is optionally selected, and the P-frame prediction value in the second GOP, the prediction error of the P-frame, and a second audio signal corresponding to the second GOP are stacked in the form of an embedding matrix to second position information to obtain a second fusion feature matrix with the second position information, and the second fusion feature matrix is input into a decoder of the Transformer framework to obtain the sequence penalty value of the target GOP.

[0016] Optionally, the Transformer framework comprises an encoder and a decoder, the encoder comprises one layer of first attention layer, two layers of first stacking and normalization layer, and one layer of first feedforward network layer, and the decoder comprises two layers of second attention layer, three layers of second stacking and normalization layer, one layer of second feedforward network layer, and one layer of linear layer.

[0017] The inputting the P-frame prediction value, the prediction error of the P-frame, and a target audio signal corresponding to the target GOP into a Transformer framework to infer a sequence penalty value of the target GOP comprises:

[0018] The P-frame prediction value of the target GOP, the prediction error of the P-frame of the target GOP, and the target audio signal corresponding to the target GOP are superimposed with first position information in the form of an embedding matrix to obtain a first fusion feature matrix with first position information. The first fusion feature matrix is input into an encoder of the Transformer framework, passes through a first attention layer and a first layer first superposition and normalization layer to obtain a normalized first attention score, the first attention score is input into a first feedforward network layer for linear activation to obtain a first activation value, and the first activation value is input into a second layer first superposition and normalization layer to obtain a first output feature of the encoder.

[0019] In the continuous audio and video frames, optionally a second GOP other than the target GOP, the P-frame prediction value in the second GOP, the prediction error of the P-frame, and the second audio signal corresponding to the second GOP are superimposed with second position information in the form of an embedding matrix to obtain a second fusion feature matrix with second position information. The second fusion feature matrix is input into a decoder of the Transformer framework, passes through a first layer second attention layer and a first layer second superposition and normalization layer to obtain a normalized second attention score, the second attention score and the first output feature are input into a second layer second attention layer to obtain a first fusion attention feature with target abnormal content information, and the first fusion attention feature is sequentially input into a second layer second superposition and normalization layer, a second feedforward network layer, a third layer second superposition and normalization layer, and a linear layer to obtain a sequence penalty value of the target GOP.

[0020] Optionally, the P-frame prediction value of the target GOP, the prediction error of the P-frame of the target GOP, and the target audio signal corresponding to the target GOP are superimposed with first position information in the form of an embedding matrix to obtain a first fusion feature matrix with first position information. The first fusion feature matrix is input into an encoder of the Transformer framework, passes through a first attention layer and a first layer first superposition and normalization layer to obtain a normalized first attention score, including:

[0021] The position matrix with the P-frame prediction value, the prediction error of the P-frame, and the target audio signal in the target GOP is superimposed with the embedding matrix of the P-frame prediction value, the prediction error of the P-frame, and the target audio signal in the target GOP to obtain a first fusion feature matrix with first position information, wherein the dimension of the embedding matrix is 3*64.

[0022] The embedding matrix is multiplied with a query matrix weight, a key matrix weight and a value matrix weight respectively to obtain three matrices of a query matrix, a key matrix and a value matrix, and the three matrices are calculated by a softmax function to obtain the normalized first attention score.

[0023] Optionally, the three matrices are calculated by a softmax function to obtain the normalized first attention score, including:

[0024]

[0025] wherein Z represents the normalized first attention score, Q represents the query matrix, K represents the key matrix, V represents the value matrix, d k represents the dimension of the value matrix.

[0026] Optionally, the plurality of continuous audio and video frames and the sequence penalty value are input into a generator of a generative adversarial network for training to generate an audio and video frame with abnormal content; the audio and video frame with abnormal content is input into the trained generative adversarial network, and a probability score of a discrimination result output by a discriminator in the generative adversarial network is output, including:

[0027] A first probability distribution of the sequence penalty value after superimposing random noise and a second probability distribution of the sequence penalty value of a normal GOP are calculated by superimposing random noise in the sequence penalty value of the audio and video frame with abnormal content according to the following formula, and an abnormal value in the first probability distribution and the second probability distribution is taken as the probability score of the discrimination result output by the discriminator, wherein the normal GOP is any one other GOP in the plurality of continuous audio and video frames except a target GOP:

[0028]

[0029] wherein D represents the discriminator, G represents the generator of the generative adversarial network, x represents the normal GOP, represents the sequence penalty value of the normal GOP, represents the sequence penalty value of the target GOP, and z represents random noise.

[0030] Optionally, whether the audio and video frame with abnormal content violates a rule is judged based on a tolerance of the probability score, including:

[0031] If the probability score of the discrimination result output by the discriminator is greater than the tolerance of the probability distribution, it is determined that the audio and video frame with abnormal content violates the rule, wherein the tolerance of the probability distribution is 0.8.

[0032] Optionally, the method further comprises: when it is determined that the audio-video frame with abnormal content violates the regulation, not displaying all video frames of the plurality of continuous audio-video frames and not playing all audio signals in the plurality of continuous audio-video frames.

[0033] In a second aspect, an embodiment of the present application provides an electronic device, comprising:

[0034] a processor and a memory;

[0035] The memory stores computer readable instructions, which are executed by the processor to implement the above method.

[0036] In a third aspect, an embodiment of the present application provides a display screen, comprising:

[0037] a display and a processor, wherein the processor is configured to execute the above method.

[0038] In a fourth aspect, an embodiment of the present application provides a display screen further comprising an audio playing device,

[0039] The processor is further configured to, when it is determined that the audio-video frame with abnormal content violates the regulation, instruct the display screen to not display all video frames of the plurality of continuous audio-video frames and instruct the audio playing device to not play all audio signals in the plurality of continuous audio-video frames.

[0040] In an embodiment of the present application, since the most different target GOP is found in the plurality of continuous audio-video frames, the most representative video frame in the audio-video frame can be found, and the P-frame prediction value and the prediction error of the target GOP are obtained by decoding, the P-frame prediction value and the prediction error of the target GOP and the corresponding target audio signal are inferred through the Transformer framework, the sequence penalty value of the target GOP is obtained, the sequence penalty value and the random noise are combined through the generative adversarial network generator to generate the audio-video frame with abnormal content, the discriminator of the generative adversarial network is trained, and the purpose of judging whether the content of the audio-video frame violates the regulation is achieved. Since the sequence penalty value combines the prediction information of the P-frame of the most representative audio-video frame, the representative content of the audio-video frame can be better represented, so that whether the regulation content exists in the video frame and the audio signal can be more quickly and accurately judged. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 A content safety management method based on multi-modal sequence penalty provided by an embodiment of the present application is shown in the schematic diagram;

[0042] Figure 2 A flow and network schematic diagram of the content safety management method based on multi-modal sequence penalty provided by an embodiment of the present application is shown in the schematic diagram;

[0043] Figure 3 A flowchart of a Transformer architecture provided for an embodiment of the present application is shown in FIG. 1.

[0044] Figure 4 A module schematic diagram of a content security management device based on multi-modal sequence penalty provided for an embodiment of the present application is shown in FIG. 2. DETAILED DESCRIPTION

[0045] The present application will be described in detail below with reference to specific embodiments shown in the drawings, but these embodiments do not limit the present application, and any structural, method, or functional changes made by those skilled in the art based on these embodiments are included in the protection scope of the present application.

[0046] If content detection is performed frame by frame for video content, such as decoding and picture restoration for each frame image in each GOP (Group of Pictures), including I frames, P frames, and B frames, and then performing OCR text recognition or content detection, a display screen needs to be equipped with a CPU with high computing power, memory with higher cost, higher network bandwidth, and other hardware and network resources, which not only consumes too much overall cost, but also requires more stringent network real-time performance. If a content detection method with an embedded whitelist is used, such as an embedded whitelist picture or whitelist text, each frame image is analyzed and restored frame by frame, and the matching result with the whitelist image is detected by OCR, if there is new content that is not in the whitelist, there may be a problem of not being able to play. If a content detection method with an embedded blacklist is used, there may be many new illegal contents that are not stored in the blacklist library and cannot be identified, and the problem of directly playing out.

[0047] In addition, for audio and video frames with continuous frames, P frames cannot usually decode complete images independently, and need to be decoded together with the previous I frame. Moreover, P frames are reference frames, and once tampered, they are prone to cause the spread of decoding errors and form illegal content. That is, the decoding of P frames depends on the decoding results of the previous I frame or P frame. Therefore, if the content needs to be identified and controlled, all image frames need to be decoded, and each image frame needs to be restored, and then content identification is performed. In this way, CPU and memory resources are consumed greatly, especially for high-resolution image frames. The time consumption for identifying and controlling the entire multimedia file is longer, and if the overall length of the multimedia file is longer, the real-time performance of normal video playback will be delayed. Moreover, the encoding and decoding of I frames and P frames depend on network bandwidth, and in a poor network state, frame loss may occur. At this time, the method of encoding and decoding images frame by frame cannot analyze the entire multimedia file. If OCR detection is used, text needs to be extracted from all video frames, and then text recognition and detection are performed, and then whether there is illegal text is judged by comparing the black and white lists. Obviously, this method has large delay, and if there is no text in the multimedia file, and the illegal content is image and audio, the purpose of content control cannot be achieved, and the practicability is not high.

[0048] To solve the above problems, the embodiment of the present application provides a content safety control method based on multi-modal sequence penalty, referring to Figure 1 , comprising:

[0049] S101: extracting at least one to-be-detected GOP (Group of Pictures) and corresponding to-be-detected audio from a plurality of continuous audio and video frames.

[0050] Exemplarily, the plurality of continuous audio and video frames in S101 can adopt a format of 1080P / 60fps and a GOP value of 25.

[0051] S102: performing HASH value difference comparison on the to-be-detected GOP and all GOPs in the continuous audio and video frames, and selecting a GOP with the largest difference in HASH value from other GOPs as a target GOP.

[0052] Each GOP corresponds to a HASH value.

[0053] Optionally, for each GOP, the HASH value H G may be calculated by the following formula, but is not limited thereto:

[0054] wherein H G represents the HASH value of each GOP, I represents the byte number of the key frame of the GOP, and P represents the byte number of the forward prediction frame of the GOP.

[0055] The average of the HASH values of all the GOPs of the plurality of continuous audio and video frames can be determined by the following formula, but is not limited thereto:

[0056]

[0057] wherein, represents the average of the HASH values of all the GOPs, n represents the number of GOPs in the plurality of continuous audio and video frames, and H G1 to H Gn represents the HASH value of the first to nth GOPs.

[0058] Target GOP may be determined by the following formula, but is not limited thereto:

[0059]

[0060] wherein, represents the target GOP, and H G represents the HASH value of each GOP, represents the average of the HASH values of all the GOPs.

[0061] For example, the HASH value H G of each GOP can be calculated by MD4 or MD5 or SHA-1, and then the average of the HASH values of all the GOPs is calculated. The GOP with the largest difference from the average is taken as the target GOP. .

[0062] Embodiments of the present application find the GOP with the largest difference from all the GOPs, and infer the sequence penalty value of the P-frame prediction value, so that the image can be compared and detected whether it is tampered without decoding the image, thereby solving the problem that the tampered P-frame can only be detected by decoding the continuous frames in sequence and cannot be detected alone.

[0063] Referring to Figure 1 , in embodiments of the present application, the normal audio and video V n includes a plurality of continuous audio and video frames GOPs, and the target GOP and the corresponding audio signal n are extracted from the normal audio and video V . For example, each GOP of the plurality of continuous audio and video frames can be compared with the HASH values of other GOPs, and the GOP with the largest difference from the HASH values of other GOPs is taken as the target GOP. Since only HASH value calculation is performed to screen the target GOP, and all the audio and video frames do not need to be decoded frame by frame, the overall calculation amount and the memory requirement of the display screen are greatly saved.

[0064] S103: extract the I frame and the P frame from the target GOP, decode the I frame to obtain a P frame prediction value of the target GOP, and compare the P frame prediction value of the target GOP with the P frame of the target GOP to obtain a prediction error of the P frame of the target GOP.

[0065] Exemplarily, the I frame can be decoded by using at least one of H.264 (or AVC), HEVC, ProRes, and DNxHD. In this way, for any one GOP, the P frame prediction value can be a P frame estimation value obtained by decoding the I frame. The prediction error of the P frame of the target GOP can be a difference between the P frame prediction value of the target GOP and the P frame of the target GOP.

[0066] S104: input the P frame prediction value of the target GOP, the prediction error of the P frame of the target GOP, and a target audio signal corresponding to the target GOP into a Transformer framework to infer a sequence penalty value of the target GOP.

[0067] The sequence penalty value can represent a prediction value of a next P frame generated based on the P frame prediction value obtained by decoding the I frame and the prediction error of the P frame.

[0068] S105: input the plurality of continuous audio-video frames, the sequence penalty value, and random noise into a generative adversarial network generator for training to generate an audio-video frame with abnormal content.

[0069] S106: input the audio-video frame with abnormal content into the trained generative adversarial network to output a probability score of a discrimination result through a discriminator in the generative adversarial network.

[0070] S107: determine whether the audio-video frame with abnormal content is in violation based on a tolerance of the probability score.

[0071] Referring to Figure 2 In an optional embodiment of the present application, a plurality of continuous audio-video frames are used as normal audio-video, and V n is used to represent that one GOP includes one I frame (key frame) and a plurality of forward prediction frames P frames. The prediction value I pv of the P frame is extracted from the GOP. The prediction error P pv of the P frame with respect to the I pe frame. The picture restoration of the video frame can be performed through the I pv frame and the P pe frame, and the prediction value I pv of the P frame and the prediction error P pe of the P frame are superimposed to obtain the picture restoration result of the video frame.

[0072]

[0073] wherein P represents the picture restoration result of the P-frame, I pv characterizes the prediction value of the P-frame and P pe characterizes the prediction error of the P-frame relative to the prediction value.

[0074] the prediction value of the P-frame I pv and the prediction error of the P-frame P pe and the audio signal corresponding to the GOP may be characterized in the form of an embedding matrix as follows:

[0075]

[0076] wherein I pv1 …I pvn characterizes the prediction value of the 1st~nth P-frame in the target GOP. The dimension of the embedding matrix may be, but is not limited to, 3*64.

[0077] The position information may be represented in the following matrix,

[0078]

[0079] The formula can be generally expressed as:

[0080]

[0081]

[0082] P pos characterizes the position information. The fusion feature matrix may be obtained in the following way:

[0083]

[0084] The prediction value of the P-frame I pv of the target GOP, the prediction error of the P-frame P pe of the target GOP, and the target audio signal corresponding to the target GOP are superimposed with the first position information in the form of an embedding matrix to obtain a first fusion feature matrix with the first position information.

[0085] Referring to the above process, after the embedding matrix of the target GOP is superimposed with the first position information to obtain the first fusion feature matrix, the first fusion feature matrix is input into the encoder of the Transformer framework to obtain the normalized first attention score. After the embedding matrix of the normal GOP is superimposed with the second position information to obtain the second fusion feature matrix, the second fusion feature matrix is input into the decoder of the Transformer framework to obtain the sequence penalty value P v. The random noise and the sequence penalty value P v are input into a generator of a generative adversarial network to generate a video and audio V a with abnormal content. a The video and audio V n with abnormal content is input into a discriminator of the generative adversarial network, and the generative adversarial network is trained. The video and audio frame to be detected is input into the trained generative adversarial network to output a probability score P s of a discrimination result, and it is determined whether the video and audio to be detected is normal / abnormal.

[0086] Referring to FIG. 6, the encoder of the Transformer framework includes a first attention layer, two first additive and normalization layers, and a first feedforward network layer. The decoder of the Transformer framework includes two second attention layers, three second additive and normalization layers, a second feedforward network layer, and a linear layer. Figure 3 For example, the first additive and normalization layer stacks the output of the first attention layer and the input of the encoder of the Transformer framework through an additive structure, and then performs normalization processing. The second additive and normalization layer stacks the output of the second attention layer and the input of the encoder of the Transformer framework through an additive structure, and then performs normalization processing.

[0087] The embedding matrix X pos is multiplied by the query matrix weight W Q , the key matrix weight W K , and the key matrix weight W V , respectively, to obtain three matrices of the query matrix Q, the key matrix K, and the key matrix V.

[0088]

[0089]

[0090]

[0091]

[0092]

[0093] For example, W Q , W K ​​, W V The values of the three weights can be initialized with random values, and the three weights have the same dimension as the embedding matrix described above.

[0094] The three matrices Q, K and V are normalized by the softmax function to obtain the first attention score Z, and the softmax function is as follows:

[0095]

[0096] where Q represents the query matrix, K represents the key matrix, and V represents the value matrix, represents the position matrix, The embedding matrix represented by represents the prediction value I of the P frame pv and the prediction error P of the P frame pe and the audio signal corresponding to the GOP The three embedding vectors, The embedding matrix represented by represents the embedding vector of the band position parameter. Optionally, the dimension of the embedding vector in the embedding matrix is 64, so the embedding matrix has a dimension of [3*64]. The dimension of the value matrix is 64.

[0097] The first attention score is input into the first feedforward network layer for linear activation to obtain the first activation value, and the formula can be as follows:

[0098] wherein, represents the first activation value, and represents the weight matrix, and represents the bias parameter, wherein the weight matrix and The initial value is initialized with a small random value, and in subsequent model optimization, the bias parameter and The initial value is generally 0 or a small constant, for example, 1e-8, and in the subsequent model optimization process, a constant from a uniform distribution can be sampled.

[0099] The first activation value is input into the second layer of the first superposition and normalization layer to obtain the first output feature of the encoder.

[0100] For example, the first activation value is input into the second layer of the first superposition and normalization layer for normalization processing, which can be implemented by the following formula:

[0101]

[0102] Referring to Figure 3It can As the first output feature of the encoder, it is input into the second attention layer of the decoder, so that the first output feature of the encoder is superimposed with the normal GOP, thereby enabling the Transformer framework to better focus on the abnormal parts.

[0103] In all GOPs of consecutive audio and video frames, select any second GOP other than the target GOP. Add the P-frame prediction value, the P-frame prediction error, and the second audio signal corresponding to the second GOP to the second GOP in the form of an embedding matrix to obtain the second fusion feature matrix with the second position information. Then input the second fusion feature matrix into the decoder of the Transformer framework.

[0104] See Figure 3 In the decoder, the second fusion feature matrix is ​​passed through the first second attention layer and the first second superposition and normalization layer to obtain the normalized second attention score. The second attention score and the first output feature are input into the second second attention layer to obtain the first fusion attention feature with target abnormal content information. The first fusion attention feature is then sequentially input into the second second superposition and normalization layer, the second feedforward network layer, the third second superposition and normalization layer, and the linear layer to obtain the sequence penalty value of the target GOP.

[0105] For example, the input to the first and second attention layers may include an embedding matrix X with added location information. pos The output of the first and second attention layers can include the attention score Z. i .

[0106] The inputs to the first layer, the second layer, and the normalized layer can include the attention score Z. i And the aforementioned embedding matrix, output attention score Z. normal .

[0107] The input to the second attention layer includes the output of the encoder and the Z-axis output of the first second attention layer. normal The output of the second attention layer includes Z. decode_normal Z decode_normal After inputting into the second feedforward network layer, the attention score Z after linear activation is obtained. ffn .

[0108] Z ffn After inputting the third layer's second stacking and normalization layers, and combining the output of the second layer's second attention layer, the normalized Z can be obtained. ffn_normal Z through linear layers ffn_normal Perform linear normalization to obtain the target GOP. a sequence penalty value P v .

[0109] In the above target GOP The sequence penalty value after superimposing random noise z on the sequence penalty value can be used to simulate the tampered or replaced audio and video frames (i.e., audio and video frames with abnormal content) , the The generated audio and video frames with abnormal content are input into the generator of the generative adversarial network for training. The audio and video frames with abnormal content are input into the trained generative adversarial network, and the probability score of the discrimination result output by the discriminator in the generative adversarial network is output.

[0110] Exemplarily, the first probability distribution of the sequence penalty value after superimposing random noise on the audio and video frames with abnormal content and the second probability distribution of the sequence penalty value of the normal GOP can be calculated by superimposing random noise on the sequence penalty value of the audio and video frames with abnormal content according to the following formula, and the abnormal values in the first probability distribution and the second probability distribution are taken as the probability score of the discrimination result output by the discriminator of the generative adversarial network, wherein the normal GOP is any one of the other GOPs in the plurality of continuous audio and video frames except the target GOP:

[0111]

[0112] Wherein, D represents the discriminator, G represents the generator of the generative adversarial network, x represents the normal GOP, represents the normal sequence penalty value output by the Transformer, represents the abnormal sequence penalty value containing random noise, and z represents the random noise.

[0113] If the probability score of the discrimination result output by the discriminator is greater than the tolerance of the probability distribution, it is determined that the audio and video frames with abnormal content are in violation, wherein the tolerance of the probability distribution is 0.8.

[0114] When it is determined that the audio and video frames with abnormal content are in violation, all video frames of the plurality of continuous audio and video frames are not displayed, and all audio signals in the plurality of continuous audio and video frames are not played.

[0115] Based on the same inventive concept, referring to Figure 4 The embodiments of the present application also provide a content security management device based on multi-modal sequence penalty, which comprises:

[0116] A video frame extraction module is configured to extract at least one video group GOP to be detected from a plurality of continuous audio and video frames (i.e., normal audio and video frames).

[0117] an audio signal extraction module configured to extract corresponding to-be-detected audio from a plurality of continuous audio-video frames;

[0118] a sequence penalty generation module configured to perform HASH value difference comparison between the to-be-detected GOP and all GOPs in the continuous audio-video frames, select a GOP with the largest HASH value difference from other GOPs as a target GOP, extract an I frame and a P frame from the target GOP, decode the I frame to obtain a P frame prediction value of the target GOP, compare the P frame prediction value of the target GOP with a P frame of the target GOP to obtain a prediction error of the P frame of the target GOP, and input the P frame prediction value of the target GOP, the prediction error of the P frame of the target GOP, and a target audio signal corresponding to the target GOP into a Transformer framework to obtain a sequence penalty value of the target GOP through inference;

[0119] an abnormal audio-video generation module configured to input the plurality of continuous audio-video frames, the sequence penalty value, and random noise into a generator of a generative adversarial network for training to generate an audio-video frame with abnormal content;

[0120] a discriminator training module configured to input the audio-video frame with abnormal content into the generative adversarial network, and output a probability score of a discrimination result through a discriminator in the generative adversarial network;

[0121] a discrimination detection module configured to determine whether the audio-video frame with abnormal content is in violation based on a tolerance of the probability score.

[0122] Embodiments of the present application provide a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned content safety management method based on multi-modal sequence penalty when executing the computer program.

[0123] Embodiments of the present application provide a display screen, comprising:

[0124] a display and a processor, the processor is used to execute the above-mentioned content safety management method based on multi-modal sequence penalty.

[0125] Optionally, the display screen further comprises an audio playing device,

[0126] The processor is further configured to instruct the display screen not to display all video frames of the plurality of continuous audio-video frames and instruct the audio playing device not to play all audio signals in the plurality of continuous audio-video frames when determining that the audio-video frame with abnormal content is in violation.

[0127] Illustratively, the audio playing device can be a loudspeaker.

[0128] In optional embodiments of the present application, the display screen can include, but is not limited to, a processor, a touch panel, a voice input device, a speaker, a display, and the like.

[0129] The memory can be a random access memory, a read only memory, a non-volatile, a programmable ROM, an erasable PROM, an electrically erasable, a flash memory, an optical memory, a register, and the like. The processor can be a general purpose processor, which can be a processor that performs specific steps and / or operations by reading and executing computer programs stored in the memory, and can use the memory stored in the memory in the process of executing the steps and / or operations. The general purpose processor can be a central processing unit, an ASIC, an FPGA, and the like. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor or the instruction in the form of software. The method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in the processor for execution.

[0130] In the above embodiments, all or part of the embodiments can be realized by software, hardware, firmware, or any combination thereof. When realized by software, all or part of the embodiments can be realized in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center and the like integrated with one or more available media. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), optical medium (for example, DVD), or solid state disk (SSD) and the like.

[0131] It is to be noted that, as used in this document, the terminology "first", "second", etc. is merely used to differentiate one entity or action from another, and does not necessarily imply or require any actual relationship or order between or among the entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0132] Each of the embodiments in the present specification is described in a related manner, and the same or similar parts among the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the description of the method embodiments. The above only describes the preferred embodiments of the present application, and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A content security management method based on a multi-modal sequence penalty, characterized in that, The method comprises: extracting at least one to-be-detected video group GOP and corresponding to-be-detected audio from a plurality of continuous audio-video frames; performing HASH value difference comparison on the to-be-detected GOP and all GOPs in the continuous audio-video frames, and selecting a GOP with the largest difference in HASH value from other GOPs as a target GOP, wherein each GOP corresponds to a HASH value; extracting an I frame and a P frame from the target GOP, decoding the I frame to obtain a P frame prediction value of the target GOP, and comparing the P frame prediction value of the target GOP with a P frame of the target GOP to obtain a prediction error of the P frame of the target GOP; inputting the P frame prediction value of the target GOP, the prediction error of the P frame of the target GOP, and a target audio signal corresponding to the target GOP into a Transformer framework to infer a sequence penalty value of the target GOP, wherein the sequence penalty value represents a prediction value of a next frame P frame generated based on the P frame prediction value obtained through I frame decoding and the prediction error of the P frame; inputting the plurality of continuous audio-video frames, the sequence penalty value, and random noise into a generator of a generative adversarial network to generate an audio-video frame with abnormal content; inputting the audio-video frame with abnormal content into the generative adversarial network to output a probability score of a discrimination result through a discriminator in the generative adversarial network; judging whether the audio-video frame with abnormal content violates a rule based on a tolerance of the probability score.

2. The method of claim 1, wherein, inputting the P frame prediction value of the target GOP, the prediction error of the P frame of the target GOP, and the target audio signal corresponding to the target GOP into a Transformer framework to infer a sequence penalty value of the target GOP, comprising: stacking first position information in the form of an embedding matrix to obtain a first fusion feature matrix with the first position information, and inputting the first fusion feature matrix into an encoder of the Transformer framework to obtain normalized first attention scores; optionally selecting a second GOP other than the target GOP from the continuous audio-video frames, stacking second position information in the form of an embedding matrix to obtain a second fusion feature matrix with the second position information, and inputting the second fusion feature matrix into a decoder of the Transformer framework to obtain the sequence penalty value of the target GOP.

3. The method of claim 1, wherein, The Transformer framework comprises an encoder and a decoder, the encoder comprises one layer of first attention layer, two layers of first stacking and normalization layer, and one layer of first feedforward network layer, and the decoder comprises two layers of second attention layer, three layers of second stacking and normalization layer, one layer of second feedforward network layer, and one layer of linear layer. The P-frame prediction value, the prediction error of the P-frame, and the target audio signal corresponding to the target GOP are input into a Transformer framework to infer a sequence penalty value of the target GOP, including: The P-frame prediction value of the target GOP, the prediction error of the P-frame of the target GOP, and the target audio signal corresponding to the target GOP are stacked with first position information in the form of an embedding matrix to obtain a first fusion feature matrix with first position information, the first fusion feature matrix is input into an encoder of the Transformer framework, and after passing through a first attention layer and a first layer first stacking and normalization layer, a normalized first attention score is obtained, the first attention score is input into a first feedforward network layer for linear activation to obtain a first activation value, and the first activation value is input into a second layer first stacking and normalization layer to obtain a first output feature of the encoder; In the continuous audio and video frames, optionally a second GOP other than the target GOP, the P-frame prediction value in the second GOP, the prediction error of the P-frame, and the second audio signal corresponding to the second GOP are stacked with second position information in the form of an embedding matrix to obtain a second fusion feature matrix with second position information, the second fusion feature matrix is input into a decoder of the Transformer framework, and after passing through a first layer second attention layer and a first layer second stacking and normalization layer, a normalized second attention score is obtained, the second attention score and the first output feature are input into a second layer second attention layer to obtain a first fusion attention feature with target abnormal content information, and the first fusion attention feature is sequentially input into a second layer second stacking and normalization layer, a second feedforward network layer, a third layer second stacking and normalization layer, and a linear layer to obtain a sequence penalty value of the target GOP.

4. The method of claim 3, wherein, The P-frame prediction value of the target GOP, the prediction error of the P-frame of the target GOP, and the target audio signal corresponding to the target GOP are stacked with first position information in the form of an embedding matrix to obtain a first fusion feature matrix with first position information, the first fusion feature matrix is input into an encoder of the Transformer framework, and after passing through a first attention layer and a first layer first stacking and normalization layer, a normalized first attention score is obtained, including: The position matrix with the P-frame prediction value, the prediction error of the P-frame, and the target audio signal in the target GOP is stacked with the embedding matrix of the P-frame prediction value, the prediction error of the P-frame, and the target audio signal in the target GOP to obtain a first fusion feature matrix with first position information, wherein the embedding matrix has a dimension of 3*64; The embedding matrix is multiplied by a query matrix weight, a key matrix weight, and a value matrix weight respectively to obtain three matrices of a query matrix, a key matrix, and a value matrix, and the three matrices are calculated by a softmax function to obtain the normalized first attention score.

5. The method of claim 4, wherein, The three matrices are calculated by a softmax function to obtain the normalized first attention score, including: wherein Z represents the normalized first attention score, Q represents the query matrix, K represents the key matrix, V represents the value matrix, d k represents the dimension of the value matrix.

6. The method of claim 1, wherein, The plurality of continuous audio and video frames and the sequence penalty value are input into a generator of a generative adversarial network for training, to generate an audio and video frame with abnormal content; the audio and video frame with abnormal content is input into the trained generative adversarial network, and a probability score of a discrimination result output by a discriminator in the generative adversarial network is output, including: A first probability distribution of the sequence penalty value of the audio and video frame with abnormal content and a second probability distribution of the sequence penalty value of a normal GOP are calculated by superimposing random noise on the sequence penalty value of the audio and video frame with abnormal content according to the following formula, and abnormal values in the first probability distribution and the second probability distribution are taken as the probability score of the discrimination result output by the discriminator, wherein the normal GOP is any one GOP other than a target GOP in the plurality of continuous audio and video frames: wherein D represents the discriminator, G represents the generative adversarial network generator, x represents the normal GOP, a sequence penalty value representing the normal GOP, a sequence penalty value representing the target GOP, and z represents random noise.

7. The method of claim 6, wherein, Based on a tolerance of the probability score, it is determined whether the audio and video frame with abnormal content is in violation, including: If the probability score of the discrimination result output by the discriminator is greater than the tolerance of the probability distribution, it is determined that the audio and video frame with abnormal content is in violation, wherein the tolerance of the probability distribution is 0.

8.

8. The method of claim 1, wherein, The method further includes: When it is determined that the audio and video frame with abnormal content is in violation, all video frames of the plurality of continuous audio and video frames are not displayed, and all audio signals in the plurality of continuous audio and video frames are not played.

9. An electronic device, comprising: Including: A processor and a memory; The memory has computer readable instructions stored thereon, and the computer readable instructions are executed by the processor to implement the method of any one of claims 1 to 8.

10. A display screen, characterized by Including: A display and a processor, and the processor is configured to execute the method of any one of claims 1 to 7.

11. The display screen of claim 10, wherein, The display screen further includes an audio playing device, The processor is further configured to, when it is determined that the audio and video frame with abnormal content is in violation, instruct the display screen to not display all video frames of the plurality of continuous audio and video frames, and instruct the audio playing device to not play all audio signals in the plurality of continuous audio and video frames.

Images