Multi-modal security fence method based on vector matching

Abstract

The invention discloses a multi-modal security fence method based on vector matching. The method comprises the following steps: acquiring to-be-audited information, loading a pre-training model set, vectorizing the to-be-audited information, and generating to-be-audited vectors; the to-be-audited information comprises a live broadcast real-time stream and a newly added media file; establishing connection with a violation vector database set, wherein the violation vector database set comprises vector features of various types of violation data; according to the violation vector database set, obtaining the compliance similarity of the to-be-audited vector, and judging whether the to-be-audited information is safely broadcasted or not; the pre-training model set is composed of a text vector model, a picture vector model and a tone vector model, and is used for generating vector information from various types of to-be-audited information. According to the technical scheme, the text, the picture, the audio and the video of the media asset can be detected, intercepted and modified in all directions, the broadcasting safety is ensured, the content auditing efficiency is greatly improved, and the broadcasting safety is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal audio-visual retrieval and compliance supervision in an IPTV environment, and in particular to a multimodal safety fence method based on vector matching. Background Art

[0002] With the development of IPTV services, the number of human-computer interaction scenarios has grown rapidly. The maturity of the content production technology of generative AI and AIGC (artificial intelligence generated content) has brought about a qualitative improvement in the video content and richness in the IPTV environment, which can better meet the viewing needs of users. At the same time, it also brings about the technical requirements for multi-dimensional security retrieval and processing of text, pictures, audio and video content.

[0003] Unlike the content played on the Internet platform, IPTV content comes from multiple sources, and it is difficult to grasp the content at the source end. Therefore, in order to ensure the effectiveness of the content played, it is necessary to strictly control the multimodal content during playback. In real applications, IPTV has diverse characteristics in both content and regulatory dimensions. The existing video review and broadcast guarantee technology mainly relies on frame-by-frame comparison of video content with graphic features, or even manual frame-by-frame viewing review, which is at a low level in terms of results and efficiency. There are also human errors and omissions in accuracy, and there are greater content risks. Existing solutions are also difficult to meet the security requirements of live broadcasts.

[0004] Therefore, a technical solution is needed that can quickly search and match the types of playback content such as text, pictures, audio and video, and implement different processing strategies according to different types of content, so as to improve the review efficiency of IPTV content and meet the needs of safe broadcasting. Summary of the invention

[0005] To achieve the above objectives, the present application provides a multi-modal safety fence method based on vector matching, comprising the following steps:

[0006] Obtain the information to be reviewed, load the pre-trained model set, vectorize the information to be reviewed, and generate a vector to be reviewed; the information to be reviewed includes live real-time streams and newly added media files;

[0007] Establishing a connection with a violation vector database set, wherein the violation vector database set includes vector features of multiple types of violation data;

[0008] According to the violation vector database set, the compliance similarity of the vector to be reviewed is obtained to determine whether the information to be reviewed is safe to broadcast;

[0009] Among them, the pre-trained model set consists of a text vector model, a picture vector model and a timbre vector model, which are used to generate vector information from various types of information to be reviewed; the types of information to be reviewed include: text, picture, audio and video.

[0010] The types of breached data include: text, images, audio, and video.

[0011] Wherein, when the type is text, the vectorization processing is implemented by text vectorization;

[0012] The text vectorization refers to: processing the text content through a text vector model to obtain a high-dimensional vector representation;

[0013] The text vector model is constructed based on semantic vectors, Chinese STS dataset and self-annotated data training.

[0014] When the type is a picture, the vectorization processing includes: cropping and converting the picture to generate a pre-recognized picture, loading a picture vector model, and generating a vector representation of the pre-recognized picture; wherein the picture vector model is trained and generated using the Flickr30K-CN data set and self-annotated data;

[0015] When the type is a picture, the vectorization processing further includes: identifying text information of the information to be reviewed, performing text vectorization, and generating a text-based vector representation.

[0016] When the type is audio, the vectorization processing includes: performing speech recognition on the audio to generate text information; performing text vectorization on the text information to generate a text-based vector representation;

[0017] When the type is audio, the vectorization processing further includes: loading a timbre vector model, extracting the timbre of the audio, and obtaining a vectorized representation of the timbre.

[0018] Furthermore, timbre extraction refers to: detecting voice activity in audio, splitting voice segments of multiple speakers, extracting the voiceprints of the speakers in the voice segments, and obtaining a vectorized representation of the timbre by a timbre vector model;

[0019] Among them, the timbre vector model is a voiceprint recognition model trained based on a sensitive data set, and the content of the sensitive data set is the voiceprint features of pre-defined sensitive sounds.

[0020] When the type is video, the vectorization process includes:

[0021] Obtain scene features of the video and obtain vector representation for the scene features;

[0022] Obtaining picture change frames in the video, obtaining scene features of the picture change frames, and obtaining vector representations for the scene features;

[0023] When the scene features are represented by vectors, the type is converted into an image and vectorization processing is performed.

[0024] Wherein, obtaining the picture change frame in the video refers to:

[0025] Extract the HSV color space of the specified frame of the video and compare it with the previous frame using the pixel value difference method;

[0026] Obtain the average difference of the corresponding pixels of hue, saturation and brightness, and sum the corresponding weights of the differences of hue, saturation and brightness to generate a feature difference;

[0027] When the feature difference is greater than a specified threshold, the specified frame of the video is a picture change frame.

[0028] Furthermore, obtaining the compliance similarity of the vector to be reviewed is achieved through vector matching;

[0029] Vector matching uses cosine similarity to measure the similarity between two embedded vectors, expressed as: Among them, e1 and e2 are two embedded vectors respectively.

[0030] Furthermore, judging whether the information to be reviewed is safe to broadcast includes: if the compliance similarity is higher than a specified threshold, different strategies need to be adopted according to the different sources; if illegal content is detected in the live real-time stream, the content is replaced before broadcasting; if it is a media resource, it is directly intercepted, and further manual intervention is performed by staff.

[0031] The present invention conducts all-round detection on the text, pictures, audio and video of media assets, performs security detection and interception on newly added media assets; it performs dynamic detection and modification on the broadcast of existing media assets and live real-time streams, cuts and replaces illegal text, pictures, audio and video to ensure broadcast safety, greatly improves content review efficiency and ensures broadcast safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a step diagram of a multi-modal safety fence method provided according to an embodiment of the present invention;

[0033] Figure 2 It is a schematic diagram of the structure of the pre-trained model set provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The vector-based multimodal security fence method provided by the present invention first collects illegal media resource data or sensitive voiceprint features as standards for illegal supervision, and vectorizes the media resource data and saves them as a violation vector database set; uses a pre-trained model to generate multiple vectors for the content to be reviewed and then performs vector matching, blocks and intercepts the content to be broadcast with a high matching degree, or further classifies the content to be broadcast after interception, so as to implement a classified content execution processing strategy.

[0035] The specific implementation of the present invention is described in detail below with reference to the accompanying drawings.

[0036] Before the method provided by the present invention is executed, a violation vector database set is established. The violation vector database set is a feature set including a large number of violation texts, pictures, and timbres, wherein the timbre includes the typical voice features of designated sensitive persons, and the speech of the person can be identified by the typical voice features. The violation vector database set is an updateable feature library, and the staff can continuously add specific content of violations and voice features of sensitive people as needed.

[0037] The steps of the vector-based multi-modal safety fence method provided by the present invention are as follows: Figure 1 As shown, including:

[0038] Step S100: obtaining the information to be reviewed, loading the pre-trained model set, vectorizing the information to be reviewed, and generating a vector to be reviewed;

[0039] The information to be reviewed supported by the present invention includes live real-time streams and newly added media files; generally speaking, live real-time streams have higher requirements for accuracy and timeliness of review.

[0040] In this step, the type of information to be reviewed is determined, where the types of information to be reviewed include: text, picture, audio and video; in the pre-trained model set, different training models are loaded according to their types, where one piece of picture information, audio information and video information may generate multiple vector data.

[0041] Specifically, the pre-trained model set is as follows Figure 2 As shown, it is composed of a text vector model, a picture vector model and a timbre vector model, and is used to generate vector information from various types of information to be reviewed;.

[0042] 1) When the type is text, the vectorization processing is implemented by text vectorization;

[0043] The text vectorization refers to: processing the text content through a text vector model to obtain a high-dimensional vector representation;

[0044] Among them, the text vector model is constructed based on semantic vectors, Chinese STS dataset and self-annotated data training.

[0045] For example: Based on the pre-trained embedding model, the CoSENT method is used for training, using BAAI / bge-large-zh-noinstruct combined with the manually selected Chinese STS dataset, and supplemented with self-labeled data for training and generation. The model generated by training in this way has more obvious discrimination and higher accuracy than the basic model;

[0046] At the same time, text vector models are also used to review image and audio content.

[0047] 2) When the type is a picture, the vectorization processing includes: cropping and converting the picture to generate a pre-recognition picture, loading the picture vector model, and generating a vector representation of the pre-recognition picture based on the characteristics of the picture itself;

[0048] The image vector model uses the Flickr30K-CN dataset, supplemented by self-annotated data training, and is trained and generated based on OpenAI CLIP.

[0049] When the type is a picture, the vectorization processing also generates vectors for the text features of the picture, including: using OCR to recognize the text information of the information to be reviewed (picture), performing text vectorization processing, and generating a text-based vector representation.

[0050] Based on the above operations, two types of vector data can be generated for the image, one based on the features of the image itself, and the other based on the features of the text.

[0051] 3) When the type is audio, the vectorization processing includes: performing speech recognition on the audio to generate text information; performing text vectorization on the text information to generate a text-based vector representation;

[0052] At the same time, timbre extraction is also performed, that is, the timbre vector model in the pre-trained model is loaded, timbre extraction is performed on the audio, and a vectorized representation of the corresponding timbre is obtained.

[0053] Specifically, timbre extraction refers to: detecting the speech activity of the speaker in the audio, splitting the speech segments of multiple speakers, extracting the voiceprint of the speaker in the speech segments, and obtaining a vectorized representation of the timbre through a timbre vector model;

[0054] Among them, the timbre vector model is a model generated by ECAPA-TDNN training based on a sensitive data set, and the content of the sensitive data set is the voiceprint features of pre-defined sensitive sounds, such as the voices of bad artists in the IPTV industry and the voiceprint features of common sensitive sounds.

[0055] 4) When the type is video, the vectorization process includes:

[0056] The scene features of the video are obtained, and a vector representation is obtained for the scene features. Specifically, video scene change detection is first performed to obtain split video frames. The video frames are used as pictures, and a vector representation of the video scene is obtained through a picture vector model.

[0057] The input video contains a large number of information frames. If every frame is paid attention to, there will be disadvantages of large amount of calculation and information redundancy. Therefore, the present invention targets the picture vectors of similar scenes, performs feature average pooling on them, and finally obtains the feature representation of the similar scene, thereby compressing the features of the similar scene;

[0058] On the other hand, for frames with large changes between pictures, that is, for changing scenes, this step captures the picture change frames to obtain the scene features of the picture change frames in the video, and then further obtains a vector representation for the scene features.

[0059] Specifically, the method for obtaining the picture change frame in the video is:

[0060] Extract the HSV color space of the specified frame of the video and compare it with the previous frame using the pixel value difference method;

[0061] Obtain the average difference of the corresponding pixels of hue, saturation and brightness, and sum the corresponding weights of the differences of hue, saturation and brightness to generate a feature difference;

[0062] When the feature difference is greater than a specified threshold, the specified frame of the video is a picture change frame.

[0063] Step S110: Establishing a connection with a violation vector database set, wherein the violation vector database set includes vector features of various types of violation data; the types of various types of violation data include: text, picture, audio and video;

[0064] In this step, the compliance similarity of the vector to be reviewed is obtained according to the violation vector database set.

[0065] Among them, cosine similarity is used to measure the similarity between two embedded vectors, expressed as:

[0066] Among them, e1 and e2 are two embedded vectors respectively.

[0067] Step S120: judging whether the information to be reviewed is safe to broadcast according to the compliance similarity of the vector to be reviewed;

[0068] Since the content to be reviewed provided by the present invention includes: media resources and live real-time streams, the requirements for the real-time degree of playback of the content from these two sources are completely different; therefore, when the compliance similarity is higher than the specified threshold, different strategies need to be adopted according to the different sources, specifically, Figure 1 The S120 part is shown:

[0069] If illegal content is detected in the live broadcast stream, the content needs to be replaced before broadcasting; if it is a media resource, it will be intercepted directly and further manual intervention will be required by staff.

[0070] The present invention is applied in the actual application scenario of IPTV. Through the user's input, the content produced by AI, and the content responded by AI, the text, picture, audio and video are matched to obtain the similarity of the characteristics of the materials to be reviewed and the judgment result of whether they pass or not. The multi-card inference cluster between the calculation responds quickly and can judge whether the content is compliant in a very short time. It improves the user's operating experience and the efficiency of safe production.

[0071] For example, when an IPTV user interacts with a large language model on a large screen, during the process of the user's text input and the large model's response to the user's language, the actual content of the user's input is reviewed. If sensitive words and illegal words appear, it will prompt that the current question cannot be answered correctly.

[0072] If it is an audit of AIGC content when it is published or created, users use prompt words to create and submit pictures, audio, and video content in AI capabilities. The large model responds to the user's prompt words in a multimodal manner and determines whether the content meets security requirements by traversing the feature library.

[0073] Through the multimodal security fence method provided by the present invention, on a Tesla T4 graphics card, it is possible to determine whether new media resources or live streams are in violation within 20 milliseconds, thereby improving the audit efficiency and increasing the detection coverage to more than 99%.

[0074] The above disclosures are only several specific embodiments of the present invention; however, the present invention is not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A multi-modal safety fence method based on vector matching, characterized in that: The following steps are involved: Obtaining information to be reviewed, loading a pre-trained model set, vectorizing the information to be reviewed, and generating a vector to be reviewed; the information to be reviewed includes a live real-time stream and newly added media files; Establishing a connection with a violation vector database set, wherein the violation vector database set includes vector features of multiple types of violation data; According to the violation vector database set, the compliance similarity of the vector to be reviewed is obtained to determine whether the information to be reviewed is safe to broadcast; Among them, the pre-trained model set consists of a text vector model, a picture vector model and a timbre vector model, which are used to generate vector information from various types of information to be reviewed; the types of information to be reviewed include: text, picture, audio and video.

2. The multi-modal safety fence method according to claim 1, characterized in that: The multiple types of illegal data include: text, picture, audio and video.

3. The multi-modal safety fence method according to claim 2, characterized in that: When the type is text, the vectorization processing is implemented by text vectorization; The text vectorization refers to: processing the text content through a text vector model to obtain a high-dimensional vector representation; The text vector model is constructed based on semantic vectors, Chinese STS dataset and self-annotated data training.

4. The multi-modal safety fence method according to claim 3, characterized in that: When the type is a picture, the vectorization processing includes: cropping and converting the picture to generate a pre-recognized picture, loading a picture vector model, and generating a vector representation of the pre-recognized picture; wherein the picture vector model is generated by training using the Flickr30K-CN data set and self-annotated data; When the type is a picture, the vectorization processing further includes: identifying text information of the information to be reviewed, performing text vectorization, and generating a text-based vector representation.

5. The multi-modal safety fence method according to claim 3, characterized in that: When the type is audio, the vectorization processing includes: performing speech recognition on the audio to generate text information; performing text vectorization on the text information to generate a text-based vector representation; When the type is audio, the vectorization processing further includes: loading a timbre vector model, performing timbre extraction on the audio, and obtaining a vectorized representation of the timbre.

6. The multi-modal safety fence method according to claim 5, characterized in that: The timbre extraction means: detecting voice activity on the audio, splitting the voice segments of multiple speakers, extracting the voiceprints of the speakers in the voice segments, and obtaining a vectorized representation of the timbre by the timbre vector model; The timbre vector model is a voiceprint recognition model trained based on a sensitive data set, and the content of the sensitive data set is the voiceprint features of predefined sensitive sounds.

7. The multi-modal safety fence method according to claim 3, characterized in that: When the type is video, the vectorization processing includes: Acquire scene features of the video, and obtain vector representation for the scene features; Obtaining a picture change frame in the video, obtaining a scene feature of the picture change frame, and obtaining a vector representation for the scene feature; When obtaining a vector representation for the scene feature, the type is converted into a picture and vectorization processing is performed.

8. The multi-modal safety fence method according to claim 7, characterized in that: Obtaining the picture change frame in the video means: Extract the HSV color space of the specified frame of the video and compare it with the previous frame using the pixel value difference method; Obtain the average difference of the corresponding pixels of hue, saturation and brightness, and sum the corresponding weights of the differences of hue, saturation and brightness to generate a feature difference; When the feature difference value is greater than a specified threshold, the specified frame of the video is a picture change frame.

9. The multi-modal safety fence method according to claim 7, characterized in that: The obtaining of the compliance similarity of the vector to be reviewed is achieved by vector matching; The vector matching uses cosine similarity to measure the similarity between two embedded vectors, expressed as: Among them, e1 and e2 are two embedded vectors respectively.

10. The multi-modal safety fence method according to claim 7, characterized in that: Determining whether the information to be reviewed is safe to broadcast includes: if the compliance similarity is higher than the specified threshold, taking different strategies according to the different sources; if illegal content is detected in the live real-time stream, the content is replaced before broadcasting; if it is a media resource, it is directly intercepted and further manually intervened by staff.

Images