Video processing method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a video processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: under the condition that a target picture exists in a to-be-detected video, based on a multi-modal content understanding result of the to-be-detected video, extracting each target video frame in the to-be-detected video, and extracting a target picture position and target picture information of each target video frame; the target picture is a picture which is not allowed to be released by each media release platform; inputting each target video frame and the target picture position and the target picture information corresponding to each target video frame into a generative large language model to obtain a target picture processing mode for each target video frame; and processing the target picture in each target video frame according to the target picture processing mode corresponding to each target video frame, and generating a processed video corresponding to the to-be-detected video. By adopting the method, the video can be accurately audited efficiently.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a video processing method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] In the field of Internet advertising, advertisers need to ensure that the advertising content in the video is creative and meets the review requirements of the media delivery platform, so that when the video is delivered on the media delivery platform, it can pass the review smoothly and attract more attention.

[0003] At present, the traditional video review method is manual review. Under the manual review method, it is often the case that the video is approved by the manual review but rejected by the media delivery platform, which makes advertisers face the loss of manpower due to delayed review. At the same time, due to the ever-changing review standards of the media delivery platform, manual review often cannot keep up with the speed of change of the review standards, and cannot conduct comprehensive and accurate review.

[0004] Therefore, there is a problem in traditional technologies that they cannot efficiently and accurately review the videos that are released. Summary of the invention

[0005] Based on this, it is necessary to provide a video processing method, device, computer equipment, computer-readable storage medium and computer program product that can improve the accuracy of the review of videos to be released in response to the above technical problems.

[0006] A video processing method, comprising:

[0007] In the case where there is a target screen in the video to be detected, based on the multimodal content understanding result of the video to be detected, each target video frame in the video to be detected is extracted, and the target screen position and target screen information of each target video frame are extracted; the target screen is a screen that is not allowed to be released by each media delivery platform;

[0008] Inputting each target video frame and the target picture position and target picture information corresponding to each target video frame into the generative large language model to obtain a target picture processing method for each target video frame;

[0009] According to the target picture processing method corresponding to each target video frame, the target picture in each target video frame is processed to generate a processed video corresponding to the video to be detected.

[0010] In one embodiment, the method further comprises:

[0011] Extracting semantic content texts of the video to be detected in different modes from the multimodal content understanding results of the video to be detected;

[0012] A first search sentence generated based on the semantic content text is input into a pre-constructed target text vector library. When a target text vector matching the first search sentence is retrieved, it is determined that a target picture exists in the video to be detected. The pre-constructed target text vector library stores multiple target text vectors, and the multiple target text vectors are hierarchically stored according to a multi-level index structure.

[0013] In one embodiment, the method further comprises:

[0014] Obtain the review results of new video materials returned by each media delivery platform, and extract the target word library and target picture library from the review results of the new video materials;

[0015] Based on the multimodal content understanding results of the target vocabulary and target image library, the target text vector set is extracted;

[0016] The target text vectors in the target text vector set are formed into a multi-level index structure and stored in the target text vector library.

[0017] In one embodiment, the method further comprises:

[0018] Extracting semantic content texts of the video to be detected in different modes from the multimodal content understanding results of the video to be detected;

[0019] According to the semantic text content, a first prompt word is generated to instruct the generative large language model to recognize the target screen feature;

[0020] A target picture feature set is obtained, and a first prompt word, a video to be detected and a target picture feature set are input into a generative large language model, so that the target picture recognition function of the generative large language model is indicated by the first prompt word, and based on each video frame of the video to be reviewed and the target picture feature set, it is identified whether the video to be detected has a target picture.

[0021] In one embodiment, the method further comprises:

[0022] Obtain the review results of new video materials returned by each media delivery platform, and extract keyword summaries targeting target screen features from the review results of new video materials;

[0023] Input the keyword summary into the generative large language model for few-shot learning to extract the characteristic keywords of each target screen;

[0024] Each target screen feature keyword is associated with each other by establishing a knowledge graph, so that each target screen feature keyword is stored in a target screen feature set in a multi-level index structure.

[0025] In one embodiment, each target video frame and the target screen position and target screen information corresponding to each target video frame are input into a generative large language model to obtain a target screen processing method for each target video frame, including:

[0026] Extracting target picture features and target text features of each target video frame from the multimodal content understanding results of the video to be detected;

[0027] Inputting a second search sentence generated based on the target screen feature and the target text feature into a pre-built target screen processing rule library, and extracting a second prompt word instructing the generative large language model to judge the target screen processing method from the target screen processing rule when a target screen processing rule matching the second search sentence is retrieved;

[0028] The second prompt word is input into the generative large language model, so as to use the second prompt word to instruct the generative large language model to determine a target picture processing method for each target video frame.

[0029] In one embodiment, the target picture in each target video frame is processed according to the target picture processing method corresponding to each target video frame to generate a processed video corresponding to the video to be detected, including:

[0030] When the target picture processing method is content replacement, the target picture position and target picture information corresponding to each target video frame are input as reference samples into a pre-built content replacement method library to obtain a target content replacement method matching each target video frame;

[0031] Generate target picture replacement content for each target video frame according to the target content replacement method through a generative artificial intelligence model;

[0032] The target screen replacement content of each target video frame is placed at the target screen position of each target video frame, and the synthesized video is rendered.

[0033] A video processing device, comprising:

[0034] The extraction module is used to extract each target video frame in the video to be detected based on the multimodal content understanding result of the video to be detected, and to extract the target picture position and target picture information of each target video frame when there is a target picture in the video to be detected; the target picture is a picture that is not allowed to be released by each media delivery platform;

[0035] An input module, used for inputting each target video frame and the target picture position and target picture information corresponding to each target video frame into the generative large language model to obtain a target picture processing method for each target video frame;

[0036] The replacement module is used to process the target picture in each target video frame according to the target picture processing method corresponding to each target video frame, and generate a processed video corresponding to the video to be detected.

[0037] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0038] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0039] A computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0040] The above-mentioned video processing method, device, computer equipment, storage medium and computer program product, by extracting each target video frame in the video to be detected based on the multimodal content understanding result of the video to be detected when there is a target picture in the video to be detected, and extracting the target picture position and target picture information of each target video frame; the target picture is a picture that is not allowed to be released by each media delivery platform; each target video frame and the target picture position and target picture information corresponding to each target video frame are input into the generative large language model to obtain a target picture processing method for each target video frame; according to the target picture processing method corresponding to each target video frame, the target picture in each target video frame is processed to generate a processed video corresponding to the video to be detected; in this way, based on the multimodal content understanding result of the video to be detected, the video to be detected can be fully identified in real time by context understanding. The text, pictures and sounds in the video can be accurately extracted to accurately extract the target video frame with problems and the picture position and picture information of the target picture with problems in the target video frame, thereby improving the accuracy of the review of the video to be released. Furthermore, the target picture processing method for the target video frame can be determined based on the target picture position and target picture information of the target video frame through the generative large language model, and the target picture in the target video frame is processed according to the target picture processing method of the target video frame, thereby simulating human operation to automatically process the characteristic parts of the target picture, such as coding, deletion, partial redrawing or content replacement, which not only reduces manual intervention and improves the modification efficiency of the video to be detected, but also ensures the naturalness and continuity of the picture content of the video to be detected after masking or replacement, which is helpful for the processed video to pass the review when it is released to the media delivery platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 A diagram of an application environment of a video processing method in an embodiment;

[0043] Figure 2 is a flow chart of a video processing method in one embodiment;

[0044] Figure 3 A schematic diagram of a process of performing automatic pre-detection on a video to be detected in one embodiment;

[0045] Figure 4 is a schematic diagram of a video detection result in an embodiment;

[0046] Figure 5 It is a schematic diagram of a detection result visualization interface corresponding to a pre-automatic detection in an embodiment;

[0047] Figure 6 A schematic diagram of a display page for description information of rejection characteristics of a sensitive word in an embodiment;

[0048] Figure 7 A schematic diagram of a display page of feature description information of rejection of a violation screen in an embodiment;

[0049] Figure 8 A schematic diagram of a flow chart of a method for processing an illegal screen in one embodiment;

[0050] Fig. 9 A schematic diagram of an automatic page processing in one embodiment;

[0051] Fig.10 is a flow chart of a video processing method in another embodiment;

[0052] Fig.11 is a structural block diagram of a video processing method device in one embodiment;

[0053] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] The video processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. When there is a target screen in the video to be detected, the server 104 extracts each target video frame in the video to be detected based on the multimodal content understanding result of the video to be detected, and extracts the target screen position and target screen information of each target video frame; the target screen is a screen that is not allowed to be released by each media delivery platform; the server 104 inputs each target video frame and the target screen position and target screen information corresponding to each target video frame into the generative large language model to obtain the target screen processing method for each target video frame; the server 104 processes the target screen in each target video frame according to the target screen processing method corresponding to each target video frame, and generates a processed video corresponding to the video to be detected. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0056] In an exemplary embodiment, Figure 2 As shown, a video processing method is provided, which is applied to Figure 1 The server 14 in the example is used as an example to illustrate, including the following steps S202 to S206. Among them:

[0057] S202, when there is a target picture in the video to be detected, based on the multimodal content understanding result of the video to be detected, extract each target video frame in the video to be detected, and extract the target picture position and target picture information of each target video frame; the target picture is a picture that is not allowed to be released by each media delivery platform.

[0058] The video to be detected may refer to a video that needs to be delivered on various media delivery platforms. The media delivery platform needs to review the video before publishing it to determine whether there is sensitive information in the video. Before submitting the video to the media delivery platform, the advertiser can use the video processing method of the present application to pre-review the video, and then submit the video that has passed the review to the media delivery platform to reduce the probability of the video being rejected by the media delivery platform.

[0059] Among them, the target screen may refer to the illegal screen, which may be the illegal screen that does not meet the review standards of the media delivery platform, such as "duplicate screens", "fake pop-ups", "fake buttons", "fake arrows", "private parts", etc. The illegal screen may also refer to the screen that contains sensitive words such as "amount", "information can be viewed at will", "casual chat", "bath", "automatically add friends", etc., which are prohibited from voting, or the screen may contain sensitive words with incomplete descriptions such as "next to", or sensitive words that violate public order and good morals.

[0060] Among them, the multimodal content understanding result of the video to be detected may refer to the multimodal feature information obtained after comprehensive recognition of the text, images and sounds in the video to be detected by using various technologies such as automatic speech recognition (ASR), optical character recognition (OCR), and artificial intelligence generated content understanding (AIGC).

[0061] The target video frame may refer to a video frame having an illegal picture.

[0062] The target picture position may refer to the picture position of the offending picture in the offending video frame.

[0063] The target screen information may be text information describing features of the illegal screen.

[0064] Among them, the media delivery platform may be a platform for playing videos delivered by advertisers.

[0065] Optionally, the server can use multiple technologies such as automatic speech recognition (ASR), optical character recognition (OCR), artificial intelligence generated content understanding (AIGC), etc. to comprehensively identify the text, pictures and sounds in the video to be detected, and obtain a multimodal content understanding result of the video to be detected. When there are illegal pictures in the video to be detected, based on the multimodal content understanding result of the video to be detected, the illegal video frames in the video to be detected are extracted, and the illegal picture positions and illegal picture information of each illegal video frame are extracted.

[0066] Such multimodal content recognition method as step S202 can not only detect explicit illegal content, but also identify implicit rejection points in the picture through context understanding by analyzing multiple levels and features of video content, including but not limited to scenes, objects, character behaviors, and semantic content of text and voice. Explicit illegal content refers to illegal content with clear vocabulary or picture features, which can be easily detected through rules; invisible rejection points may refer to content that requires previous and subsequent associations or appears in different pictures. Since these are not easy to be found, they need to be detected with the help of context understanding technology.

[0067] For example, if a person in a video is extremely excited, swallowing or spitting, and the body is twisted and out of control, the method of this application will use image recognition technology and semantic understanding to comprehensively identify whether the individual is drinking a beverage product or under the influence of banned drugs. The former may be approved due to excitement and joy, while the latter may be rejected for displaying ugly behavior. At this time, it is necessary to distinguish the rejection features based on the context with actions and sounds that are appropriate or provocative. The method of this application will mark the video content as a possible violation risk. Common detection tools on the market do not use semantic understanding. As long as sensitive words appear, the video is judged to be in violation and a one-size-fits-all treatment is made. The method of this application can deeply judge the violation of the video from multiple dimensions through multimodal content understanding, thereby avoiding a one-size-fits-all treatment.

[0068] Step S204 , inputting each target video frame and the target picture position and target picture information corresponding to each target video frame into the generative large language model to obtain a target picture processing method for each target video frame.

[0069] The generative large language model may be an LLM (Large Language Model) model.

[0070] Among them, the target picture processing method for each target video frame may refer to a processing method for the illegal picture in each illegal video frame. In each embodiment of the present application, the target picture processing method may be to code or delete the illegal picture, or to use AIGC technology to partially redraw or replace the content of the illegal picture, so as to effectively process the illegal content while maintaining the naturalness and continuity of the video content.

[0071] Optionally, the server inputs each illegal video frame and the illegal picture position and illegal picture information corresponding to each illegal video frame into the generative large language model to obtain the illegal picture processing method for each illegal video frame. For example, if the illegal video frame contains words such as "most" and "first" that arbitrarily exaggerate the facts, the server inputs the illegal video frame, the illegal picture position where the words "most" and "first" appear in the illegal video frame, and the illegal picture information such as "most" and "first" into the generative large language model to obtain the processing method for the words "most" and "first" in the illegal video frame. Specifically, the words "most" and "first" can be coded.

[0072] S206 , processing the target picture in each target video frame according to the target picture processing method corresponding to each target video frame, and generating a processed video corresponding to the video to be detected.

[0073] The processed video may refer to a new video obtained by rendering after blurring, deleting, partially redrawing or replacing the content of the illegal scenes in each illegal video in the video to be detected.

[0074] Optionally, the server processes the illegal pictures in each illegal video frame according to the illegal picture processing method corresponding to each illegal video frame to generate a processed video corresponding to the video to be detected. For example, the illegal picture processing method corresponding to the illegal video frame is Coding Processing When the illegal picture of the illegal video frame can be coded to obtain the coded video frame, the illegal picture processing method corresponding to the illegal video frame is as follows: Deletion Processing When the illegal picture of the illegal video frame can be deleted, the video frame after the illegal picture is deleted is obtained, and the illegal picture processing method corresponding to the illegal video frame is Partial redrawing When the illegal picture of the illegal video frame is partially redrawn to obtain a video frame after partial redrawing, when the illegal picture processing method corresponding to the illegal video frame is to generate content replacement processing, content can be generated for the illegal picture of the illegal video frame, and the illegal picture is replaced with the generated content to obtain a video frame after content replacement.

[0075] In the above-mentioned video processing method, when there is a target picture in the video to be detected, based on the multimodal content understanding result of the video to be detected, each target video frame in the video to be detected is extracted, and the target picture position and target picture information of each target video frame are extracted; the target picture is a picture that is not allowed to be released by each media delivery platform; each target video frame and the target picture position and target picture information corresponding to each target video frame are input into the generative large language model to obtain a target picture processing method for each target video frame; according to the target picture processing method corresponding to each target video frame, the target picture in each target video frame is processed to generate a processed video corresponding to the video to be detected; in this way, based on the multimodal content understanding result of the video to be detected, the text, pictures and Sound, thereby accurately extracting the target video frame with problems and the picture position and picture information of the target picture with problems in the target video frame, thereby improving the accuracy of the review of the video to be released. Furthermore, the target picture processing method for the target video frame can be determined based on the target picture position and target picture information of the target video frame through the generative large language model, and the target picture in the target video frame is processed according to the target picture processing method of the target video frame, for example, coding, deletion, partial redrawing or content replacement, thereby simulating human operation to automatically process the characteristic parts of the target picture, which not only reduces manual intervention and improves the modification efficiency of the video to be detected, but also ensures the naturalness and continuity of the picture content of the video to be detected after masking or replacement, which helps to pass the review when the processed video is released to the media delivery platform.

[0076] In actual application, it is determined whether the target picture exists in the video to be detected by pre-automatic detection, and after it is determined that the target picture exists in the video to be detected, the above steps S202 to S206 are executed.

[0077] In an exemplary embodiment, the method also includes: extracting semantic content text of the video to be detected in different modalities from the multimodal content understanding results of the video to be detected; inputting a first search statement generated based on the semantic content text into a pre-constructed target text vector library, and when a target text vector matching the first search statement is retrieved, determining that a target picture exists in the video to be detected; the pre-constructed target text vector library stores multiple target text vectors, and the multiple target text vectors are hierarchically stored according to a multi-level index structure.

[0078] The semantic content text of the video to be detected in different modes may include the semantic content text of the video to be detected in modes such as text, picture and sound.

[0079] Among them, the first search statement is a query statement generated based on the semantic content text of the video to be detected in the modes of text, picture and sound, etc., and is used to search in a pre-built target text vector library.

[0080] The pre-built target text vector library may refer to a pre-built illegal text vector library, in which a plurality of illegal text vectors are stored, and the plurality of illegal text vectors are hierarchically stored according to a multi-level index structure.

[0081] Among them, the target text vector that matches the first search statement may refer to the illegal text vector that matches the first search statement, and matching means that the text vector similarity is greater than a preset threshold. In practical applications, RAG (Retrieval-Augmented Generation) technology can be used for retrieval matching. RAG retrieval-augmented generation technology is an artificial intelligence technology that combines information retrieval technology with language generation models. This technology retrieves relevant information from an external knowledge base and inputs it as a prompt to a large language model (LLMs) to enhance the model's ability to handle knowledge-intensive tasks, such as question-answering, text summarization, and content generation.

[0082] Among them, the multi-level index structure can divide multiple illegal text vectors into multiple levels to form multi-level retrieval topics, which is conducive to improving retrieval efficiency.

[0083] Optionally, before step S202, it is necessary to perform a pre-detection on the video to be detected. For details, see Figure 3 When a new video to be detected is uploaded, multiple technologies such as automatic speech recognition (ASR), optical character recognition (OCR), and artificial intelligence generated content understanding (AIGC) are used to understand the multimodal content of the video to be detected. The semantic content text of the video to be detected in different modes is extracted from the multimodal content understanding results of the video to be detected. Then, the first search statement Query1 generated based on the semantic content text is input into the pre-built illegal text vector library, and the RAG (Retrieval-Augmented Generation) technology is used for retrieval matching. When the illegal text vector matching the first search statement Query1 is retrieved, it is determined that the video to be detected has illegal scenes, that is, there is high-risk content, and it is determined whether the AIGC technology needs to be used to generate content to replace the illegal scenes, or whether the automatic coding technology needs to be used to code the illegal scenes.

[0084] In actual applications, when illegal features are detected, see Figure 4It will associate all the rejected problem frames of the video to be tested on the advertising media platform, down to the second, the copy in the picture or the text information in the background, and list the processing suggestions and review points, and send an alarm notification to the corresponding designer's IM software.

[0085] Figure 5 An exemplary visualization interface for detection results corresponding to pre-automatic detection is provided. In actual applications, by extracting historical video review results and sensitive words and illegal images in the review standards disclosed by various media delivery platforms, deep learning technology is used to perform real-time analysis of videos. The pre-review results of different media delivery platforms can be given for the same video to be delivered. According to the review tendencies of different media delivery platforms, different levels of review suggestions can be provided to advertisers, such as "safe to invest, can be tried, high probability of rejection", and a visual display can be performed. The video can be automatically reviewed before it is released, thereby realizing pre-review of the video and effectively controlling the risks before release. Figure 5 The interface shown displays the risk level distribution, the total number of sensitive words, the characteristic distribution of sensitive words, the total number of illegal pictures, the characteristic distribution of illegal pictures, the review of sensitive words by each media delivery platform, and the review of illegal pictures by each media delivery platform.

[0086] Figure 6 An exemplary display page for description information of rejection characteristics of sensitive words is provided. Figure 7 An exemplary display page for description information of rejected features of illegal images is provided. Figure 6 and 7 Each rejection feature information on the display page includes the risk level, feature label, keywords, rejection case, rejection source, audit points, audit solution, uploader, upload time and other information of a rejection feature. The risk level can be "high", "medium" or "low". When the risk level is "high", it means that there is a high probability of rejection. When the risk level is "medium", it means that you can try to review it. When the risk level is "low", it means that you can ignore it but be vigilant. The feature label can be "banned", "public order and good morals", "incomplete description", "duplicate screen", "fake arrow", "fake button", etc., which are used to represent the audit standards violated by the rejection feature. Each feature label corresponds to multiple keywords. The rejection source refers to the media delivery platform that rejected the review. The audit points refer to the rejection reasons provided by the media delivery platform that rejected the review. The audit solution provides a processing method for the rejection feature.

[0087] In this embodiment, the semantic content text of the video to be detected in different modes is extracted from the multimodal content understanding results of the video to be detected; the first search sentence generated based on the semantic content text is input into a pre-constructed target text vector library, and when a target text vector matching the first search sentence is retrieved, it is determined that the video to be detected has a target screen; the pre-constructed target text vector library stores a plurality of target text vectors, and the plurality of target text vectors are hierarchically stored according to a multi-level index structure; in this way, the first search sentence can be generated based on the semantic content text of the video to be detected in different modes, and search and match can be performed in the pre-constructed target text vector library. Since the target text vectors in the pre-constructed target text vector library are stored in a multi-level index structure, the search and matching efficiency can be improved, thereby quickly determining whether the video to be detected has a target screen.

[0088] In an exemplary embodiment, the method also includes: obtaining the review results of new video materials returned by each media delivery platform, and extracting a target vocabulary and a target image library from the review results of the new video materials; extracting a target text vector set based on the multimodal content understanding results of the target vocabulary and the target image library; and forming a multi-level index structure for each target text vector in the target text vector set and storing it in the target text vector library.

[0089] The new video material review result may refer to a rejection result returned by the media delivery platform after the advertiser delivers the new video to the media delivery platform.

[0090] The target word library may refer to a violation word library, and the words in the violation word library are extracted from the review results of the video materials by various media delivery platforms.

[0091] The target gallery may refer to a violation gallery, and the pictures and videos in the violation gallery are obtained by summarizing the review results of the video materials by various media delivery platforms.

[0092] Among them, the multimodal content understanding results of the target vocabulary and target image library may refer to the multimodal feature information obtained after comprehensive recognition of text, images and sounds of the illegal vocabulary and illegal image library using a variety of technologies such as automatic speech recognition (ASR), optical character recognition (OCR), and artificial intelligence generated content understanding (AIGC).

[0093] The target text vector set may refer to a violation text vector set, in which a plurality of violation text vectors are stored.

[0094] The target text vector library may refer to a vector library formed by hierarchically storing the illegal text vectors in the illegal text vector set according to a multi-level index structure.

[0095] Optionally, the illegal text vector library is kept updated dynamically in real time. For details, see Figure 3 , obtain the new video material review results returned in real time by each media delivery platform, and extract the high-risk word library and rejected video / picture library from the new video material review results, then use automatic speech recognition (ASR), optical character recognition (OCR), artificial intelligence generated content understanding (AIGC) and other technologies to perform multimodal content understanding and recognition on the high-risk word library and rejected video / picture library, extract the illegal text, form an illegal text vector set, and form a multi-level topic index for each illegal text vector in the illegal text vector, which is stored in the illegal text vector library.

[0096] In this embodiment, the review results of new video materials returned by each media delivery platform are obtained, and a target vocabulary and a target picture library are extracted from the review results of the new video materials; based on the multimodal content understanding results of the target vocabulary and the target picture library, a target text vector set is extracted; each target text vector in the target text vector set is formed into a multi-level index structure and stored in the target text vector library; in this way, according to the review results of new video materials returned by each media delivery platform, new violation features fed back in the review results can be automatically extracted, new violation samples can be continuously learned to adapt to the constantly changing violation features, and the problem that manual review cannot keep up with the speed of change of the review standards of each media delivery platform can be solved. At the same time, when the video is delivered to each media delivery platform, a comprehensive and accurate pre-review of the video can be performed.

[0097] In an exemplary embodiment, the method also includes: extracting semantic content text of the video to be detected in different modalities from the multimodal content understanding results of the video to be detected; generating a first prompt word instructing the generative large language model to identify target screen features based on the semantic text content; obtaining a target screen feature set, and inputting the first prompt word, the video to be detected and the target screen feature set into the generative large language model, so as to use the first prompt word to indicate the target screen recognition function of the generative large language model, and identify whether the video to be detected has a target screen based on each video frame of the video to be reviewed and the target screen feature set.

[0098] Among them, the first prompt word can be a prompt word prompt1 used to instruct the generative large language model to identify the features of the illegal pictures. The generative large language model can identify the illegal video frames containing illegal pictures in the video to be detected based on the prompt word prompt1.

[0099] The target screen feature may refer to an illegal screen feature.

[0100] The target screen recognition function may refer to an illegal screen recognition function.

[0101] The target picture feature set may refer to a violation picture feature set consisting of a plurality of violation picture features.

[0102] Optionally, before step S202, it is necessary to perform a pre-detection on the video to be detected. For details, see Figure 3 When a new video to be detected is uploaded, a variety of technologies such as automatic speech recognition (ASR), optical character recognition (OCR), and artificial intelligence generated content understanding (AIGC) are used to understand the multimodal content of the video to be detected. The semantic content text of the video to be detected in different modes is extracted from the multimodal content understanding results of the video to be detected, and the first prompt word prompt1 for instructing the generative large language model to identify the features of the illegal screen is generated according to the semantic text content. Then, the server obtains the target screen feature set, and inputs the first prompt word prompt1, the video to be detected, and the illegal screen feature set into the generative large language model. Prompt1 is used to indicate the illegal screen recognition function of the generative large language model. Based on each video frame of the video to be reviewed and the illegal screen feature set, it is identified whether the video to be detected has illegal screens, that is, whether there is high-risk content. In the case of high-risk content, an alarm notification is sent, and it is determined whether the AIGC technology needs to be used to generate content to replace the illegal screen, or whether the automatic coding technology needs to be used to code the illegal screen.

[0103] In this embodiment, the semantic content text of the video to be detected in different modes is extracted from the multimodal content understanding result of the video to be detected; according to the semantic text content, a first prompt word is generated to instruct the generative large language model to identify the target screen feature; the target screen feature set is obtained, and the first prompt word, the video to be detected and the target screen feature set are input into the generative large language model, so as to use the first prompt word to indicate the target screen recognition function of the generative large language model, and identify whether the video to be detected has a target screen based on each video frame of the video to be reviewed and the target screen feature set; in this way, the first prompt word can be generated based on the semantic content text of the video to be detected in different modes, and the generative large language model can automatically identify whether the video to be detected has a target screen under the prompt of the first prompt word, thereby improving the detection accuracy and efficiency of the target screen.

[0104] In an exemplary embodiment, the method also includes: obtaining the review results of new video materials returned by each media delivery platform, and extracting keyword summaries for target screen features from the review results of the new video materials; inputting the keyword summaries into a generative large language model for few-sample learning to extract keywords for each target screen feature; associating each target screen feature keyword by establishing a knowledge graph to store each target screen feature keyword in a target screen feature set in a multi-level index structure.

[0105] The keyword summary may refer to a keyword summary of illegal screen features extracted from the review results of new video material.

[0106] The target screen feature keyword may refer to a keyword corresponding to the illegal screen feature.

[0107] Optionally, the target screen feature set is kept updated in real time and dynamically. Specifically, after the video is delivered to each media delivery platform, each media delivery platform will detect the video. For a media delivery platform, when there is illegal content in the video, the media delivery platform will refuse to review it and provide reasons for the rejection. The server obtains the reasons for rejection returned by each media delivery platform, and extracts a keyword summary for the illegal screen features from the reasons for rejection returned by each media delivery platform, and inputs the keyword summary as a few-shot into the generative large language model for few-shot learning, learns the keywords in the few-shot samples, and then establishes a knowledge graph through the ability to establish a knowledge base graph, associates each keyword, and stores each illegal screen feature keyword in the illegal screen feature set in a multi-level index structure.

[0108] In this embodiment, by obtaining the review results of new video materials returned by each media delivery platform, and extracting keyword summaries for target screen features from the review results of new video materials; inputting the keyword summaries into the generative large language model for few-sample learning, and extracting keywords for each target screen feature; associating the keywords for each target screen feature by establishing a knowledge graph, so as to store the keywords for each target screen feature in a target screen feature set in a multi-level index structure; in this way, the new target screen features fed back in the review results can be automatically extracted according to the review results of new video materials returned by each media delivery platform, and new violation samples can be continuously learned to adapt to the constantly changing violation features, and the problem that manual review cannot keep up with the speed of change of the review standards of each media delivery platform can be solved. When the video is delivered to each media delivery platform, a comprehensive and accurate pre-review of the video can be performed. At the same time, by associating the keywords for each target screen feature by establishing a multi-level index structure, the subsequent retrieval efficiency can be improved, and new violation samples can also be quickly classified and relationships can be established, which is conducive to context retrieval.

[0109] In an exemplary embodiment, each target video frame and the target screen position and target screen information corresponding to each target video frame are input into a generative large language model to obtain a target screen processing method for each target video frame, including: extracting target screen features and target text features of each target video frame from the multimodal content understanding results of the video to be detected; inputting a second search sentence generated based on the target screen features and the target text features into a pre-built target screen processing rule library, and when a target screen processing rule matching the second search sentence is retrieved, extracting a second prompt word from the target screen processing rule to instruct the generative large language model to determine the target screen processing method; and inputting the second prompt word into the generative large language model to use the second prompt word to instruct the generative large language model to determine the target screen processing method for each target video frame.

[0110] The target picture features and the target text features of the target video frame may refer to the illegal picture features and the illegal text features of the illegal video frame.

[0111] The second search statement may be a query statement generated based on the illegal screen processing rule, and is used to search in a pre-built target screen processing rule library.

[0112] The target screen processing rule base may be a rule base formed by a plurality of illegal screen processing rules.

[0113] The target screen processing rule that matches the second search statement may refer to the illegal screen processing rule that matches the second search statement, and matching means that the similarity of the illegal screen processing rule is greater than a preset threshold.

[0114] The second prompt word may be a prompt word prompt2 used to instruct the generative large language model to determine a processing method for the offending screen. The generative large language model may determine a processing method for the offending screen based on the prompt word prompt2.

[0115] Among them, the target screen processing method can be an illegal screen processing method, a coding replacement method, or a content replacement method. The coding replacement method refers to coding the illegal screen, and the content replacement method refers to generating content to replace the illegal screen.

[0116] Optionally, when it is determined that the video to be detected has an illegal picture, it is necessary to input each illegal video frame and the illegal picture position and illegal picture information corresponding to each illegal video frame into the generative large language model to obtain the illegal picture processing method for each illegal video frame. For details, see Figure 8, extract the illegal picture features and illegal text of each illegal video frame from the multimodal content understanding results of the video to be detected, then input the illegal text into a pre-built illegal text vector library to obtain an index matching the illegal text, then generate a second search statement based on the index matching the illegal text and the illegal picture features, input the second search statement into a pre-built illegal picture processing rule library, and when the illegal picture processing rule matching the second search statement is retrieved, extract the second prompt word prompt2 for instructing the generative large language model to judge the illegal picture processing method from the illegal picture processing rule, then input the second prompt word prompt2 into the generative large language model, so as to use the second prompt word prompt2 to indicate the picture processing decision function of the generative large language model to determine the illegal picture processing method for each illegal video frame, wherein the picture processing decision function is a function of the generative large language model, which is used to determine the processing method of the target picture, which can be understood as the illegal picture processing method judgment function, which is used to judge the processing method of the illegal picture.

[0117] In this embodiment, target picture features and target text features of each target video frame are extracted from the multimodal content understanding results of the video to be detected; a second search sentence generated based on the target picture features and the target text features is input into a pre-constructed target picture processing rule library, and when a target picture processing rule matching the second search sentence is retrieved, a second prompt word instructing the generative large language model to determine the target picture processing method is extracted from the target picture processing rule; the second prompt word is input into the generative large language model, so as to use the second prompt word to instruct the generative large language model to determine the target picture processing method for each target video frame; in this way, the appropriate target picture processing method can be accurately matched based on context understanding, and better video processing effects can be flexibly achieved according to the styles of different videos.

[0118] In an exemplary embodiment, the target picture in each target video frame is processed according to the target picture processing method corresponding to each target video frame to generate a processed video corresponding to the video to be detected, including: when the target picture processing method is content replacement, the target picture position and target picture information corresponding to each target video frame are input as reference samples into a pre-built content replacement method library to obtain a target content replacement method matching each target video frame; a target picture replacement content for each target video frame is generated according to the target content replacement method through a generative artificial intelligence model; the target picture replacement content of each target video frame is placed at the target picture position of each target video frame, and the synthesized processed video is rendered.

[0119] The content replacement method library may be a vector library storing a plurality of content replacement methods.

[0120] The target content replacement method may refer to a method for replacing illegal content. In actual applications, the method for replacing illegal content may be to generate text for replacement or to generate an image for replacement.

[0121] Among them, the target screen replacement content may refer to the illegal screen replacement content. For example, when the illegal screen of "skirt is too short" appears, the content with the same pattern as the skirt may be generated to cover the hem of the skirt in the screen. The content with the same pattern as the skirt is the replacement content of the illegal screen.

[0122] Optionally, when the illegal picture processing method for each illegal video frame is determined, if the corresponding illegal picture processing method is to generate content replacement, the illegal picture can be replaced by generating image content replacement or generating text content replacement. By inputting the illegal picture position and illegal picture information corresponding to each illegal video frame as reference samples into a pre-built content replacement method library, the content replacement method matching each illegal video frame can be obtained as generating image content replacement or generating text content replacement, see Figure 8 In the case where the content replacement method that matches each illegal video frame is to generate image content replacement, ComfyUI is used to generate image content, and the illegal screen content is replaced with the generated image content. In the case where the content replacement method that matches each illegal video frame is to generate text content replacement, LLM's multimodal content understanding function is used to generate text content, and the illegal screen content is replaced with the generated text content, and the synthesized video is rendered.

[0123] For example, if Fig. 9 As shown, if the character in the video is wearing a revealing short skirt and involves a rejection point, it can be understood that the skirt is too short and the skirt is automatically extended to below the knee, or a foreground is generated for partial covering, while keeping the other features and background of the character unchanged. It can not only repair or replace the illegal picture, but also create new visual effects without destroying the overall beauty of the video and the narrative coherence. This automated processing method not only improves processing efficiency, but also reduces manual intervention, while ensuring the naturalness and coherence of the video content after covering or replacing. Therefore, through the video processing method of this embodiment, advertisers can ensure that their videos can pass the review smoothly when they are delivered to the advertising media platform and obtain higher traffic.

[0124] In actual applications, in addition to generating content replacement, target screen processing also includes coding processing. The coding processing method is to directly code the illegal screen and then render the synthesized video.

[0125] like Figure 8The video processing flow shown can use context understanding algorithms combined with automated AIGC processing flows to automatically and accurately redraw or replace parts of the content when illegal content is detected, rather than simply blurring or deleting it. This maintains the naturalness and narrative coherence of the video content while processing illegal content, greatly improving user experience and content value.

[0126] In this embodiment, when the target screen processing method is content replacement, the target screen position and target screen information corresponding to each target video frame are input as reference samples into a pre-constructed content replacement method library to obtain a target content replacement method matching each target video frame; a target screen replacement content for each target video frame is generated according to the target content replacement method through a generative artificial intelligence model; the target screen replacement content of each target video frame is placed at the target screen position of each target video frame, and the synthesized processed video is rendered; in this way, the target screen position and target screen information corresponding to each target video frame can be given as reference samples to the content replacement method library for matching, so as to further accurately determine the specific content replacement method. Due to the specific target screen position and target screen information, the size, placement position and placement time period of the generated replacement content can be accurately determined. On the whole, on the basis of adopting a matching content replacement method, video synthesis can be accurately performed to improve the video quality of the processed video.

[0127] In another embodiment, Fig.10 As shown, a video processing method is provided, which is applied to Figure 1 Taking the server 104 in the example as an example, the following steps are included:

[0128] Step S1002, when there is a target screen in the video to be detected, based on the multimodal content understanding result of the video to be detected, extract each target video frame in the video to be detected, and extract the target screen position and target screen information of each target video frame; the target screen is a screen that is not allowed to be released by each media delivery platform.

[0129] Step S1004: extracting target picture features and target text features of each target video frame from the multimodal content understanding result of the video to be detected.

[0130] Step S1006, inputting a second search statement generated based on the target screen features and the target text features into a pre-constructed target screen processing rule library, and when a target screen processing rule matching the second search statement is retrieved, extracting a second prompt word from the target screen processing rule to instruct the generative large language model to determine a target screen processing method.

[0131] Step S1008: input the second prompt word into the generative large language model, so as to use the second prompt word to instruct the generative large language model to determine a target picture processing method for each of the target video frames.

[0132] Step S1010 , processing the target picture in each of the target video frames according to the target picture processing method corresponding to each of the target video frames, and generating a processed video corresponding to the video to be detected.

[0133] It should be noted that the specific definition of the above steps can refer to the specific definition of a video processing method above.

[0134] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0135] Based on the same inventive concept, the embodiment of the present application also provides a video processing device for implementing the above-mentioned video processing method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more video processing device embodiments provided below can refer to the limitations on the video processing method above, and will not be repeated here.

[0136] In an exemplary embodiment, Fig.11 As shown, a video processing device is provided, including: an extraction module 1102, an input module 11011 and a replacement module 1106, wherein:

[0137] The extraction module 1102 is used to extract each target video frame in the video to be detected based on the multimodal content understanding result of the video to be detected, and to extract the target picture position and target picture information of each target video frame; the target picture is a picture that is not allowed to be released by each media delivery platform;

[0138] An input module 11011 is used to input each target video frame and the target picture position and target picture information corresponding to each target video frame into a generative large language model to obtain a target picture processing method for each target video frame;

[0139] The replacement module 1106 is used to process the target picture in each target video frame according to the target picture processing method corresponding to each target video frame, and generate a processed video corresponding to the video to be detected.

[0140] In one of the embodiments, the device also includes: a detection module, which is used to extract semantic content text of the video to be detected in different modalities from the multimodal content understanding results of the video to be detected; input a first search statement generated based on the semantic content text into a pre-constructed target text vector library, and when a target text vector matching the first search statement is retrieved, it is determined that the video to be detected has a target picture; the pre-constructed target text vector library stores multiple target text vectors, and the multiple target text vectors are hierarchically stored according to a multi-level index structure.

[0141] In one of the embodiments, the detection module is also used to obtain the review results of new video materials returned by each media delivery platform, and extract a target vocabulary and a target image library from the review results of the new video materials; extract a target text vector set based on the multimodal content understanding results of the target vocabulary and the target image library; and form a multi-level index structure for each target text vector in the target text vector set and store it in the target text vector library.

[0142] In one of the embodiments, the device also includes: a detection module, which is used to extract the semantic content text of the video to be detected in different modes from the multimodal content understanding results of the video to be detected; generate a first prompt word instructing the generative large language model to identify the target screen features according to the semantic text content; obtain the target screen feature set, and input the first prompt word, the video to be detected and the target screen feature set into the generative large language model, so as to use the first prompt word to indicate the target screen recognition function of the generative large language model, and identify whether the video to be detected has the target screen based on each video frame of the video to be reviewed and the target screen feature set.

[0143] In one of the embodiments, the detection module is also used to obtain the review results of new video materials returned by each media delivery platform, and extract keyword summaries for target screen features from the review results of the new video materials; input the keyword summaries into the generative large language model for few-sample learning to extract keywords for each target screen feature; and associate each target screen feature keyword by establishing a knowledge graph to store each target screen feature keyword in a target screen feature set in a multi-level index structure.

[0144] In one of the embodiments, the input module 11011 is specifically used to extract target picture features and target text features of each target video frame from the multimodal content understanding results of the video to be detected; input a second search sentence generated based on the target picture features and the target text features into a pre-built target picture processing rule library, and when a target picture processing rule matching the second search sentence is retrieved, extract a second prompt word from the target picture processing rule to instruct the generative large language model to determine the target picture processing method; input the second prompt word into the generative large language model to use the second prompt word to instruct the generative large language model to determine the target picture processing method for each target video frame.

[0145] In one of the embodiments, the replacement module 1106 is specifically used to input the target screen position and target screen information corresponding to each target video frame as reference samples into a pre-built content replacement method library when the target screen processing method is content replacement, so as to obtain a target content replacement method matching each target video frame; generate a target screen replacement content for each target video frame according to the target content replacement method through a generative artificial intelligence model; place the target screen replacement content of each target video frame at the target screen position of each target video frame, and render the synthesized processed video.

[0146] Each module in the above video processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0147] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.12 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a video processing method is implemented.

[0148] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0149] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above-mentioned video processing method. The steps of the video processing method here may be the steps of the video processing method in each of the above-mentioned embodiments.

[0150] In one embodiment, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the above-mentioned video processing method. The steps of the video processing method here can be the steps of the video processing method in each of the above-mentioned embodiments.

[0151] In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the processor executes the steps of the above-mentioned video processing method. The steps of the video processing method here can be the steps of the video processing method in each of the above-mentioned embodiments.

[0152] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0153] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A video processing method, characterized in that: The method comprises: In the case where there is a target screen in the video to be detected, based on the multimodal content understanding result of the video to be detected, each target video frame in the video to be detected is extracted, and the target screen position and target screen information of each target video frame are extracted; the target screen is a screen that is not allowed to be released by each media delivery platform; Inputting each of the target video frames and the target picture position and target picture information corresponding to each of the target video frames into a generative large language model to obtain a target picture processing method for each of the target video frames; According to the target picture processing method corresponding to each target video frame, the target picture in each target video frame is processed to generate a processed video corresponding to the video to be detected.

2. The method according to claim 1, characterized in that The method further comprises: Extracting semantic content texts of the video to be detected in different modes from the multimodal content understanding results of the video to be detected; A first search statement generated based on the semantic content text is input into a pre-constructed target text vector library, and when a target text vector matching the first search statement is retrieved, it is determined that the target picture exists in the video to be detected; the pre-constructed target text vector library stores a plurality of target text vectors, and the plurality of target text vectors are hierarchically stored according to a multi-level index structure.

3. The method according to claim 2, characterized in that The method further comprises: Obtaining the new video material review results returned by each media delivery platform, and extracting a target word library and a target picture library from the new video material review results; Extracting a target text vector set based on the multimodal content understanding results of the target vocabulary and the target image library; The target text vectors in the target text vector set are formed into a multi-level index structure and stored in the target text vector library.

4. The method according to claim 1, characterized in that: The method further comprises: Extracting semantic content texts of the video to be detected in different modes from the multimodal content understanding results of the video to be detected; According to the semantic text content, generating a first prompt word instructing the generative large language model to recognize a target picture feature; A target picture feature set is obtained, and the first prompt word, the video to be detected and the target picture feature set are input into the generative large language model, so as to use the first prompt word to indicate the target picture recognition function of the generative large language model, and identify whether the target picture exists in the video to be detected based on each video frame of the video to be reviewed and the target picture feature set.

5. The method according to claim 4, characterized in that The method further comprises: Obtaining the new video material review results returned by each media delivery platform, and extracting a keyword summary for the target screen features from the new video material review results; Inputting the keyword summary into the generative large language model for few-sample learning to extract the characteristic keywords of each target screen; The target screen feature keywords are associated with each other by establishing a knowledge graph, so that the target screen feature keywords are stored in the target screen feature set in a multi-level index structure.

6. The method according to claim 1, characterized in that The step of inputting each of the target video frames and the target picture position and target picture information corresponding to each of the target video frames into a generative large language model to obtain a target picture processing method for each of the target video frames includes: Extracting target picture features and target text features of each target video frame from the multimodal content understanding result of the video to be detected; inputting a second search sentence generated based on the target picture feature and the target text feature into a pre-built target picture processing rule library, and extracting a second prompt word instructing the generative large language model to determine a target picture processing method from the target picture processing rule when a target picture processing rule matching the second search sentence is retrieved; The second prompt word is input into the generative large language model, so as to use the second prompt word to instruct the generative large language model to determine a target picture processing method for each of the target video frames.

7. The method according to claim 1, characterized in that The processing of the target pictures in each of the target video frames according to the target picture processing method corresponding to each of the target video frames to generate a processed video corresponding to the video to be detected includes: In the case where the target picture processing method is content replacement, the target picture position and target picture information corresponding to each target video frame are input as reference samples into a pre-built content replacement method library to obtain a target content replacement method matching each target video frame; Generate target picture replacement content for each target video frame according to the target content replacement method through a generative artificial intelligence model; The target screen replacement content of each target video frame is placed at the target screen position of each target video frame, and the processed video is rendered and synthesized.

8. A video processing device, characterized in that: The device comprises: An extraction module is used to extract each target video frame in the video to be detected based on the multimodal content understanding result of the video to be detected, and to extract the target picture position and target picture information of each target video frame when there is a target picture in the video to be detected; the target picture is a picture that is not allowed to be released by each media delivery platform; An input module, used for inputting each of the target video frames and the target picture position and target picture information corresponding to each of the target video frames into a generative large language model to obtain a target picture processing method for each of the target video frames; The replacement module is used to process the target picture in each of the target video frames according to the target picture processing method corresponding to each of the target video frames, and generate a processed video corresponding to the video to be detected.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images