An artificial intelligence-based video management method

Abstract

The present application belongs to the technical field of video management, and particularly relates to a video management method based on artificial intelligence. The method first extracts the interest value of the user during the process of watching the video through facial recognition and expression analysis, and determines the time frame of attention concentration. Then, the text features and scene features of the time frame and the key frames of the advertisement video are extracted, the cross-modal matching model is used for semantic alignment and matching score of the video and the advertisement, and the advertisements with high matching degree are screened and inserted into the video. The system further collects the interactive behavior data generated after the user watches, calculates the advertisement sticking score, dynamically adjusts the advertisement delivery strategy, and realizes the continuous optimization of the advertisement effect. The method improves the relevance of the advertisement and the video content and the user experience, and has the advantages of high precision and high adaptability.

Description

Technical Field

[0001] This invention belongs to the field of video management technology, and in particular relates to a video management method based on artificial intelligence. Background Technology

[0002] As video platforms accelerate their commercialization, ad placement has become a core issue in balancing platform revenue and user experience. It's not enough for ads to simply reach users; the quality of user interaction with ads must be accurately measured (e.g., continued engagement after clicking, consistency of interaction). Meanwhile, users' increased sensitivity to ad intrusion is forcing platforms to build more refined ad performance evaluation systems to optimize their placement strategies.

[0003] Current video management technologies often rely solely on click-through rates and dwell time to evaluate the effectiveness of advertisements within videos, failing to consider the significant impact of ad placement on performance evaluation and thus underestimating the potential value of advertisements. Summary of the Invention

[0004] In view of the technical problems existing in the above-mentioned background art, the present invention proposes a video management method based on artificial intelligence.

[0005] To achieve the above objectives, the technical solution adopted by the present invention includes the following steps:

[0006] First, acquire clean video data, use facial recognition to locate the center of the user's pupils, combine facial expressions to calculate the user's interest value, and determine the time frame of the user's attention concentration.

[0007] Images from video data at the time frames where user attention is focused are selected, and feature extraction is performed on the images to extract image text features and image scene features; content features of the video to be inserted are extracted, including advertising text features and advertising scene features;

[0008] Cross-modal matching is used to align the features of the image at the time frame where the user's attention is focused with those of the advertisement to be inserted. The cross-modal matching model outputs a matching score between the video and the advertisement. A first judgment threshold is set. If the matching score is greater than the first judgment threshold, the advertisement is included in the candidate advertisement pool.

[0009] Sort the ads in the candidate ad pool in descending order of matching score, determine the maximum number of ads to be inserted based on the total video duration, and select the top N ads as the ads to be inserted.

[0010] The ad to be inserted is placed immediately after the attention-focusing time frame of the video, generating a video containing the ad and pushing it to the user.

[0011] Get user interaction data when watching videos containing ads, calculate user stickiness score for ads based on interaction data. If stickiness score is greater than the preset second judgment threshold, the ad is determined to be a valid match with the video and the ad insertion strategy is retained. If it is less than or equal to the preset second judgment threshold, the next ad in the candidate ad pool is selected and the insertion and judgment process is re-executed.

[0012] Before extracting the content features of the video to be inserted, the method further includes extracting key feature frames from the video. The key frame extraction involves extracting one frame each from the beginning, middle, and end of the video as key feature frames.

[0013] Preferably, the specific implementation of using facial recognition to locate the center of the user's pupils and combining facial expressions to calculate the user's interest value includes:

[0014] First, the ROI of the eye is extracted, the edge points of the pupil are extracted, and the least squares method is used to fit an ellipse to the edge points of the pupil. The center of the ellipse is the center of the pupil.

[0015] Lightweight CNN is used to extract initial feature maps of facial ROIs. in For feature map size, Given the number of channels, calculate the distance weights between each location in the feature map and key facial expression nodes to generate a spatial weight map. , in, For feature map location This corresponds to the Euclidean distance from the nearest facial expression node in the original image. The length of the diagonal of the facial ROI;

[0016] Global average pooling is performed on the feature map channel dimension, and the importance of each channel is learned through a fully connected layer to generate a channel weight map. , ,in, This is the feature map of channel c. For global average pooling, It is a fully connected layer. For the Sigmoid function;

[0017] The weight map is fused with the base feature map, and then global pooling is used to obtain the expression feature vector. , ,in For element-wise multiplication, For outer product extension;

[0018] Calculate the standard deviation of pupil center movement within the sliding window, and obtain the stability after normalization. ,in The number of frames within the window. The average value of the pupil center within the window. For frame t, the horizontal and vertical coordinates of the pupil center in the video frame image coordinate system are used to calculate the pupil stability coefficient. , where D is the diagonal length of the eye region ROI;

[0019] The facial feature vector is processed using a pre-trained SVM. Rating, outputting positive facial expressions ;

[0020] Interest values ​​are obtained by weighted fusion of pupil stability coefficient and facial expression activity.

[0021] Preferably, the specific implementation of determining the user's attention focus time frame after obtaining the interest value is as follows:

[0022] Average interest value is calculated using a sliding window. ,in Let be the average interest value of the t-th frame. The size of the sliding window. This represents the original interest value for the (tw)th frame;

[0023] when When the value exceeds a set threshold, it is marked as a candidate frame, and the duration of consecutive candidate frames is counted. ,like If the duration exceeds the set threshold, all frames within the interval will be considered as time frames where the user's attention is focused.

[0024] Preferably, the selected video data includes images from time frames where the user's attention is focused. Feature extraction is performed on these images, including text features and scene features. The content features of the video to be inserted, including advertising text features and advertising scene features, are extracted. Specifically, the extraction of these features includes text feature extraction and scene feature extraction.

[0025] The text feature extraction is as follows:

[0026] A text detector is used to detect text regions in images of attention-focused time frames and keyframes in advertising videos, thereby locating text regions in the images.

[0027] The detected text regions are processed using a neural network for text recognition, and a text sequence is output.

[0028] The pre-trained BERT model encodes word vectors into the identified text sequence to obtain basic text feature vectors.

[0029] The GPT few-shot learning model is used to semantically expand the basic text feature vectors to generate relevant extended text vectors.

[0030] The basic text feature vector and the extended text feature vector are fused to obtain the final text features;

[0031] The scene features are extracted as follows:

[0032] The Transformer model is used to extract visual feature vectors of both image scenes;

[0033] The pre-trained CLIP model maps image features to a semantic space and matches them with text of a preset scene category to generate scene semantic feature vectors.

[0034] The scene features are obtained by concatenating and fusing the visual feature vector of the image scene with the semantic feature vector of the scene.

[0035] Preferably, the extraction of text features and scene features involves performing feature extraction operations on the user attention focus time frames extracted from all videos and the three key feature frames extracted from the beginning, middle and end of the advertising video, and then performing a weighted operation to obtain the final text features and scene features.

[0036] Preferably, the method of aligning the features of the image at the user's attention focus time frame with the features of the advertisement to be inserted through cross-modal matching, and outputting the matching score between the video and the advertisement through the cross-modal matching model, is implemented as follows:

[0037] The image text feature vector and image scene feature vector of the time frame in which user attention is focused are mapped to the same high-dimensional semantic space as the ad text feature vector and ad scene feature vector of the ad to be inserted through a cross-modal mapping network.

[0038] Calculate the cosine similarity between the image text mapping vector and the advertisement text mapping vector to obtain the text matching degree. ;

[0039] Calculate the cosine similarity between the image scene mapping vector and the advertisement scene mapping vector to obtain the scene matching degree.

[0040] The matching score is obtained by weighted summation of text matching score and scene matching score. ,in, These are the weighting coefficients.

[0041] Preferably, the interaction data for videos containing advertisements includes the ad click-through rate and the percentage of time the ad is displayed.

[0042] Preferably, the method for calculating the user's stickiness score to the advertisement based on interaction data is as follows:

[0043] First, calculate the synergy index. ,in For ad click-through rate, This refers to the percentage of time an ad is displayed.

[0044] Calculate the adhesion fraction ,in For sticky fractions.

[0045] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0046] 1. By combining facial recognition with pupil tracking and expression recognition technology, the system calculates user interest values ​​in real time, accurately locates the time frame of user attention concentration, and achieves intelligent determination of personalized ad insertion positions.

[0047] 2. Simultaneously extract textual and scene features from the image, and use various pre-trained models such as BERT, GPT, Transformer, and CLIP to perform deep semantic modeling and fusion of video content and advertising content, thereby improving the consistency between advertising and video context.

[0048] 3. By using a cross-modal mapping network to map video frames and advertising content to a unified high-dimensional semantic space, and using semantic matching degree to measure the fit between the two, the most relevant advertisements are selected for insertion, effectively improving user acceptance.

[0049] 4. Calculate stickiness score based on real user interaction data (such as click-through rate, dwell time, etc.), dynamically evaluate advertising effectiveness, form a feedback mechanism, automatically adjust advertising strategy, and achieve self-learning and continuous optimization of advertising effectiveness. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a schematic diagram of the structural process of an artificial intelligence-based video management method. Detailed Implementation

[0052] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0053] Numerous specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways than those described herein, and therefore the invention is not limited to the specific embodiments disclosed in the following specification.

[0054] In this example, with the rapid growth of video content in social media, streaming platforms, and online education, ad insertion has become one of the main ways for video platforms to monetize. However, existing ad insertion methods in video management often use fixed time points or simple keyword matching for push notifications, lacking a deep understanding of user viewing behavior. This easily leads to a disconnect between ads and video content, thus affecting user experience and ad placement effectiveness. Therefore, an AI-based video management method is proposed to facilitate ad and video matching, thereby improving user experience and ad placement effectiveness. The specific implementation process is as follows: Figure 1 As shown.

[0055] To achieve real-time recognition of user interests, clean video data is first acquired. Facial recognition is used to locate the center of the user's pupil, and the user's interest value is calculated based on facial expressions to determine the time frame of user attention concentration. Specifically, the implementation of locating the user's pupil center using facial recognition and calculating the user's interest value based on facial expressions includes: firstly, extracting the eye ROI, extracting pupil edge points, and then fitting an ellipse to the pupil edge points using the least squares method. The center of the ellipse is the pupil center. This algorithm obtains the optimal ellipse parameters, including center coordinates, major and minor axis lengths, and rotation angles, by minimizing the error from all edge points to the boundary of the fitted ellipse. Finally, the geometric center of the fitted ellipse is considered the center position of the user's pupil in the current frame, serving as the core reference data for subsequent calculations of pupil stability and user interest values. This method has the advantages of high computational accuracy and strong robustness, effectively coping with interference from real-world scenarios such as slight occlusion and changes in lighting. A lightweight CNN is used to extract the initial feature map of the facial ROI. in For feature map size, Given the number of channels, calculate the distance weights between each location in the feature map and key facial expression nodes to generate a spatial weight map. , in, For feature map location This corresponds to the Euclidean distance from the nearest facial expression node in the original image. The diagonal length of the facial ROI is given; global average pooling is performed on the feature map channel dimension, and the importance of channels is learned through a fully connected layer to generate a channel weight map. , ,in, This is the feature map of channel c. For global average pooling, It is a fully connected layer. The sigmoid function is used to fuse the weight map with the base feature map, and then global pooling is applied to obtain the expression feature vector. , ,in For element-wise multiplication, For outer product expansion; calculate the standard deviation of pupil center movement within the sliding window, and obtain stability after normalization. ,in The number of frames within the window. The average value of the pupil center within the window. For frame t, the horizontal and vertical coordinates of the pupil center in the video frame image coordinate system are used to calculate the pupil stability coefficient. Where D is the diagonal length of the eye ROI region; the expression feature vector is processed using a pre-trained SVM. Rating, outputting positive facial expressions Interest values ​​are obtained by weighted fusion of pupil stability coefficient and facial expression activity.

[0056] After obtaining the interest value, the user's attention focus time frame is determined, and the average interest value is calculated using a sliding window. ,in Let be the average interest value of the t-th frame. The size of the sliding window. The original interest value for the (tw)th frame; when When the value exceeds a set threshold, it is marked as a candidate frame, and the duration of consecutive candidate frames is counted. ,like If the duration exceeds the set threshold, all frames within the interval will be considered as time frames where the user's attention is focused.

[0057] To ensure semantic consistency and scene coherence between the advertisement and the video content, this invention further extracts multimodal features from video keyframes and advertisement video content. Images from time frames where user attention is focused are selected from the video data, and feature extraction is performed on these images, extracting image text features and image scene features. Content features of the advertisement video to be inserted are also extracted, including advertisement text features and advertisement scene features. Specifically, the advertisement video to be inserted is processed first. Before extracting the content features of the advertisement video to be inserted, key feature frames need to be extracted from the advertisement video. The key frame extraction involves extracting one frame each from the beginning, middle, and end of the advertisement video as key feature frames. This is done to ensure that only key frames are processed.

[0058] Then, text feature extraction and scene feature extraction were performed on each user attention focus time frame and the three key feature frames extracted from the beginning, middle and end of the video.

[0059] The text feature extraction process involves using a text detector to detect text regions in the image at the attention focus time frame and keyframes in the advertising video, locating text regions within the image. The detected text regions are then used to perform text recognition using a neural network, outputting a text sequence. A pre-trained BERT model is used to encode word vectors into the recognized text sequence, obtaining a basic text feature vector. A GPT few-shot learning model is then used to semantically extend the basic text feature vector, generating related extended text vectors. Finally, the basic and extended text feature vectors are fused to obtain the final text features. Specifically, to achieve semantic understanding of the image text in video keyframes and advertising content, the text feature extraction steps include multiple stages, aiming to transform the text information in the original image into deep vector features that can be used for semantic matching. This process specifically includes the following operations: First, an advanced text detector (such as an algorithm based on EAST or DBNet) is used to detect text regions in the image at the user's attention focus time frame and key feature frames in the advertising video. This step can accurately locate potential text regions in the image and output the position information of the text boxes for subsequent processing. Next, a deep learning-based neural network text recognition model (such as CRNN, Rosetta, etc.) is applied to the detected text regions to convert the text information in the image into a machine-readable text sequence. The recognition process can handle rotation, blurring, or complex backgrounds in the image, ensuring that the text information is restored as accurately as possible. The recognized text sequence is then input into a pre-trained BERT (Bidirectional Encoder Representations from Transformers) model for word vector encoding. The BERT model can capture semantic connections between texts through bidirectional contextual modeling, thereby outputting high-quality basic text feature vectors. To further enhance semantic representation capabilities, this invention introduces the GPT (Generative Pre-trained Transformer) few-shot learning model to semantically extend the basic text feature vectors. The GPT model generates extended text content related to the original semantics, mining potential meanings and contextual supplements to form a set of extended text feature vectors. This stage not only improves the understanding of short texts or abstract sentences but also enhances the ability to recognize hidden connections in the context of advertisements and videos. Finally, the basic text feature vector and the extended text feature vector are fused using methods such as weighted concatenation, attention mechanisms, or feature overlay to generate the final text feature representation. This final text feature has a richer semantic dimension, accurately expressing the textual connotation in the original image and providing stronger semantic support for subsequent cross-modal matching.

[0060] The scene feature extraction method employs a Transformer model to extract visual feature vectors from both images. Based on a pre-trained CLIP model, image features are mapped to a semantic space and matched with predefined scene category text to generate scene semantic feature vectors. These vectors are then concatenated and fused to obtain the scene features. Specifically, to achieve accurate understanding and semantic modeling of the scenes presented in the user attention-focused frame image and the advertisement keyframe image, the scene feature extraction method includes three core steps: visual feature extraction, semantic feature mapping, and multimodal feature fusion. The specific process is as follows: First, a Transformer-based image coding model is used to extract visual features from the image. This model fully utilizes the self-attention mechanism to model the contextual relationships of different regions in the image, thereby extracting the global scene features of the image. Compared to traditional convolutional neural networks (CNNs), the Transformer structure has a stronger ability to model scene elements in complex backgrounds and can identify high-level semantic categories such as "city street scene," "office space," and "natural scenery." Subsequently, the extracted image visual features are input into the pre-trained CLIP (Contrastive Language-Image Pretraining) model to further map the image to the semantic space. The CLIP model has been trained on massive image-text pairs and possesses powerful cross-modal understanding capabilities, converting image content into semantic vectors consistent with language descriptions. In this invention, several typical scene categories are pre-defined with text descriptions (such as "family kitchen," "business conference room," and "outdoor sports scene"). Image features are compared and matched with these text categories to obtain the scene semantic feature vector corresponding to the image. Finally, to balance the original visual information of the image with its semantic representation, a feature concatenation and fusion strategy is adopted to jointly encode the image visual feature vector and the scene semantic feature vector. The fusion operation can employ vector-level concatenation, weighted superposition, or attention-guided mechanisms, aiming to form a composite scene feature vector that simultaneously possesses low-level visual features and high-level semantic labels. This scene feature not only has strong semantic expressive power but also improves the scene consistency judgment between videos and advertisements in the subsequent cross-modal matching stage, thereby enhancing the relevance of advertising and user acceptance.

[0061] The extraction of text and scene features involves performing feature extraction operations on the user attention-focused time frames extracted from all videos, as well as three key feature frames extracted from the beginning, middle, and end of the advertisement video. These features are then weighted to obtain the final text and scene features. In other words, text and scene feature extraction is actually performed on two separate parts: first, selecting attention-focused time frames from the videos watched by the user for analysis; and second, extracting one key frame from the beginning, middle, and end of each advertisement video to be inserted, reflecting the overall content of the advertisement. After extracting text and scene features from these two image frames, a weighted calculation method is used to highlight the more representative frames, ultimately obtaining more accurate and comprehensive video and advertisement features for subsequent matching to determine the fit between the advertisement and the video.

[0062] Next, to improve the accuracy of ad recommendations, cross-modal matching is used to align the features of the image at the user's attention focus time frame with the features of the ad to be inserted. The cross-modal matching model outputs a matching score between the video and the ad. A first judgment threshold is set; if the matching score is greater than the first judgment threshold, the ad is included in the candidate ad pool. Specifically, the image text feature vector and image scene feature vector at the user's attention focus time frame are mapped to the same high-dimensional semantic space as the ad text feature vector and ad scene feature vector of the ad to be inserted through a cross-modal mapping network. The cosine similarity between the image text mapping vector and the ad text mapping vector is calculated to obtain the text matching score. Calculate the cosine similarity between the image scene mapping vector and the advertisement scene mapping vector to obtain the scene matching degree. The matching score is obtained by weighted summation of text matching score and scene matching score. ,in, These are the weighting coefficients.

[0063] The ads in the candidate ad pool are sorted in descending order of matching score. The maximum number of ads to be inserted is determined by combining the total video duration. The top N ads are selected as the ads to be inserted and are placed immediately after the attention focus time frame of the video. The video containing the ads is then generated and pushed to the user.

[0064] Finally, user interaction data while watching videos containing ads is obtained. Based on this data, a user engagement score is calculated. If the engagement score is greater than a preset second threshold, the ad and video are considered a valid match, and the ad insertion strategy is retained. If the score is less than or equal to the preset second threshold, the next ad in the candidate ad pool is selected, and the insertion and judgment process is repeated. The interaction data for videos containing ads includes ad click-through rate and ad dwell time percentage. The user engagement score is calculated based on the interaction data as follows: first, a synergy index is calculated. ,in For ad click-through rate, The percentage of time spent on an ad; calculating the stickiness score. ,in The stickiness score is used to determine the ad's effectiveness. If the score is above the threshold, it indicates a good match between the ad and the video, and the current ad insertion strategy is considered effective. The ad's insertion record in the current video will be retained. If the stickiness score is below or equal to the threshold, it indicates poor ad performance. The system will select the next ad from the candidate ad pool and re-execute the insertion and judgment process until the effectiveness condition is met or the candidate pool is exhausted.

[0065] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments for application in other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A video management method based on artificial intelligence, characterized in that, Includes the following steps: First, acquire clean video data, use facial recognition to locate the center of the user's pupils, combine facial expressions to calculate the user's interest value, and determine the time frame of the user's attention concentration. Images from video data at the time frames when the user's attention is focused are selected, and feature extraction is performed on the images to extract text features and scene features. Extract the content features of the video to be inserted, including ad text features and ad scene features; Cross-modal matching is used to align the features of the image at the time frame where the user's attention is focused with those of the advertisement to be inserted. The cross-modal matching model outputs a matching score between the video and the advertisement. A first judgment threshold is set. If the matching score is greater than the first judgment threshold, the advertisement is included in the candidate advertisement pool. Sort the ads in the candidate ad pool in descending order of matching score, determine the maximum number of ads to be inserted based on the total video duration, and select the top N ads as the ads to be inserted. The ad to be inserted is placed immediately after the attention-focusing time frame of the video, generating a video containing the ad and pushing it to the user. Get user interaction data when watching videos containing ads, calculate user stickiness score for ads based on interaction data. If stickiness score is greater than the preset second judgment threshold, the ad is determined to be a valid match with the video and the ad insertion strategy is retained. If it is less than or equal to the preset second judgment threshold, the next ad in the candidate ad pool is selected and the insertion and judgment process is re-executed. Before extracting the content features of the video to be inserted, the method further includes extracting key feature frames from the video. The key feature frames are extracted by extracting one frame each from the beginning, middle and end of the video. The specific implementation of using facial recognition to locate the center of the user's pupils and combining facial expressions to calculate the user's interest value includes: First, the ROI of the eye is extracted, the edge points of the pupil are extracted, and the least squares method is used to fit an ellipse to the edge points of the pupil. The center of the ellipse is the center of the pupil. Lightweight CNN is used to extract initial feature maps of facial ROIs. in For feature map size, Given the number of channels, calculate the distance weights between each location in the feature map and key facial expression nodes to generate a spatial weight map. , in, For feature map location This corresponds to the Euclidean distance from the nearest facial expression node in the original image. The length of the diagonal of the facial ROI; Global average pooling is performed on the feature map channel dimension, and the importance of each channel is learned through a fully connected layer to generate a channel weight map. , ,in, This is the feature map of channel c. For global average pooling, It is a fully connected layer. For the Sigmoid function; The weight map is fused with the base feature map, and then global pooling is used to obtain the expression feature vector. , ,in For element-wise multiplication, For outer product extension; Calculate the standard deviation of pupil center movement within the sliding window, and obtain the stability after normalization. ,in The number of frames within the window. The average value of the pupil center within the window. For frame t, the horizontal and vertical coordinates of the pupil center in the video frame image coordinate system are used to calculate the pupil stability coefficient. , where D is the diagonal length of the eye region ROI; The facial feature vector is processed using a pre-trained SVM. Rating, outputting positive facial expressions ; Interest values ​​are obtained by weighted fusion of pupil stability coefficient and facial expression activity. The interactive data for videos containing ads includes ad click-through rate and ad dwell time percentage; The method for calculating user engagement score based on interaction data is as follows: First, calculate the synergy index. ,in For ad click-through rate, This refers to the percentage of time an ad is displayed. Calculate the adhesion fraction ,in For sticky fractions.

2. The video management method based on artificial intelligence according to claim 1, characterized in that, The specific implementation of determining the user's attention focus time frame after obtaining the interest value is as follows: Average interest value is calculated using a sliding window. ,in Let be the average interest value of the t-th frame. The size of the sliding window. This represents the original interest value for the (tw)th frame; when When the value exceeds a set threshold, it is marked as a candidate frame, and the duration of consecutive candidate frames is counted. ,like If the duration exceeds the set threshold, all frames within the interval will be considered as time frames where the user's attention is focused.

3. The video management method based on artificial intelligence according to claim 1, characterized in that, The extraction of text features and scene features involves performing feature extraction operations on the user attention focus time frames extracted from all videos and the three key feature frames extracted from the beginning, middle and end of the advertising video, and then performing a weighted operation to obtain the final text features and scene features.

4. The video management method based on artificial intelligence according to claim 1, characterized in that, The process involves selecting images from the time frames where user attention is focused within the selected video data, extracting features from these images, including text features and scene features; and extracting the content features of the video to be inserted, including advertising text features and advertising scene features. Specifically, the extraction of these features includes text feature extraction and scene feature extraction. The text feature extraction is as follows: A text detector is used to detect text regions in images of attention-focused time frames and keyframes in advertising videos, thereby locating text regions in the images. The detected text regions are processed using a neural network for text recognition, and a text sequence is output. The pre-trained BERT model encodes word vectors into the identified text sequence to obtain basic text feature vectors. The GPT few-shot learning model is used to semantically expand the basic text feature vectors to generate relevant extended text vectors. The basic text feature vector and the extended text feature vector are fused to obtain the final text features; The scene features are extracted as follows: The Transformer model is used to extract the visual feature vectors of the scene from both images; The pre-trained CLIP model maps image features to a semantic space and matches them with text of a preset scene category to generate scene semantic feature vectors. The scene features are obtained by concatenating and fusing the visual feature vector of the image scene with the semantic feature vector of the scene.

5. The video management method based on artificial intelligence according to claim 1, characterized in that, The process of aligning the features of the image at the user's attention focus time frame with the features of the advertisement to be inserted through cross-modal matching, and outputting the matching score between the video and the advertisement through the cross-modal matching model, is implemented as follows: The image text feature vector and image scene feature vector of the time frame in which user attention is focused are mapped to the same high-dimensional semantic space as the ad text feature vector and ad scene feature vector of the ad to be inserted through a cross-modal mapping network. Calculate the cosine similarity between the image text mapping vector and the advertisement text mapping vector to obtain the text matching degree. ; Calculate the cosine similarity between the image scene mapping vector and the advertisement scene mapping vector to obtain the scene matching degree. ; The matching score is obtained by weighted summation of text matching score and scene matching score. ,in, These are the weighting coefficients.

Images