A video cover generation method and device

Abstract

The application discloses a video cover generation method and device, the method comprises the following steps: filtering the frame images in the target video through preset filtering conditions to obtain candidate frame images, and calculating the video frame features of each candidate frame image; calculating the text features of the text description corresponding to the video frame features; introducing a cross-modal interaction mode to measure the semantic correlation degree of each candidate frame image and the text features; calculating the selection probability of each candidate frame image to become a cover frame according to the correlation degree, so as to determine the video frame with high quality and the highest correlation with the text description as a candidate cover based on the probability value; and in response to the cover style specified by the user, the candidate cover is personalized processed to generate a target cover. The device comprises a processor and a memory. The application solves the problems that the current automatic generation effect of the video cover is poor, and the correlation degree between the generated cover and the video content is poor. The application can automatically generate a personalized cover for the video, improve the user experience, and enhance the attraction of the video.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of video processing, in particular to a video cover generation method and device. BACKGROUND

[0002] In recent years, with the popularity of mobile Internet and smart terminals represented by mobile phones and video shooting devices, the threshold for people to shoot and share videos is getting lower and lower. This greatly promotes the rapid development of short video platforms. According to statistics, many video social platforms have more than 1 billion active users and more than 20 billion video plays per day. However, for a large amount of video data, many non-professional video creators usually do not have enough time or professional knowledge to choose an attractive video cover frame. And most video websites will use the first few frames of the video as the default cover when the user does not upload a cover. However, these default covers are usually of low quality, with blurred or dark pictures, and cannot accurately reflect the main content of the video, thereby affecting the click rate and popularity of the video. At the same time, the original video image frame often has the shortcomings of insufficient content appeal and poor image style popularity. Therefore, how to realize intelligent video cover generation is an important problem to be solved.

[0003] Traditional cover generation methods are mainly based on image frame analysis of video modal. They often start from the color, clarity, brightness, and content richness of the video frame to comprehensively evaluate the image aesthetic information of the frame. However, this method relies too much on manually designed evaluation indicators and cannot optimally adapt to complex and diverse video data. With the rapid development of deep learning, deep learning-based video frame generation schemes can also generate video covers. However, this technical solution does not comprehensively consider the role of video text modal data, and the generated pictures often do not conform to the theme expressed by the video content.

[0004] In the real short video platform scene, people often pay attention to the video cover and the corresponding text description at the same time, as well as the individualized artistic style of the cover. If the cover selection and description are quite different and do not have a popular style, it will greatly affect the browsing interest and click tendency. Therefore, a video cover that is consistent with the video description and has high quality and individualized style can improve user experience and enhance video appeal. SUMMARY

[0005] The present application provides a video cover generation method and device, which solves the problem of poor automatic generation effect of the current video cover, poor correlation between the generated cover and the video content, and lack of novelty. The present application can automatically generate individualized covers for videos, improve user experience, and enhance video appeal. Details are described below:

[0006] A video cover generation method, the method comprising:

[0007] filtering frame images in the target video through preset filtering conditions to obtain candidate frame images, calculating video frame features of each candidate frame image; calculating text features of a text description corresponding to the video frame features;

[0008] introducing a cross-modal interaction mode to measure a semantic correlation degree of each candidate frame image and the text features; calculating a selection probability of each candidate frame image to become a cover frame according to the correlation degree, to determine a high-quality video frame most relevant to the text description as a candidate cover based on a probability value;

[0009] in response to a user-specified cover style, performing personalized processing on the candidate cover to generate a target cover.

[0010] The filtering of the frame images in the target video through preset filtering conditions specifically includes:

[0011] filtering the frame images in the target video in terms of image brightness, definition, and color unification to obtain a first filtering result;

[0012] determining a transition frame image based on an edge quality change rate of adjacent two frame images in the first filtering result that meet a time sequence, and filtering out the transition frame image to obtain the candidate frame image.

[0013] The calculating of the selection probability of each candidate frame image to become a cover frame according to the correlation degree, to determine a high-quality video frame most relevant to the text description as a candidate cover based on a probability value specifically includes:

[0014] mapping the text features and the video frame features into the same semantic space; calculating a distance between the text features and the video frame features in the semantic space, and determining a similarity between the text features and the video frame features based on the distance;

[0015] calculating a probability of each candidate frame image being selected as a candidate cover based on the similarity; and determining a candidate frame image with the maximum probability value as the candidate cover.

[0016] The method further includes: constructing a target model for selecting the candidate cover, to select the candidate cover for the video based on the target model, wherein a loss function of the target model is:

[0017]

[0018] wherein ||·|| represents a square sum of pixel values, F is a candidate cover selected by the model, I truth is an actual candidate cover.

[0019] ​The personalized processing of the candidate cover according to the user-specified cover style to generate the target cover specifically comprises:

[0020] determining a specific word of the user-specified cover style; and mapping the specific word into a first feature vector based on a word vector model;

[0021] mapping text embedding features corresponding to the candidate frame image into a second feature vector based on the word vector model; and generating text-guided features based on the first feature vector and the second feature vector processed by a text encoder;

[0022] generating stylized text embedding information matching the user-specified cover style based on the text-guided features and the text embedding features corresponding to the candidate cover; and processing the candidate cover based on the stylized text embedding information to obtain the target cover with the user-specified cover style.

[0023] In a second aspect, a video cover generation device is provided, and the device comprises a processor and a memory, wherein the memory stores program instructions, and the processor invokes the program instructions stored in the memory to enable the device to perform the method in any one of the first aspect.

[0024] In a third aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, wherein the computer program comprises program instructions, and the program instructions are executed by a processor to enable the processor to perform the method in any one of the first aspect.

[0025] The technical scheme provided by the present application has the following beneficial effects:

[0026] 1. The method can simultaneously refer to two modal information of video and text to realize the automatic generation of a cross-modal video cover.

[0027] 2. The method performs double-branch feature extraction on the video and the corresponding text description in the video platform, including video branch preprocessing and frame feature extraction, and text branch feature extraction; then the semantic correlation between the candidate frame and the text is calculated through a cross-attention mechanism, and the correlation degree is calculated as a probability value of becoming a cover; after that, the candidate frame image with the maximum probability is output as the candidate cover of the video; the process of calculating and determining the candidate cover by using two modal information of video and text can guarantee the selection efficiency and the cover quality, and avoid the phenomenon that the cover is irrelevant to the video content or the picture quality is poor.

[0028] 3. In order to meet the individualized needs of users, the present application can also realize the individualized style migration, so that the user-specified style features can be migrated to the candidate cover, and then the target cover with the user-specified style is generated; in this way, the individualized needs of users are met, the interestingness and attractiveness of the cover are increased, and the present application has strong application value. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 Flowchart of a method for generating a video cover;

[0030] Figure 2 Application flowchart of a method for generating a video cover;

[0031] Figure 3 Another application flowchart of a method for generating a video cover;

[0032] Figure 4 Block diagram of a device for generating a video cover. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application are described in further detail below.

[0034] Embodiment 1

[0035] The embodiments of the present application provide a method for generating a video cover, which comprises the following steps:

[0036] S1: filtering the frame images in the target video through preset filtering conditions to obtain candidate frame images, the filtering conditions being related to the image quality and display effect of the frame images;

[0037] For example, filtering the black frame and white frame at the beginning of the video, the blurred picture that flashes quickly, or the picture with insufficient brightness; narrowing the selection range of the candidate cover frame to reduce the subsequent calculation burden.

[0038] S2: calculating the video frame features of each candidate frame image;

[0039] S3: calculating the text features of the text description corresponding to the target video;

[0040] S4: introducing a cross-modal interaction mode to measure the semantic correlation degree between each candidate frame image and the description text; calculating the selection probability of each candidate frame image to become a cover frame according to the correlation degree, so as to determine the video frame with high quality and the most relevant text description as the candidate cover based on the probability value;

[0041] S5: in response to the user-specified cover style, performing personalized processing on the candidate cover to generate a target cover.

[0042] After determining the candidate cover, the user can directly use it as the final video cover, or can perform personalized and stylized processing on it to generate a cover with a specified style. For example, the user inputs a selected reference image with a popular style, and then generates a diffusion image through personalized image diffusion processing, so as to successfully migrate the video frame to generate a personalized video cover.

[0043] Example 2

[0044] The scheme in Example 1 will be further described below with specific examples and calculation formulas:

[0045] Specifically, step S1 above filters the frame images in the target video based on preset filtering conditions to obtain candidate frame images, including:

[0046] 1) Filter the frame images in the target video based on image brightness, sharpness, and color uniformity to obtain the first filtering result;

[0047] 2) Based on the edge quality change rate of two adjacent frames that satisfy the time series in the first filtering result, determine the transition frame image, filter out the transition frame image, and obtain the candidate frame image.

[0048] Specifically, for current videos, directly processing all frame images would introduce a large amount of data and be very time-consuming. While sampling video frames at uniform time intervals for subsequent cover selection might miss important video frames, hindering cover selection. However, considering the existence of low-quality frames and blurry transition frames in the video, these frames lack aesthetic appeal and cannot be considered as cover frame candidates. Therefore, this embodiment of the invention preprocesses all frame images to eliminate unqualified frame images, while simultaneously narrowing the frame image processing range, thereby reducing the processing load and improving processing efficiency.

[0049] Specifically, in step 1) above, the frame images in the target video are filtered for image brightness, sharpness, and color uniformity to obtain a first filtering result, including:

[0050] The relative brightness of each frame of the image is determined based on the following formula:

[0051] Luminance (I) rgb )=02126I r +0.7152I g +0.0722I b

[0052] Among them, I r I g and I b These represent the pixel mean values ​​of the red, green, and blue (RGB) channels of the frame image, respectively. Luminance (I rgb () indicates the relative brightness of the frame image.

[0053] The sharpness of each frame of the image is determined based on the following formula:

[0054]

[0055] wherein I gray represents the gray pixel value of the video frame image, Δ x I gray and Δ y I gray respectively represent the gradients of the gray image in the horizontal direction and the vertical direction, Sharpness(I gray ) represents the sharpness of the frame image.

[0056] The color uniformity of each frame image is calculated based on the following formula:

[0057]

[0058] wherein hist(I gray ) is the normalized brightness histogram of the frame image, and then the values are sorted in descending order, the cumulative distribution at the top 5% of the histogram values is calculated, to obtain Uniformity(I gray ), which represents the color uniformity of the frame image, the smaller the value, the more uniform the color distribution in the image, and vice versa.

[0059] The above three indicators (the relative brightness of each frame image, the sharpness of each frame image, and the color uniformity of each frame image) are sorted from large to small, and the first filtering result is obtained by screening through the preset proportion threshold, which retains the image frames that simultaneously meet the top 40% of the corresponding sorting results of the three indicators.

[0060] Since the transition frame usually exists in the video scene conversion process, accordingly, in the embodiment of the application, the shot boundary detection is adopted to realize the recognition of the transition frame, specifically, the edge change rate between adjacent frames arranged in time sequence in the first filtering result is detected to recognize the boundary of the video scene, the frame image located near the boundary is determined, and the frame image is discarded. That is, the frame image is the transition frame, which belongs to the low-quality frame image, and is not considered as a cover, so it can be removed.

[0061] wherein, after obtaining the candidate frame image, the video frame feature of each candidate frame image is calculated in the above step S2, including:

[0062] Based on the image encoder, each candidate frame image is processed in combination with the self-attention mechanism to obtain the video frame feature corresponding to each candidate frame image.

[0063] In this embodiment of the invention, video frame feature extraction is implemented using a network model. For example, the CLIP model can be used. CLIP is a large-scale visual language pre-trained model whose image encoder uses a VisionTransformer neural network model and processes image information through a self-attention mechanism. Unlike traditional convolutional neural networks, it uses a set of Transformer modules to extract image features, giving the model better generalization ability and scalability. In this embodiment of the invention, for the candidate frame image set V = {I1, I2, ..., I...} n}, where n represents the number of video frames, and each candidate frame image I i For i∈[1,n] input into the CLIP image encoder, its feature representation can be obtained as:

[0064]

[0065] in, CLIP represents the image encoder. d represents the length of the feature dimension.

[0066] Next, in step S3 above, the text features of the text description corresponding to the target video are calculated, which includes: processing the text description corresponding to the target video based on the text encoder to obtain the text features.

[0067] Following the network model from the previous embodiment, this embodiment of the invention still uses the CLIP model to extract text features. The CLIP model's text encoder uses the BERT language model. The BERT model uses the Transformer module to process text information, which can automatically encode and represent text, thereby achieving better semantic understanding and expression. Specifically, for the text description T corresponding to the target video V, it is input into the CLIP text encoder to obtain the feature representation as follows:

[0068] u=φ(T)

[0069] Where φ(·) represents CLIP's text encoder.

[0070] Continue to combine Figure 2 As shown, after determining the video frame features and text features of the candidate frame images, it is necessary to determine their relationship, that is, to evaluate their correlation. Step S4 in this embodiment of the invention determines the relationship between the text features and each video frame feature, and determines the candidate cover image in the candidate frame image based on the relationship, including:

[0071] 1) Map text features and video frame features to the same semantic space;

[0072] 2) calculating the distance of the text features and the video frame features in the semantic space, and determining the similarity between the text features and the video frame features based on the distance;

[0073] 3) determining the probability of each candidate frame image being selected as a candidate cover based on the similarity calculation;

[0074] 4) determining the candidate frame image with the maximum probability value as the candidate cover.

[0075] When the above method is executed, for the obtained video frame features v i and the text features u, the cross-attention interaction is used to measure the semantic correlation between each candidate frame image and the text description in the embodiment of the application. Specifically, the video frame mapping network and the text mapping network are used respectively to map the two modal features of the video frame features and the text features to a common semantic space, i.e.:

[0076]

[0077] wherein f visual (·) and f textual (·) are full-connection neural networks.

[0078] Then, in this common semantic space, the video frame features and the text features have semantic comparability. Therefore, the cosine distance can be used to measure the semantic similarity between the video frame features and the text features as:

[0079]

[0080] Next, a target model for assisting in selecting the candidate cover can be constructed, which is mainly used for probability calculation and candidate cover selection based on the probability value:

[0081] 5) constructing a target model for selecting the candidate cover, so as to select the candidate cover for the video based on the target model, wherein the loss function of the target model is:

[0082]

[0083] wherein ||·||2 F represents the sum of squares of pixel values, is the candidate cover selected by the model, I truth is the actual candidate cover.

[0084] Specifically, based on all {s i}, i∈[1,n], the probability of each video frame being selected as a cover can be obtained as:

[0085]

[0086] Where, p i ∈[0,1] represents the relevance of the i-th video frame feature relative to other frame features and text features. λ is a scaling factor, which assigns a higher probability value to video frames with high semantic relevance, while compressing other video frames with low semantic relevance. Accordingly, in the modeling process, the video frame with the highest corresponding probability value is selected as the candidate cover:

[0087]

[0088] Where, j * This is the index subscript for the cover frame.

[0089] During the training of the target model, for each video, the prediction of video frames can be minimized. With the labeled video cover I truth The distance between the training data (i.e., the training data includes: training videos, relevant information, and labeled video covers) is used to enable the model to predict the correct cover frame as accurately as possible. Therefore, the model's training loss function is set as follows:

[0090]

[0091] Among them, ||·|| F This represents the sum of squared pixel values. In practical applications, Adam can be used as the optimizer in the training and optimization process of the target model, with the learning rate set to 1e-4 and the momentum parameter set to 0.9.

[0092] In real life, due to the diverse content themes and varying shooting quality, videos do not always possess the latest and most popular styles, which affects the appeal of candidate covers to users.

[0093] To address this issue, this invention proposes a personalized style transfer intelligent generation method to transform selected video candidate frame images into cover images with personalized popular styles.

[0094] Specifically, step S5 in Embodiment 1 above, in response to the user-specified cover style, performs personalized processing on the candidate cover to generate the target cover, including:

[0095] 1) Determine specific words for the cover style specified by the user;

[0096] 2) Map specific words to first feature vectors based on word vector models;

[0097] 3) Map the text embedding features corresponding to the candidate frame images into a second feature vector based on the word vector model;

[0098] 4) generating text-guided features based on the text encoder processing the first feature vector and the second feature vector;

[0099] 5) generating stylized text embedding information matching the user-specified cover style based on the text-guided features and the text embedding features corresponding to the candidate cover;

[0100] 6) processing the candidate cover based on the stylized text embedding information to obtain a target cover with the user-specified cover style.

[0101] Among them, the step 6) of processing the candidate cover based on the stylized text embedding information to obtain a target cover with the user-specified cover style comprises:

[0102] (1) training an initial image generation model based on specific words and images with style represented by the specific words;

[0103] (2) determining a historical candidate cover with the same text embedding features as the candidate cover;

[0104] (3) training the initial image generation model based on the text-guided features, the first feature vector, the text embedding features of the historical candidate cover, and the historical candidate cover to obtain an image generation model capable of matching and processing the candidate cover based on the text embedding features to generate the target cover;

[0105] (4) processing the candidate cover based on the stylized text embedding information by the image generation model to generate the target cover.

[0106] The specific path of generating the stylized target cover based on the stylized text embedding information by the image generation model is shown in Figure 3 The personalized style transfer is divided into two steps, including: (1) learning the text semantic embedding features of the stylized image and the video candidate frame image respectively in the image generation process; (2) generating a stylized video cover under the text semantic prompt of the stylized video frame image.

[0107] Specifically, the user first selects an image style that meets personal preferences, and selects some images with the style as learning data, and trains the initial model accordingly, for example: based on a pre-trained generation model (the model in the embodiment of the present application is a latent diffusion model (Latent Diffusion Model, LDM)), learn the text semantic embedding corresponding to the style. For example: use a specific word S* to represent the image style selected by the user, input the image into the generation model for training, and only the word vector model in the model is learned by mapping the specific word to the word vector through the word table to optimize the parameters, while the remaining text encoder and image generator keep the parameters fixed.

[0108] During training, the specific word S* is mapped to a feature vector s via the word vector model * , which is used to train the model i . For example, the feature vector of the text embedding feature corresponding to the candidate frame image is input into the text encoder to form a text-guided feature, which is then input into the image generator to guide the diffusion model to generate an image. Therefore, the loss function of the training is:

[0109] s * = argmin E z～ε(x)，τ，∈～N(0，1)，t [||∈-∈ θ (z t , t, τ(S * , S i ))|| 2 ]

[0110] where ∈ is the sampling noise, ∈ θ is the noise predictor, which contains three inputs: the denoising result z t of the previous step, the denoising step number t, and the text-guided feature τ(s * , s i ).

[0111] Through this process, the stylized image feature can be fine-tuned into the text embedding feature vector, i.e., the stylized migration generation of other images can be achieved by learning s * , i.e., the frame image without the style is processed to form an image with the style, such as ink painting, oil painting, cartoon, etc. It can also be a custom style, similar to a custom style “template” / “feature”, and the model learns the custom style to process the input frame image to form an image with the custom style. To maintain the content of the original video candidate frame image, similar to learning the style reflecting text embedding s * , the text embedding feature v * of the candidate video frame is obtained, and then according to the text embeddings v * and s * representing the candidate cover and the target style respectively, a video frame text semantic prompt with stylization can be constructed: ‘image v * with style s * ’, which is input into the trained image generation model. The model can successfully understand the content in the text semantic prompt based on the learning results, and then determine the processing method corresponding to the current candidate cover, and then process the candidate cover to generate a target video cover with the user-specified style.

[0112] Embodiment 3

[0113] An apparatus for generating a video cover, the apparatus comprising: a processor and a memory, the memory having stored therein program instructions, the processor invoking the program instructions stored in the memory to cause the apparatus to perform the following method steps in embodiment 1:

[0114] filtering frame images in the target video through preset filtering conditions to obtain candidate frame images, and calculating video frame features of each candidate frame image;

[0115] introducing a cross-modal interaction mode to measure the semantic correlation degree of each candidate frame image and the text feature, and calculating a selection probability of each candidate frame image to become a cover frame according to the correlation degree to determine a high-quality video frame most relevant to the text description as a candidate cover based on the probability value;

[0116] In response to a user-specified cover style, the candidate cover is personalized to generate a target cover.

[0117] In particular, filtering frame images in the target video through preset filtering conditions is:

[0118] filtering frame images in the target video with respect to image brightness, definition, and color unification to obtain a first filtering result;

[0119] determining transition frame images based on the edge quality change rate of adjacent two frame images in the first filtering result that meet the time sequence, and filtering out the transition frame images to obtain candidate frame images.

[0120] In particular, calculating a selection probability of each candidate frame image to become a cover frame according to the correlation degree, and determining a high-quality video frame most relevant to the text description as a candidate cover based on the probability value is:

[0121] mapping the text feature and the video frame feature into the same semantic space, calculating the distance between the text feature and the video frame feature in the semantic space, and determining the similarity between the text feature and the video frame feature based on the distance;

[0122] calculating the probability of each candidate frame image being selected as a candidate cover based on the similarity, and determining the candidate frame image with the maximum probability value as the candidate cover.

[0123] In particular, the apparatus further comprises: constructing a target model for selecting a candidate cover, and selecting a candidate cover for the video based on the target model, wherein the loss function of the target model is:

[0124]

[0125] wherein ||·|| represents the sum of squares of pixel values, F ​For the model selected candidate cover, I truth For the actual candidate cover.

[0126] Among them, in response to the user specified cover style, the candidate cover is personalized to generate the target cover, specifically:

[0127] Determine the specific word of the user specified cover style; Map the specific word to the first feature vector based on the word vector model;

[0128] Map the text embedding feature corresponding to the candidate frame image to the second feature vector based on the word vector model; Process the first feature vector and the second feature vector to generate the text guided feature based on the text encoder;

[0129] Generate stylized text embedding information matching the user specified cover style based on the text guided feature and the text embedding feature corresponding to the candidate cover; Process the candidate cover based on the stylized text embedding information to obtain the target cover with the user specified cover style.

[0130] It should be pointed out here that the device description in the above embodiments corresponds to the method description in the embodiments, and the embodiments of the present application will not be repeated here.

[0131] The execution subject of the processor and the memory described above can be a computer, a single-chip microcomputer, a microcontroller, etc. with computing function. In specific implementation, the execution subject is not limited by the embodiments of the present application, and is selected according to the needs in actual application.

[0132] The memory 2 and the processor 1 transmit data signals through the bus 3, and the embodiments of the present application will not be repeated here.

[0133] Based on the same inventive concept, the embodiments of the present application also provide a computer readable storage medium, which includes a stored program. When the program runs, it controls the device where the storage medium is located to execute the method steps in the above embodiments.

[0134] The computer readable storage medium includes but is not limited to flash memory, hard disk, solid state disk, etc.

[0135] It should be pointed out here that the readable storage medium description in the above embodiments corresponds to the method description in the embodiments, and the embodiments of the present application will not be repeated here.

[0136] In the above embodiments, all or part of them can be realized by software, hardware, firmware or any combination thereof. When realized by software, it can be realized in the form of computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the flow or function according to the embodiments of the present application is generated in whole or in part.

[0137] The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in or transmitted from a computer readable storage medium. The computer readable storage medium can be any available medium or data storage device that can be accessed by a computer and includes a tangible, non-transitory computer readable medium integrated into a server, data center, or other data storage device. The available medium can be a magnetic, semiconductor, or other medium.

[0138] It should be understood by those of ordinary skill in the art that the above discussion of any embodiment is merely exemplary and is not intended to suggest the scope of the protection of the present application is limited to these examples; the above embodiments or technical features among different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes to the aspects of one or more embodiments of the present application as described above, which are not provided in detail for the sake of brevity.

[0139] The model of each device in the embodiments of the present application is not limited, except for the special description, and any device that can complete the above functions can be used.

[0140] Those skilled in the art can understand that the drawings are only a schematic diagram of a preferred embodiment, and the above-mentioned serial numbers of the embodiments of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0141] The above is only a preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for generating video cover images, characterized in that, The method includes: The frame images in the target video are filtered using preset filtering conditions to obtain candidate frame images. The video frame features of each candidate frame image are calculated. The text features of the text description corresponding to the video frame features are calculated. A cross-modal interaction method is introduced to measure the semantic relevance between each candidate frame image and text features; the probability of each candidate frame image becoming the cover frame is calculated based on the relevance, and the high-quality video frame that is most relevant to the text description is selected as the candidate cover based on the probability value. In response to the user-specified cover style, the candidate covers are personalized to generate the target cover; Specifically, filtering the frame images in the target video using preset filtering conditions involves: The frame images in the target video are filtered for image brightness, sharpness, and color uniformity to obtain the first filtering result; Based on the edge quality change rate of two adjacent frames that satisfy the time series in the first filtering result, the transition frame image is determined, and the transition frame image is filtered out to obtain the candidate frame image. The step of calculating the probability of each candidate frame image becoming the cover frame based on relevance, and determining the high-quality video frame most relevant to the text description as the candidate cover frame based on the probability value, specifically involves: Text features and video frame features are mapped to the same semantic space; the distance between text features and video frame features in the semantic space is calculated, and the similarity between text features and video frame features is determined based on the distance; The probability of each candidate frame image being selected as a candidate cover is determined based on similarity calculation; the candidate frame image with the highest probability value is selected as the candidate cover. The method further includes: constructing a target model for selecting candidate covers, and selecting candidate covers for the video based on the target model, wherein the loss function of the target model is: ； in, Represents the sum of squares of pixel values. Candidate covers selected for the model. This is the actual candidate cover; The process of personalizing candidate covers in response to user-specified cover styles and generating a target cover specifically involves: Identify specific words for the user-specified cover style; map these specific words into a first feature vector based on a word vector model; The text embedding features corresponding to the candidate frame image are mapped to a second feature vector based on a word vector model; the first and second feature vectors are then processed by a text encoder to generate text guidance features. Based on text guidance features and text embedding features corresponding to candidate covers, stylized text embedding information matching the cover style specified by the user is generated; candidate covers are processed based on stylized text embedding information to obtain a target cover with the cover style specified by the user. Calculate and determine the relative brightness of each frame of the image: ； in, and These represent the average pixel values ​​of the red, green, and blue channels of the frame image, respectively. Indicates the relative brightness of the frame image; Calculate and determine the sharpness of each frame of the image: ； in, Represents the grayscale pixel value of a video frame image. and These represent the grayscale images in... Gradients in the horizontal and vertical directions, Indicates the sharpness of the frame image; The color uniformity of each frame of the image is calculated based on the following formula: ； in, The normalized luminance histogram of the frame image is generated, and then the values ​​are sorted in descending order. The cumulative distribution of the top 5% of histogram values ​​is calculated to obtain an indicator of the color uniformity of the frame image. ; The two modal features, video frame features and text features, are mapped to a common semantic space, namely: ； ； in, and These are fully connected neural networks.

2. A video cover generation device, characterized in that, The device includes a processor and a memory, the memory storing program instructions, and the processor calling the program instructions stored in the memory to cause the device to perform the method of claim 1.

3. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method of claim 1.

Images