Video cover image generation method and device, equipment and storage medium

Abstract

The invention discloses a video cover image generation method and device, equipment and a storage medium, and the method comprises the steps: segmenting a target video into a plurality of video clips, and enabling the image similarity between adjacent video images in each video clip to be greater than a first similarity; inputting each video clip into an action recognition model for action recognition, and outputting an action recognition category and an action recognition score of each video clip; based on a plurality of action recognition categories and a plurality of action recognition scores corresponding to the plurality of video clips, screening the plurality of video clips to obtain a first clip of which the action recognition category is matched with a preset action category and the action recognition score is greater than the preset action score; generating a video cover image of the target video based on the first clip; and displaying the dynamic cover image can fully attract the attention of the user and the attention of the user.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, equipment and storage medium for generating a cover image of a video. Background Art

[0002] At present, video websites usually display the cover image of the video to attract users' attention and focus, so that users will be interested in and look forward to the video.

[0003] In the related art, a static cover image is generally generated for a video, and the static cover image is displayed for the video.

[0004] However, static cover images have limitations and cannot express the rich and important content of the video. They cannot fully attract users' attention and focus, and reduce users' interest and expectations for the video. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a method, device, equipment and storage medium for generating a cover image of a video. The generated cover image can express and convey the rich and important content of the target video. Displaying the cover image for the video can fully attract user attention and user focus, thereby increasing user interest and expectations for the target video.

[0006] On the one hand, an embodiment of the present application provides a video cover image method, the method comprising:

[0007] The target video is segmented to obtain a plurality of video segments; the image similarity between adjacent frame video images in each video segment is greater than the first similarity;

[0008] Performing action recognition on each video clip through an action recognition model to obtain an action recognition category and an action recognition score for each video clip;

[0009] According to the action recognition category and the action recognition score of each video segment, the multiple video segments are screened to obtain a first segment; the action recognition category of the first segment matches a preset action category, and the action recognition score of the first segment is greater than a preset action score;

[0010] A cover is generated for the first segment to obtain a dynamic cover image of the target video.

[0011] Optionally, the adjacent frame video images in the target video include a first video image and a second video image; and the segmenting the target video to obtain a plurality of video segments includes: performing similarity calculation on the first video image and the second video image according to a pixel average and a pixel variance of the first video image, a pixel average and a pixel variance of the second video image, and a pixel covariance of the first video image and the second video image, to obtain image similarity between the first video image and the second video image;

[0012] According to the image similarity between the first video image and the second video image, the target video is segmented into segments to obtain the multiple video segments.

[0013] Optionally, segmenting the target video into segments to obtain the multiple video segments according to the image similarity between the first video image and the second video image includes:

[0014] If the image similarity between the first video image and the second video image is less than or equal to the first similarity, determining a segment segmentation point of the target video according to a video time point between the first video image and the second video image;

[0015] The target video is segmented according to the segmentation points to obtain the multiple video segments.

[0016] Optionally, screening the multiple video clips to obtain the first clip according to the action recognition category and the action recognition score of each video clip includes:

[0017] According to the action recognition category of each video clip, screening the video clips whose action recognition categories match the preset action categories among the multiple video clips as candidate clips;

[0018] According to the action recognition score of each candidate segment, the candidate segment whose action recognition score is greater than the preset action score is selected as the first segment.

[0019] Optionally, the step of obtaining the action recognition model includes:

[0020] Obtaining a sample video, an action category label and an action score label of the sample video; the action score label represents the action clarity and action expressiveness of the sample video;

[0021] Performing action recognition on the sample video by using an initial recognition model to obtain an action prediction category and an action prediction score of the sample video;

[0022] According to the difference between the action prediction category and the action category label, and the difference between the action prediction score and the action score label, the initial recognition model is trained to obtain the action recognition model.

[0023] Optionally, generating a cover for the first segment to obtain a dynamic cover image for the target video includes:

[0024] According to the image similarity between adjacent frames of video images in the first segment, the first segment is segmented to obtain a second segment; the image similarity between adjacent frames of video images in the second segment is greater than a second similarity, and the second similarity is greater than the first similarity;

[0025] A cover is generated for the second segment to obtain the dynamic cover image.

[0026] Optionally, the video duration of the second segment is greater than a preset duration, and the preset duration is less than the video duration of the first segment.

[0027] On the other hand, an embodiment of the present application provides a video cover image generation device, the device comprising: a segmentation unit, a recognition unit, a screening unit and a generation unit;

[0028] The segmentation unit is used to segment the target video into multiple video segments; the image similarity between adjacent frame video images in each video segment is greater than the first similarity;

[0029] The recognition unit is used to perform action recognition on each video clip through an action recognition model to obtain an action recognition category and an action recognition score for each video clip;

[0030] The screening unit is used to screen the multiple video clips according to the action recognition category and the action recognition score of each video clip to obtain a first clip; the action recognition category of the first clip matches a preset action category, and the action recognition score of the first clip is greater than a preset action score;

[0031] The generating unit is used to generate a cover for the first segment to obtain a dynamic cover image of the target video.

[0032] Optionally, the adjacent frame video images in the target video include a first video image and a second video image; and the segmentation unit is used to:

[0033] Performing similarity calculation on the first video image and the second video image according to the pixel average and pixel variance of the first video image, the pixel average and pixel variance of the second video image, and the pixel covariance between the first video image and the second video image to obtain image similarity between the first video image and the second video image;

[0034] According to the image similarity between the first video image and the second video image, the target video is segmented into segments to obtain the multiple video segments.

[0035] Optionally, the segmentation unit is used to:

[0036] If the image similarity between the first video image and the second video image is less than or equal to the first similarity, determining a segment segmentation point of the target video according to a video time point between the first video image and the second video image;

[0037] The target video is segmented according to the segmentation points to obtain the multiple video segments.

[0038] Optionally, the screening unit is used to:

[0039] According to the action recognition category of each video clip, screening the video clips whose action recognition categories match the preset action categories among the multiple video clips as candidate clips;

[0040] According to the action recognition score of each candidate segment, the candidate segment whose action recognition score is greater than the preset action score is selected as the first segment.

[0041] Optionally, the device further comprises: a training unit;

[0042] The training unit is used to:

[0043] Obtaining a sample video, an action category label and an action score label of the sample video; the action score label represents the action clarity and action expressiveness of the sample video;

[0044] Performing action recognition on the sample video by using an initial recognition model to obtain an action prediction category and an action prediction score of the sample video;

[0045] According to the difference between the action prediction category and the action category label, and the difference between the action prediction score and the action score label, the initial recognition model is trained to obtain the action recognition model.

[0046] Optionally, the generating unit is used to:

[0047] According to the image similarity between adjacent frames of video images in the first segment, the first segment is segmented to obtain a second segment; the image similarity between adjacent frames of video images in the second segment is greater than a second similarity, and the second similarity is greater than the first similarity;

[0048] A cover is generated for the second segment to obtain the dynamic cover image.

[0049] Optionally, the video duration of the second segment is greater than a preset duration, and the preset duration is less than the video duration of the first segment.

[0050] On the other hand, an embodiment of the present application provides a computer device, the computer device comprising a processor and a memory:

[0051] The memory is used to store a computer program and transmit the computer program to the processor;

[0052] The processor is configured to execute the method described in the above aspects according to the instructions in the computer program.

[0053] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by a processor, it is used to implement the method described in the above aspects.

[0054] Compared with the prior art, this application has at least the following advantages:

[0055] According to the technical solution of the embodiment of the present application, first, the target video is divided into multiple video clips, wherein the image similarity between adjacent frames of video images in each video clip is greater than the first similarity; this method divides the target video into multiple video clips corresponding to multiple contents, providing a basis for the subsequent screening of video clips for displaying rich and important content. Then, each video clip is input into the action recognition model for action recognition, and the action recognition category and action recognition score of each video clip are output; based on the action recognition category and action recognition score of each video clip, multiple video clips are screened to obtain the first clip, wherein the action recognition category of the first clip matches the preset action category, and the action recognition score of the first clip is greater than the preset action score; this method selects the video clip with high-quality preset action from multiple video clips as the first clip, providing a basis for the subsequent generation of cover images for expressing and delivering rich and important content. Finally, a dynamic cover image of the target video is generated based on the first clip; this method generates a dynamic cover image based on the first clip having high-quality preset action, which can express and deliver the rich and important content of the target video, so that the dynamic cover image can be displayed for the target video in the future, which can fully attract the user's attention and user attention, thereby increasing the user's interest and expectations for the target video. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A schematic diagram of a system framework involved in an application scenario in an embodiment of the present application;

[0058] Figure 2 A schematic diagram of a flow chart of a method for generating a cover image of a video provided in an embodiment of the present application;

[0059] Figure 3 A schematic diagram of the steps of a method for generating a cover image of a video provided in an embodiment of the present application;

[0060] Figure 4 A schematic structural diagram of a video cover image generating device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] At present, for videos on video websites, static cover images are generated and displayed to attract user attention and user focus, so that users become interested in and expect the video. However, after research, it is found that static cover images have limitations and cannot express the rich and important content of the video, and cannot fully attract user attention and user focus, thus reducing user interest and expectations in the video.

[0063] In order to solve this problem, in an embodiment of the present application, the target video is divided into multiple video clips, wherein the image similarity between adjacent frames of video images in each video clip is greater than a first similarity; providing a basis for the subsequent screening of video clips for displaying rich and important content. Input each video clip into the action recognition model for action recognition, and output the action recognition category and action recognition score of each video clip; based on the action recognition category and action recognition score of each video clip, screen multiple video clips to obtain a first clip, wherein the action recognition category of the first clip matches a preset action category, and the action recognition score of the first clip is greater than a preset action score; providing a basis for the subsequent generation of a cover image for expressing and delivering rich and important content. Generate a dynamic cover image of the target video based on the first clip; the dynamic cover image can express and deliver the rich and important content of the target video, so that the subsequent display of the dynamic cover image for the target video can fully attract the user's attention and user focus, thereby increasing the user's interest and expectations for the target video.

[0064] For example, one of the scenarios of the embodiments of the present application may be applied to Figure 1 The scenario shown includes a client 101 and a server 102. The client 101 determines each video in the video website as a target video and provides it to the server 102, so that the server 102 executes the implementation method provided in the embodiment of the present application to obtain a dynamic cover image of the target video, so that the dynamic cover image is displayed for the target video through the client 101.

[0065] First, in the above application scenario, although the action description of the implementation method provided in the embodiment of the present application is executed by the server 102; however, the embodiment of the present application is not limited in terms of the execution subject, as long as the actions disclosed in the implementation method provided in the embodiment of the present application are executed.

[0066] Secondly, the above scenario is only an example scenario provided by the embodiment of the present application, and the embodiment of the present application is not limited to this scenario.

[0067] The specific implementation of the method, device, equipment and storage medium for generating a cover image of a video in the embodiment of the present application is described in detail below with reference to the accompanying drawings by way of example.

[0068] First, the specific implementation method of the video cover image generation method in the embodiment of the present application is described in detail through an embodiment.

[0069] See also Figure 2 , Figure 2 A schematic flow chart of a method for generating a cover image of a video provided in an embodiment of the present application. In this embodiment, the method may include, for example, the following S201-S205.

[0070] S201: Segment the target video to obtain multiple video segments; the image similarity between adjacent frame video images in each video segment is greater than a first similarity.

[0071] In the related art, a static cover image of a video on a video website is generated and displayed to attract user attention and user focus, thereby generating user interest and expectations for the video. However, static cover images have limitations and cannot express and convey the rich and important content of the video, nor can they fully attract user attention and user focus, thereby reducing user interest and expectations for the video.

[0072] Based on this, in the embodiment of the present application, each video in the video website is determined as a target video. In order to solve the above problem and make the cover image of the target video express the rich and important content of the video, it is first necessary to divide the target video into multiple video clips corresponding to multiple contents, so as to subsequently screen the video clips used to display the rich and important content, thereby generating a cover image for expressing the rich and important content of the video. Based on this, multiple frames of video images are divided into multiple video clips; the image similarity between adjacent frames of video images in each video clip is greater than the first similarity, so that multiple video clips correspond to multiple contents.

[0073] The first similarity refers to a preset lower limit similarity of the contents of two frames of video images.

[0074] In S201, the target video is divided into multiple video segments corresponding to multiple contents. The content of the same video segment remains basically unchanged, while the content of different video segments changes, which provides a basis for subsequent screening of video segments for displaying rich and important content.

[0075] As an example of S201, the target video is video X, video X includes N frames of video images, N is a positive integer, N ≥ 4, and the first similarity is similarity Y1. The server divides video X into M video segments, M is a positive integer, 2 ≤ M < N; wherein the image similarity between adjacent frames of video images in each video segment is greater than similarity Y1, and the similarity Y1 may be 0.8.

[0076] S202: Perform action recognition on each video clip through an action recognition model to obtain an action recognition category and an action recognition score for each video clip.

[0077] S203: Screening the multiple video clips to obtain a first clip according to the multiple action recognition categories and the multiple action recognition scores corresponding to the multiple video clips; the action recognition category of the first clip matches the preset action category, and the action recognition score of the first clip is greater than the preset action score.

[0078] In an embodiment of the present application, after executing the above S201 to divide the target video into multiple video clips corresponding to multiple contents, in order to solve the above problem and make the cover image of the target video express the rich and important content of the transmitted video, it is also necessary to screen out a video clip with high-quality preset actions from the multiple video clips as the first clip, so as to subsequently generate a cover image for expressing the rich and important content of the transmitted video. Based on this, considering that the action recognition model can recognize the action category and action score of the video clip, and the action category and action score of the video clip are used to indicate whether the video clip is a video clip with high-quality preset action; each video clip is input into the action recognition model for action recognition, and the action recognition category and action recognition score of each video clip are output, so as to screen multiple video clips to obtain the first clip based on the action recognition category and action recognition score of each video clip; wherein the action recognition category of the first clip matches the preset action category, and the action recognition score of the first clip is greater than the preset action score, so that the first clip is a video clip with high-quality preset action.

[0079] Among them, the action recognition model refers to a recognition model of action categories and action scores of video clips; the action recognition category of the video clip refers to the action category of the video clip recognized by the action recognition model, and the action recognition score of the video clip refers to the action score of the video clip recognized by the action recognition model; the preset action category refers to a pre-set action category that expresses the important content of the video, and the preset action score refers to a pre-set lower limit action score indicating high quality.

[0080] S202-S203 inputs each video clip into the action recognition model for action recognition, outputs the action recognition category and action recognition score of each video clip, measures whether each video clip is a video clip with high-quality preset action, and selects the video clip with high-quality preset action from multiple video clips as the first clip, providing a basis for the subsequent generation of a cover image for expressing and conveying rich and important content.

[0081] As an example of S202-S203, based on the example of S201 above, the preset action categories include action category A, action category B and action category C, and the preset action score is action score S. Then the server inputs each video clip into the action recognition model for action recognition, and outputs the action recognition category and action recognition score of each video clip, that is, the server inputs the jth video clip of M video clips into the action recognition model for action recognition, and outputs the action recognition category and action recognition score of the jth video clip, where j is a positive integer, i=1, ..., M. The server screens the M video clips based on the action recognition category and action recognition score of each video clip to obtain the first clip; that is, the server screens the M video clips based on the action recognition category and action recognition score of the jth video clip to obtain the first clip; wherein the action recognition category of the first clip matches any one of the action category A, action category B and action category C, and the action recognition score of the first clip is greater than the action score S; the action category A, action category B and action category C can be dancing, fighting and hugging, and the action score S can be 0.8.

[0082] S204: Generate a cover for the first segment to obtain a dynamic cover image of the target video.

[0083] In an embodiment of the present application, after executing the above S202-S203 to filter out a video clip with high-quality preset actions from multiple video clips as the first clip, in order to solve the above problem so that the cover image of the target video expresses the rich and important content of the conveyed video, on the basis that the first clip has high-quality preset actions, a dynamic cover image of the target video is generated based on the first clip, so that the dynamic cover image expresses the rich and important content of the conveyed target video.

[0084] Among them, the dynamic cover image refers to a cover image that changes dynamically.

[0085] Based on the first clip having high-quality preset actions, S204 generates a dynamic cover image that can dynamically change to reflect more key content, thereby being able to express and convey the rich and important content of the target video, so that the dynamic cover image can be subsequently displayed for the target video, which can fully attract user attention and user focus, thereby increasing user interest and expectations for the target video.

[0086] As an example of S204, based on the above examples of S202-S203, the server generates a dynamic cover image of video X based on the first segment.

[0087] Through various implementation methods provided by this embodiment, first, the target video is divided into multiple video clips, wherein the image similarity between adjacent frame video images in each video clip is greater than the first similarity; this method divides the target video into multiple video clips corresponding to multiple contents, providing a basis for subsequent screening of video clips for displaying rich and important content. Then, each video clip is input into the action recognition model for action recognition, and the action recognition category and action recognition score of each video clip are output; based on the action recognition category and action recognition score of each video clip, multiple video clips are screened to obtain a first clip, wherein the action recognition category of the first clip matches the preset action category, and the action recognition score of the first clip is greater than the preset action score; this method selects a video clip with high-quality preset action from multiple video clips as the first clip, providing a basis for subsequent generation of a cover image for expressing and delivering rich and important content. Finally, a dynamic cover image of the target video is generated based on the first clip; this method generates a dynamic cover image based on the first clip having high-quality preset action, which can express and deliver the rich and important content of the target video, so that the dynamic cover image can be displayed for the target video in the future, which can fully attract the user's attention and user attention, thereby increasing the user's interest and expectations for the target video.

[0088] See also Figure 3 , Figure 3 A schematic diagram of the steps of a method for generating a cover image of a video provided in an embodiment of the present application. The steps include segmentation, action recognition, segment screening, and cover generation; wherein segmentation refers to segmenting a target video into multiple video segments, and the image similarity between adjacent frame video images in each video segment is greater than a first similarity; action recognition refers to inputting each video segment into an action recognition model for action recognition, and outputting the action recognition category and action recognition score of each video segment; segment screening refers to screening multiple video segments based on the action recognition category and action recognition score of each video segment to obtain a first segment, the action recognition category of the first segment matches a preset action category, and the action recognition score of the first segment is greater than a preset action score; cover generation refers to generating a dynamic cover image of the target video based on the first segment.

[0089] In the embodiment of the present application, when the above-mentioned S201 of dividing the target video into multiple video segments is specifically implemented, considering that the greater the image similarity between adjacent frames of video images in the target video, the more similar the contents of the adjacent frames of video images in the target video are, and the smaller the image similarity between adjacent frames of video images in the target video, the less similar the contents of the adjacent frames of video images in the target video are, the image similarity between adjacent frames of video images in the target video can be calculated, so that the target video can be divided into multiple video segments based on the image similarity between adjacent frames of video images in the target video; the pixel average and pixel variance of each frame of video images in adjacent frames of video images in the target video, and the pixel covariance of adjacent frames of video images affect the image similarity between adjacent frames of video images; based on this, when adjacent frames of video images in the target video include a first video image and a second video image, the image similarity between the first video image and the second video image can be calculated by using the pixel average and pixel variance of the first video image, the pixel average and pixel variance of the second video image, and the pixel covariance of the first video image and the second video image; based on the image similarity between the first video image and the second video image, the target video is divided into multiple video segments. Therefore, in an optional implementation of the embodiment of the present application, adjacent frame video images in the target video include a first video image and a second video image; the above S201 may include, for example, the following S2011-S2012 (not shown in the figure).

[0090] S2011: performing similarity calculation on the first video image and the second video image according to the pixel average and pixel variance of the first video image, the pixel average and pixel variance of the second video image, and the pixel covariance between the first video image and the second video image to obtain image similarity between the first video image and the second video image.

[0091] S2012: According to the image similarity between the first video image and the second video image, segment the target video to obtain a plurality of video segments.

[0092] Among them, the first video image and the second video image refer to two adjacent frames of video images in the target video; the pixel average of the first video image refers to the average value of each pixel of the first video image, and the pixel variance of the first video image refers to the variance of each pixel of the first video image; the pixel average of the second video image refers to the average value of each pixel of the second video image, and the pixel variance of the second video image refers to the variance of each pixel of the second video image; the pixel covariance of the first video image and the second video image refers to the covariance of each pixel of the first video image and each pixel of the second video image; the image similarity between the first video image and the second video image refers to the similarity in content between the first video image and the second video image.

[0093] The S2011-S2012 calculates the image similarity between the first video image and the second video image through the pixel average and pixel variance of the first video image, the pixel average and pixel variance of the second video image, and the pixel covariance between the first video image and the second video image, taking into account the similarity between the first video image and the second video image in brightness, contrast and structural information, and can more comprehensively determine whether the content of adjacent frame video images has changed, thereby more accurately dividing the target video into multiple video segments corresponding to multiple contents.

[0094] As an example of S2011-S2012, based on the example of S201 above, the adjacent frames of video images in video X include a first video image x and a second video image y. Then the server calculates the image similarity between x and y as SSIM(x, y) through the pixel average and pixel variance of x, the pixel average and pixel variance of y, and the pixel covariance of x and y. Based on SSIM(x, y), the video X is segmented into M video segments; the calculation formula of SSIM(x, y) is as follows:

[0095]

[0096] Among them, μ x is the pixel average value of x, σ x is the pixel variance of x, μ y is the pixel average value of y, σ y is the pixel variance of y, σ xy is the pixel covariance of x and y, and C1 and C2 are constants used to maintain stability.

[0097] In the embodiment of the present application, when the above S2012 is specifically implemented to divide the target video into multiple video segments based on the image similarity between the first video image and the second video image, in order to make the image similarity between adjacent frame video images in each video segment greater than the first similarity, when the image similarity between the first video image and the second video image is less than or equal to the first similarity, it is necessary to divide the first video image and the second video image; based on this, for the first video image and the second video image whose image similarity is less than or equal to the first similarity, the video time point between the first video image and the second video image is determined as the segment division point of the target video, so that the target video is divided according to the segment division point to obtain multiple video segments. Therefore, in an optional implementation of the embodiment of the present application, the above S2012, for example, may include the following S2012a-S2012b (not shown in the figure).

[0098] S2012a: If the image similarity between the first video image and the second video image is less than or equal to the first similarity, determine a segment division point of the target video according to a video time point between the first video image and the second video image.

[0099] S2012b: Segment the target video according to the segmentation points to obtain multiple video segments.

[0100] The video time point between the first video image and the second video image refers to a time point between a frame time point of the first video image and a frame time point of the second video image.

[0101] In S2012a-S2012b, when the image similarity between the first video image and the second video image is less than or equal to the first similarity, it indicates that there is a change in the content of the first video image and the second video image, and the video time point between the first video image and the second video image is determined as a segmentation point of the target video to segment the target video, so as to accurately obtain multiple video segments corresponding to multiple contents.

[0102] As an example of S2012a-S2012b, based on the above example of S2011-S2012, for x and y whose image similarity is less than or equal to similarity Y1, the server determines the video time point between x and y as the segment division point of video X; and divides video X according to the segment division points to obtain M video segments.

[0103] In the embodiment of the present application, the action recognition model in the above S202 is obtained by training the initial recognition model based on the sample video, the action category label and the action score label of the sample video. Then the action recognition model training process refers to: first, obtaining the sample video as the training input data, and obtaining the action category label and the action score label of the sample video as the training target data; then, the sample video is input into the initial recognition model for action recognition to predict the action category and action score of the sample video, thereby outputting the action prediction category and action prediction score of the sample video; finally, according to the difference between the action prediction category and the action category label, and the difference between the action prediction score and the action score label, iteratively train the model parameters of the initial recognition model until the initial recognition model converges or the number of iterations reaches a preset number to complete the training, and the trained initial recognition model is determined as the action recognition model. Therefore, in an optional implementation of the embodiment of the present application, the step of obtaining the action recognition model in the above S202, for example, may include the following S1-S3 (not shown in the figure).

[0104] S1: Obtain sample videos, action category labels and action score labels of the sample videos; the action score labels represent the action clarity and action expressiveness of the sample videos.

[0105] S2: Perform action recognition on the sample video through the initial recognition model to obtain the action prediction category and action prediction score of the sample video.

[0106] S3: According to the difference between the action prediction category and the action category label, and the difference between the action prediction score and the action score label, the initial recognition model is trained to obtain the action recognition model.

[0107] Among them, the sample video refers to the video to be trained, the action category label of the sample video refers to the action category annotated by the sample video, so as to characterize the action category of the sample video, the action score label of the sample video refers to the action score annotated by the sample video, so as to characterize the action clarity and action expressiveness of the sample video; the action prediction category of the sample video refers to the action category of the sample video obtained by the initial recognition model, and the action prediction score refers to the action score of the sample video obtained by the initial recognition model.

[0108] The S1-S3 obtains a sample video, an action category label and an action score label of the sample video; inputs the sample video into an initial recognition model for action recognition, and outputs the action prediction category and action prediction score of the sample video; trains the initial recognition model according to the difference between the action prediction category and the action category label, and the difference between the action prediction score and the action score label to obtain an action recognition model, so that the action prediction category and the action prediction score are close to the action category label and the action score label, so that the initial recognition model learns the association between the sample video and the action category label and the action score label, thereby training an action recognition model that can recognize the action category and action score of the video clip.

[0109] As an example of S1-S3, based on the above examples of S202-S203, the sample video is video Z, and the action category label of video Z is class Z , the action score labels of video Z are scores Z The server obtains video Z and the class of video Z. Z and scores Z ; The server inputs video Z into the initial recognition model for action recognition and outputs the action prediction category and action prediction score of video Z; the server calculates the action prediction category and class Z The difference between the action prediction scores and the scores Z The difference between them is used to train the initial recognition model to obtain the action recognition model.

[0110] In an embodiment of the present application, when the above S203 is implemented based on the action recognition category and action recognition score of each video clip to screen multiple video clips to obtain the first clip, on the basis that the action recognition category of the first clip matches the preset action category and the action recognition score of the first clip is greater than the preset action score, screening multiple video clips can first meet the requirements of the action recognition category matching the preset action category and then meet the requirements of the action recognition score being greater than the preset action score; based on this, first, according to the action recognition category of each video clip, screen the video clips whose action recognition category matches the preset action category as candidate clips; then, according to the action recognition score of each candidate clip, screen the candidate clips whose action recognition score is greater than the preset action score as the first clip. Therefore, in an optional implementation of the embodiment of the present application, the above S203 may, for example, include the following S2031-S2032 (not shown in the figure).

[0111] S2031: According to the action recognition category of each video segment, screen video segments whose action recognition categories match a preset action category from multiple video segments as candidate segments.

[0112] S2032: According to the action recognition score of each candidate segment, select a candidate segment whose action recognition score is greater than a preset action score from the candidate segments as the first segment.

[0113] S2031-S2032 first measures whether each video clip is a video clip with preset actions according to the action recognition category of each video clip, and screens the video clips whose action recognition categories match the preset action categories as candidate clips, providing a basic basis for the subsequent generation of cover images for expressing and conveying rich and important content; then measures whether each candidate clip is a high-quality candidate clip according to the action recognition score of each candidate clip, and screens the candidate clips whose action recognition scores are greater than the preset action scores as the first clips, providing a high-quality basis for the subsequent generation of cover images for expressing and conveying rich and important content.

[0114] As an example of S2031-S2032, based on the above-mentioned examples of S202-S203, the server first screens the video clips whose action recognition categories match any one of action category A, action category B and action category C as candidate clips according to the action recognition category of each video clip; the server then screens the candidate clips whose action recognition scores are greater than the action score S as the first clip according to the action recognition score of each candidate clip.

[0115] In the embodiment of the present application, when the above S204 is specifically implemented to generate a dynamic cover image of the target video based on the first segment, on the basis that the first segment has high-quality preset actions, it is also necessary to cut out a video segment with more stable content from the first segment, so as to subsequently generate a dynamic cover image for expressing the rich and important content of the target video; based on this, considering that the greater the image similarity between adjacent frames of video images in the second segment, the more similar and stable the content of the adjacent frames of video images is, the second segment can be cut out from the first segment based on the image similarity between adjacent frames of video images in the first segment, so that the image similarity between adjacent frames of video images in the second segment is greater than the second similarity, and the second similarity is greater than the first similarity. Therefore, in an optional implementation of the embodiment of the present application, the above S204, for example, may include the following S2041-S2042 (not shown in the figure).

[0116] S2041: According to the image similarity between adjacent frames of video images in the first segment, clip the first segment to obtain a second segment; the image similarity between adjacent frames of video images in the second segment is greater than a second similarity, and the second similarity is greater than the first similarity;

[0117] S2042: Generate a cover for the second segment to obtain a dynamic cover image.

[0118] The second similarity refers to a preset lower limit similarity at which two adjacent frames of video images are similar and stable in content.

[0119] In a specific implementation, clipping the first segment to obtain the second segment means: clipping the first segment, retaining adjacent video frames whose image similarity is greater than a second similarity, and removing adjacent video frames whose image similarity is less than or equal to the second similarity.

[0120] Generating a cover for the second segment to obtain a dynamic cover image refers to: converting the video format of the second segment into a dynamic cover format to generate a dynamic cover image.

[0121] S2041-S2042 determines whether the contents of adjacent frames of video images are similar and stable through the image similarity between adjacent frames of video images in the first segment, and cuts out a second segment with more stable content from the first segment; based on the second segment having stable high-quality preset actions, a dynamic cover image is generated that can stably and dynamically reflect more key content, thereby being able to stably express and transmit the rich and important content of the target video.

[0122] As an example of S2041-S2042, based on the example of S204, the second similarity is similarity Y2, which is greater than the similarity Y1. Then, based on the image similarity between adjacent frames of video images in the first segment, the server cuts out the second segment from the first segment, so that the image similarity between adjacent frames of video images in the second segment is greater than the similarity Y2.

[0123] In the embodiment of the present application, further considering that the dynamic cover image needs to express the rich and important content of the video, the video duration of the second segment used to generate the dynamic cover image needs to be greater than the preset duration so that the second segment has more content, and the preset duration needs to be less than the video duration of the first segment. Therefore, in an optional implementation of the embodiment of the present application, the video duration of the second segment is greater than the preset duration, and the preset duration is less than the video duration of the first segment.

[0124] Next, the specific implementation method of the video cover image generation device in the embodiment of the present application is described in detail through an embodiment.

[0125] See also Figure 4 , Figure 4 A schematic diagram of the structure of a video cover image generation device provided in an embodiment of the present application. In this embodiment, the device may specifically include: a segmentation unit 401, an identification unit 402, a screening unit 403 and a generation unit 404;

[0126] The segmentation unit 401 is used to segment the target video into multiple video segments; the image similarity between adjacent frame video images in each video segment is greater than the first similarity;

[0127] The recognition unit 402 is used to perform action recognition on each video clip through an action recognition model to obtain an action recognition category and an action recognition score for each video clip;

[0128] The screening unit 403 is used to screen the multiple video segments according to the action recognition category and the action recognition score of each video segment to obtain a first segment; the action recognition category of the first segment matches the preset action category, and the action recognition score of the first segment is greater than the preset action score;

[0129] The generating unit 404 is used to generate a cover for the first segment to obtain a dynamic cover image of the target video.

[0130] In an optional implementation of the embodiment of the present application, the adjacent frame video images in the target video include a first video image and a second video image; the segmentation unit 401 is used to:

[0131] Performing similarity calculation on the first video image and the second video image according to the pixel average and pixel variance of the first video image, the pixel average and pixel variance of the second video image, and the pixel covariance between the first video image and the second video image to obtain image similarity between the first video image and the second video image;

[0132] According to the image similarity between the first video image and the second video image, the target video is segmented into segments to obtain the multiple video segments.

[0133] In an optional implementation of the embodiment of the present application, the segmentation unit 401 is used to:

[0134] If the image similarity between the first video image and the second video image is less than or equal to the first similarity, determining a segment segmentation point of the target video according to a video time point between the first video image and the second video image;

[0135] The target video is segmented according to the segmentation points to obtain the multiple video segments.

[0136] In an optional implementation of the embodiment of the present application, the screening unit 403 is used to:

[0137] According to the action recognition category of each video clip, screening the video clips whose action recognition categories match the preset action categories among the multiple video clips as candidate clips;

[0138] According to the action recognition score of each candidate segment, the candidate segment whose action recognition score is greater than the preset action score is selected as the first segment.

[0139] In an optional implementation of the embodiment of the present application, the device further includes: a training unit;

[0140] The training unit is used to:

[0141] Obtaining a sample video, an action category label and an action score label of the sample video; the action score label represents the action clarity and action expressiveness of the sample video;

[0142] Performing action recognition on the sample video by using an initial recognition model to obtain an action prediction category and an action prediction score of the sample video;

[0143] According to the difference between the action prediction category and the action category label, and the difference between the action prediction score and the action score label, the initial recognition model is trained to obtain the action recognition model.

[0144] In an optional implementation of the embodiment of the present application, the generating unit 404 is used to:

[0145] According to the image similarity between adjacent frames of video images in the first segment, the first segment is segmented to obtain a second segment; the image similarity between adjacent frames of video images in the second segment is greater than a second similarity, and the second similarity is greater than the first similarity;

[0146] A cover is generated for the second segment to obtain the dynamic cover image.

[0147] In an optional implementation of the embodiment of the present application, the video duration of the second segment is greater than a preset duration, and the preset duration is less than the video duration of the first segment.

[0148] Through the various implementation methods provided in this embodiment, the cover image generation device of the video includes a segmentation unit, an identification unit, a screening unit and a generation unit. The segmentation unit segments the target video into multiple video segments, wherein the image similarity between adjacent frame video images in each video segment is greater than a first similarity; the unit divides the target video into multiple video segments corresponding to multiple contents, providing a basis for subsequent screening of video segments for displaying rich and important content. The identification unit inputs each video segment into an action recognition model for action recognition, and outputs an action recognition category and an action recognition score for each video segment; the screening unit screens multiple video segments based on the action recognition category and action recognition score of each video segment to obtain a first segment, wherein the action recognition category of the first segment matches a preset action category, and the action recognition score of the first segment is greater than a preset action score; the unit screens a video segment with high-quality preset actions from multiple video segments as the first segment, providing a basis for subsequent generation of a cover image for expressing and conveying rich and important content. The generation unit generates a dynamic cover image of the target video based on the first segment; based on the first segment having high-quality preset actions, the unit generates a dynamic cover image that can express and convey the rich and important content of the target video, so that the dynamic cover image can be displayed for the target video later, which can fully attract user attention and user focus, thereby increasing user interest and expectations for the target video.

[0149] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0150] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0151] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprises a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0152] The above is only a preferred embodiment of the present application and does not constitute any formal limitation to the present application. Although the present application has been disclosed as above with a preferred embodiment, it is not intended to limit the present application. Any technician familiar with the art can make many possible changes and modifications to the technical solution of the present application using the methods and technical contents disclosed above without departing from the scope of the technical solution of the present application, or modify it into an equivalent embodiment of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application still fall within the scope of protection of the technical solution of the present application.

Claims

1. A method for generating a cover image of a video, characterized in that: include: The target video is segmented to obtain a plurality of video segments; the image similarity between adjacent frame video images in each video segment is greater than the first similarity; Performing action recognition on each video clip through an action recognition model to obtain an action recognition category and an action recognition score for each video clip; According to the action recognition category and the action recognition score of each video segment, the multiple video segments are screened to obtain a first segment; the action recognition category of the first segment matches a preset action category, and the action recognition score of the first segment is greater than a preset action score; A cover is generated for the first segment to obtain a dynamic cover image of the target video.

2. The method according to claim 1, characterized in that The adjacent frame video images in the target video include a first video image and a second video image; and the segmenting of the target video to obtain a plurality of video segments includes: Performing similarity calculation on the first video image and the second video image according to the pixel average and pixel variance of the first video image, the pixel average and pixel variance of the second video image, and the pixel covariance between the first video image and the second video image to obtain image similarity between the first video image and the second video image; According to the image similarity between the first video image and the second video image, the target video is segmented into segments to obtain the multiple video segments.

3. The method according to claim 2, characterized in that The step of segmenting the target video to obtain the plurality of video segments according to the image similarity between the first video image and the second video image includes: If the image similarity between the first video image and the second video image is less than or equal to the first similarity, determining a segment segmentation point of the target video according to a video time point between the first video image and the second video image; The target video is segmented according to the segmentation points to obtain the multiple video segments.

4. The method according to claim 1, characterized in that: The step of screening the plurality of video clips to obtain a first clip according to the action recognition category and the action recognition score of each video clip comprises: According to the action recognition category of each video clip, screening the video clips whose action recognition categories match the preset action categories among the multiple video clips as candidate clips; According to the action recognition score of each candidate segment, the candidate segment whose action recognition score is greater than the preset action score is selected as the first segment.

5. The method according to claim 1, characterized in that: The step of obtaining the action recognition model includes: Obtaining a sample video, an action category label and an action score label of the sample video; the action score label represents the action clarity and action expressiveness of the sample video; Performing action recognition on the sample video by using an initial recognition model to obtain an action prediction category and an action prediction score of the sample video; According to the difference between the action prediction category and the action category label, and the difference between the action prediction score and the action score label, the initial recognition model is trained to obtain the action recognition model.

6. The method according to claim 1, characterized in that Generating a cover for the first segment to obtain a dynamic cover image for the target video includes: According to the image similarity between adjacent frames of video images in the first segment, the first segment is segmented to obtain a second segment; the image similarity between adjacent frames of video images in the second segment is greater than a second similarity, and the second similarity is greater than the first similarity; A cover is generated for the second segment to obtain the dynamic cover image.

7. The method according to claim 6, characterized in that The video duration of the second segment is greater than a preset duration, and the preset duration is less than the video duration of the first segment.

8. A video cover image generation device, characterized in that: include: Segmentation unit, recognition unit, screening unit and generation unit; The segmentation unit is used to segment the target video into multiple video segments; the image similarity between adjacent frame video images in each video segment is greater than the first similarity; The recognition unit is used to perform action recognition on each video clip through an action recognition model to obtain an action recognition category and an action recognition score for each video clip; The screening unit is used to screen the multiple video clips according to the action recognition category and the action recognition score of each video clip to obtain a first clip; the action recognition category of the first clip matches a preset action category, and the action recognition score of the first clip is greater than a preset action score; The generating unit is used to generate a cover for the first segment to obtain a dynamic cover image of the target video.

9. A computer device, characterized in that: The computer device comprises a processor and a memory: The memory is used to store a computer program and transmit the computer program to the processor; The processor is configured to execute the method according to any one of claims 1 to 7 according to the instructions in the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, it is used to implement the method according to any one of claims 1 to 7.

Images