Method and device for determining base map of video cover, equipment and medium

By automatically filtering the visual modal analysis and image quality evaluation results of video frames, the problem of time and cost of determining the video cover base map is solved, and efficient automatic determination of the video cover base map is achieved, reducing the cost of generating the video cover.

CN118972678APending Publication Date: 2024-11-15BEIJING QIYI CENTURY SCI & TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410905487.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-08
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

In the prior art, the determination of the video cover base map requires manual frame selection and screening, which is time-consuming and costly, making it difficult to be efficiently completed.

Method used

By obtaining the visual modal analysis results and picture quality evaluation results of the to-process video, the target video frame is filtered as the base map of the cover according to the pre-configured conditions, and the base map is automatically determined using the pre-trained video processing service and evaluation model, without manual intervention required .

Benefits of technology

This greatly reduces the cost and time of determining the video cover base map, improves efficiency, and reduces the cost of subsequent generation of video covers.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118972678A_ABST
    Figure CN118972678A_ABST
Patent Text Reader

Abstract

The embodiment of the invention relates to a base map determination method and device of a video cover, equipment and a medium, and the method comprises the steps: obtaining a to-be-processed video, obtaining a visual modal analysis result and a picture quality evaluation result of each video frame corresponding to the to-be-processed video, and obtaining a target video frame of which the visual modal analysis result meets a pre-configured visual modal condition and the picture quality evaluation result meets a pre-configured picture quality condition from each video frame, and determining the target video frame as a base map for generating the cover of the to-be-processed video. According to the embodiment of the invention, the base map determination cost of the video cover can be well reduced, and the base map determination efficiency of the video cover is improved, so that the subsequent cost of generating the video cover based on the base map is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of multimedia processing technology, and in particular to a method, device, equipment and medium for determining a base image of a video cover. Background Art

[0002] Usually when a video is displayed on a mobile phone or computer application, a cover image is required. Usually when generating a video cover image, the base image needs to be determined first.

[0003] In the related art, the base map is determined by extracting frames from the video and manually screening the extracted frame results. The above method has a high production cost. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a method, device, equipment and medium for determining a base image of a video cover.

[0005] An embodiment of the present disclosure provides a method for determining a base map of a video cover, which is applied to a first platform, and the method includes: obtaining a video to be processed; obtaining visual modality analysis results and picture quality evaluation results of each video frame corresponding to the video to be processed; obtaining a target video frame from each video frame, wherein the visual modality analysis result satisfies a preconfigured visual modality condition and the picture quality evaluation result satisfies a preconfigured picture quality condition, and determining the target video frame as the base map for generating the cover of the video to be processed.

[0006] Optionally, obtaining the visual modal analysis results of each video frame corresponding to the video to be processed includes: performing object recognition on the video to be processed based on a pre-trained video object recognition service to obtain first object information of each video frame; and / or, performing face analysis on the video to be processed based on a pre-trained video face analysis service to obtain first face information of each video frame; and / or, performing dialogue recognition on the video to be processed based on a pre-trained video dialogue recognition service to obtain dialogue information of each video frame; and / or, performing video transition point analysis on the video to be processed based on a pre-trained video transition point analysis service to obtain transition information of each video frame.

[0007] Optionally, obtaining the image quality evaluation results of each video frame corresponding to the video to be processed includes: performing image quality evaluation on each video frame based on a pre-trained first evaluation model to obtain the quality evaluation results of each video frame; and / or performing image visual evaluation on each video frame based on a pre-trained second evaluation model to obtain the visual evaluation results of each video frame; and / or performing inter-image motion vector calculation on each video frame based on a pre-trained third evaluation model to obtain the relative motion vector of each video frame.

[0008] Optionally, the step of obtaining a target video frame from each of the video frames, the visual modality analysis result of which satisfies a preconfigured visual modality condition and the picture quality assessment result of which satisfies a preconfigured picture quality condition, and determining the target video frame as a base map for generating a cover of the video to be processed comprises: obtaining a candidate video frame from each of the video frames, the first object information, and / or the first face information, and / or the dialogue information, and / or the transition information of which satisfies an object condition, and / or a face condition, and / or a dialogue condition, and / or a transition condition; obtaining a target video frame from the candidate video frames, the quality assessment result, and / or the visual assessment result, and / or the relative motion vector of which satisfies a quality assessment condition, and / or a visual assessment condition, and / or a relative motion condition, and determining the target video frame as a base map for generating a cover of the video to be processed.

[0009] Optionally, after determining the target video frame as the base image for generating the cover of the video to be processed, the method further includes: performing image icon detection on the base image based on a pre-trained image icon detection service to obtain icon information of the base image; obtaining a first screenshot size and whether the target screenshot contains an icon based on a pre-configured cropping strategy; and cropping the base image according to the first screenshot size and whether the target screenshot contains an icon based on the icon information of the base image to obtain a first target screenshot.

[0010] Optionally, after determining the target video frame as the base map for generating the cover of the video to be processed, the method further includes: performing image object recognition on the base map based on a pre-trained image object recognition service to obtain second object information of the base map; obtaining a second screenshot size and a target object identification and a target object quantity of the target screenshot based on a pre-configured cropping strategy; and cropping the base map according to the second screenshot size and the target object identification and the target object quantity of the target screenshot based on the second object information of the base map to obtain a second target screenshot.

[0011] Optionally, after determining the target video frame as the base image for generating the cover of the video to be processed, the method further includes: performing face recognition on the base image based on a pre-trained image face recognition service to obtain second face information of the base image; obtaining a third screenshot size and target face information of the target screenshot based on a pre-configured cropping strategy; and cropping the base image according to the third screenshot size and the target face information of the target screenshot based on the second face information of the base image to obtain a third target screenshot.

[0012] Optionally, after determining the target video frame as the base map for generating the cover of the video to be processed, the method further includes: performing optical character recognition on the base map based on a pre-trained image optical character recognition service to obtain optical character information of the base map; obtaining a fourth screenshot size and whether the target screenshot contains optical characters based on a pre-configured cropping strategy; and cropping the base map according to the fourth screenshot size and whether the target screenshot contains optical characters based on the optical character information of the base map to obtain a fourth target screenshot.

[0013] Optionally, the method further includes: storing the cover of the video to be processed generated based on the base map in a preset storage server, wherein the terminal obtains and displays the cover of the video to be processed from the storage server via the video server.

[0014] Optionally, the method further includes: receiving an operation control request sent by the second platform; and adjusting a frequency of acquiring the video to be processed based on the operation control request.

[0015] The disclosed embodiment also provides a device for determining a base map of a video cover, which is applied to a first platform, and the device includes: a video acquisition module, used to acquire a video to be processed; a first video processing module, used to acquire a visual modal analysis result of each video frame corresponding to the video to be processed; a second video processing module, used to acquire a picture quality evaluation result of each video frame corresponding to the video to be processed; and a picture selection module, used to acquire, from the video frames, a target video frame whose visual modal analysis result satisfies a preconfigured visual modal condition and whose picture quality evaluation result satisfies a preconfigured picture quality condition, and determine the target video frame as the base map for generating the cover of the video to be processed.

[0016] An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the method for determining the base image of the video cover provided in the embodiment of the present disclosure.

[0017] The embodiment of the present disclosure also provides a computer-readable storage medium, which stores a computer program, and the computer program is used to execute the method for determining the base map of the video cover as provided in the embodiment of the present disclosure.

[0018] The above-mentioned technical solution provided by the embodiment of the present disclosure first obtains the visual modal analysis results and image quality evaluation results of each video frame corresponding to the video to be processed, and obtains the target video frame whose visual modal analysis results meet the preconfigured visual modal conditions and whose image quality evaluation results meet the preconfigured image quality conditions from each video frame to determine it as the base map for generating the cover of the video to be processed. The above-mentioned method does not require manual selection of the base map, which can greatly reduce the cost of determining the base map of the video cover, improve the efficiency of determining the base map of the video cover, thereby reducing the subsequent cost of generating the video cover based on the base map.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] Figure 1 A schematic diagram of a flow chart of a method for determining a base image of a video cover provided by an embodiment of the present disclosure;

[0023] Figure 2 A schematic diagram of a flow chart of a method for determining a base image of a video cover provided by an embodiment of the present disclosure;

[0024] Figure 3 A schematic diagram of a process for determining a base image of a video cover provided by an embodiment of the present disclosure;

[0025] Figure 4 A schematic diagram of a process for determining a base image of a video cover provided by an embodiment of the present disclosure;

[0026] Figure 5 A schematic diagram of a process for determining a base image of a video cover provided by an embodiment of the present disclosure;

[0027] Figure 6 A schematic diagram of the structure of a system for determining a base image of a video cover provided by an embodiment of the present disclosure;

[0028] Figure 7 A schematic diagram of the structure of a device for determining a base image of a video cover provided by an embodiment of the present disclosure;

[0029] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0031] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0032] Based on the description of the above background technology, when a video is displayed in a mobile phone or computer application, a cover image is needed to display the video information. When generating a video cover image, the base image needs to be determined first. The existing method extracts frames from the video and manually screens the extracted frame results to determine the base image. There are technical problems such as a long process time-consuming and high cost, and it is difficult to efficiently determine the base image of the video cover.

[0033] Based on the above problems, the disclosed embodiment first obtains the visual modality analysis results and image quality evaluation results of each video frame corresponding to the video to be processed, and obtains the target video frame whose visual modality analysis results meet the preconfigured visual modality conditions and whose image quality evaluation results meet the preconfigured image quality conditions from each video frame to determine as the base map for generating the cover of the video to be processed. The above method does not require manual selection of the base map, which can greatly reduce the cost of determining the base map of the video cover, improve the efficiency of determining the base map of the video cover, and thus reduce the cost of subsequently generating the video cover based on the base map. The following is a detailed explanation:

[0034] Figure 1 The present invention provides a flow chart of a method for determining a base image of a video cover, which can be applied to a first platform, such as a computer, a mobile phone, a tablet computer, a television, a server, etc., without limitation. Figure 1 As shown, the method mainly includes the following steps S102 to S106:

[0035] Step S102, obtaining the video to be processed.

[0036] In the embodiments of the present disclosure, there are many ways to obtain videos to be processed. As an example, video online information is monitored in real time, and new videos that are online are obtained as videos to be processed, so that the base map of the video cover can be quickly determined for the new videos that are online. The base map corresponding to the new video can also be stored so that the base map can be quickly obtained when used to make the video cover, thereby improving the efficiency of video cover generation and the efficiency of video cover display. As another example, videos uploaded by other platforms or terminals are received as videos to be processed. The above are only examples of ways to obtain videos to be processed, and the embodiments of the present disclosure do not make specific limitations.

[0037] It should be noted that the video to be processed is a decrypted video. Therefore, when the above-mentioned new video or uploaded video is an encrypted video, the encrypted video needs to be decrypted as the video to be processed.

[0038] It is understandable that the first platform can simultaneously determine the background image of the video cover for one or more videos. Therefore, one or more videos can be selected as videos to be processed according to actual application processing needs, that is, the videos to be processed can be one or more.

[0039] Step S104, obtaining visual modality analysis results and picture quality evaluation results of each video frame corresponding to the video to be processed.

[0040] In an embodiment of the present disclosure, after obtaining the video to be processed, the visual modality analysis results and the picture quality evaluation results of each video frame corresponding to the video to be processed are obtained; wherein the visual modality analysis results may be one or more of the object information, face information, dialogue information and transition information of each video frame; the picture quality evaluation results may be one or more of the quality evaluation results, visual evaluation results and relative motion vectors of each video frame.

[0041] In the disclosed embodiment, one or more pre-trained video processing services can be used to simultaneously process the same video to be processed, and obtain visual modality analysis results of each video frame. The pre-trained video processing service can be one or more of a pre-trained video object recognition service, a video face analysis service, a video dialogue recognition service, and a video transition point analysis service.

[0042] In the disclosed embodiment, one or more pre-trained evaluation models can be used to simultaneously process the same video to be processed, and obtain the image quality evaluation results of each video frame. The pre-trained evaluation model can be one or more of a first evaluation model for quality evaluation such as pre-trained image clarity and blur, a second evaluation model for aesthetic evaluation such as image light and shadow, and overall beautiful matching, and a third evaluation model for calculating motion vectors between images.

[0043] For example, by simultaneously performing video object recognition and video dialogue recognition on the processed video, and performing image quality evaluation on the processed video, it is possible to obtain which objects (such as people, animals, furniture, and plants, etc.) each video frame includes, which dialogues, and the image quality score.

[0044] In the disclosed embodiment, in order to improve the effect of determining the background image of the video cover, it is necessary to improve the visual modal analysis results and picture quality evaluation results of each video frame. Therefore, one or more video processing services and evaluation models can be selected to simultaneously perform visual modal analysis and picture quality evaluation on the same video to be processed, and obtain the visual modal analysis results and picture quality evaluation results corresponding to each video frame of the video to be processed.

[0045] It should be noted that, in the process where multiple pre-trained video processing services and evaluation models simultaneously perform visual modal analysis and image quality evaluation on the same video to be processed, the multiple pre-trained video processing services and evaluation models run independently. Therefore, when multiple pre-trained video processing services and evaluation models run independently and process the same video to be processed at the same time, it is ensured that each video frame is aligned, so that the visual modal analysis results and image quality evaluation results of each video are also aligned, thereby ensuring the accuracy and effect of the background image determination of the video cover.

[0046] Step S106, obtaining a target video frame from each video frame whose visual modality analysis result satisfies a preconfigured visual modality condition and whose picture quality evaluation result satisfies a preconfigured picture quality condition, and determining the target video frame as a base image for generating a cover of the video to be processed.

[0047] In the embodiments of the present disclosure, different visual modal analysis results correspond to different visual modal conditions, which can be selected and set according to actual application needs. For example, the visual modal conditions can be one or more of object conditions, face conditions, dialogue conditions, and transition conditions; different picture quality evaluation results correspond to different picture quality conditions, which can be selected and set according to actual application needs. For example, the picture quality conditions can be one or more of quality evaluation conditions, visual evaluation conditions, and relative motion conditions.

[0048] In the disclosed embodiment, one or more base images may be selected based on the visual modality analysis results and the picture quality evaluation results of each video frame and the preconfigured visual modality conditions and picture quality conditions.

[0049] For example, the visual modal conditions are number 1 and actor A, face size B, expression C, open eyes, and one picture is retained for the same video transition; the picture quality conditions are picture quality score greater than or equal to D and aesthetic evaluation score greater than or equal to E; the visual modal analysis results and picture quality evaluation results of video frames X and Y meet the specific contents of the aforementioned visual modal conditions and picture quality conditions, and belong to the same video transition. Video frames X and video Y can be randomly retained, or in order to further improve the efficiency of determining the background image of the video cover, video frames with relatively high picture quality scores and aesthetic evaluation scores can be retained. The specific settings can be selected according to the needs of the application scenario.

[0050] In the above embodiment, the visual modal analysis results and the picture quality evaluation results of each video frame corresponding to the video to be processed are first obtained, and the target video frame whose visual modal analysis result satisfies the preconfigured visual modal condition and the picture quality evaluation result satisfies the preconfigured picture quality condition is obtained from each video frame to be determined as the base map for generating the cover of the video to be processed. The above method does not require manual selection of the base map, which can greatly reduce the cost of determining the base map of the video cover and improve the efficiency of determining the base map of the video cover, thereby reducing the cost of subsequently generating the video cover based on the base map.

[0051] In some embodiments, obtaining visual modality analysis results of each video frame corresponding to the video to be processed includes: performing object recognition on the video to be processed based on a pre-trained video object recognition service to obtain first object information of each video frame; and / or performing face analysis on the video to be processed based on a pre-trained video face analysis service to obtain first face information of each video frame; and / or performing dialogue recognition on the video to be processed based on a pre-trained video dialogue recognition service to obtain dialogue information of each video frame; and / or performing video transition point analysis on the video to be processed based on a pre-trained video transition point analysis service to obtain transition information of each video frame.

[0052] In practical applications, in order to improve the efficiency of determining the background image of the video cover while ensuring the effect of determining the background image of the video cover, multiple pre-trained video processing services can be used to process the videos to be processed at the same time to obtain the visual modal analysis results of each video frame.

[0053] In the embodiments of the present disclosure, a video object recognition service is pre-trained to obtain video samples, each of which has a corresponding object label, and the video object recognition service is trained on the video samples through a neural network (such as a convolutional neural network), thereby obtaining a pre-trained video object recognition service that can directly perform video object recognition on the processed video and obtain the first object information of each video frame.

[0054] The first object information may be understood as which objects are included in each video frame, that is, video frames with object labels. For example, the first object information of a video frame includes objects such as people, tables, and chairs.

[0055] In the embodiments of the present disclosure, a video face analysis service is pre-trained to obtain video samples, each of which has a corresponding face label, and the video face analysis service is trained on the video samples through a neural network (such as a convolutional neural network), thereby obtaining a pre-trained video face analysis service that can directly perform face analysis on the processed video and obtain the first face information of each video frame.

[0056] Among them, the first face information can be understood as information such as the number of faces, face size, expression, and eye opening and closing in each video frame.

[0057] In the disclosed embodiment, a video dialogue recognition service is pre-trained to obtain video samples, each of which has a corresponding dialogue label, and the video samples are trained for the video dialogue recognition service through a neural network (such as a convolutional neural network), thereby obtaining a pre-trained video dialogue recognition service that can directly perform dialogue recognition on the processed video and obtain the dialogue information of each video frame.

[0058] The dialogue information may be understood as information such as whether each video frame includes dialogues, the specific dialogue content, and the position of the dialogues in the video frame.

[0059] In an embodiment of the present disclosure, a video transition point analysis service is pre-trained to obtain video samples, each of which has a corresponding transition point label, and the video transition point analysis service is trained on the video samples through a neural network (such as a convolutional neural network), thereby obtaining a pre-trained video transition point analysis service that can directly perform video transition point analysis on the video to be processed and obtain transition information of each video frame.

[0060] The transition information may be understood as information such as which video scene each video frame belongs to, which video frame is a transition video frame, and so on.

[0061] Therefore, by pre-training one or more video processing services, the visual modal analysis results of each video frame can be obtained, such as one or more of the object information, face information, dialogue information and transition information of each video frame, so that one or more visual modal analysis results can be selected for screening according to the base map of the video cover to further meet the subsequent video cover production needs.

[0062] In some embodiments, obtaining the image quality evaluation results of each video frame corresponding to the video to be processed includes: performing image quality evaluation on each video frame based on a pre-trained first evaluation model to obtain the quality evaluation results of each video frame; and / or performing image visual evaluation on each video frame based on a pre-trained second evaluation model to obtain the visual evaluation results of each video frame; and / or performing inter-image motion vector calculation on each video frame based on a pre-trained third evaluation model to obtain the relative motion vector of each video frame.

[0063] In the embodiment of the present disclosure, frame extraction is performed on the video to be processed, and obtaining multiple frame extraction results can be understood as obtaining each video frame corresponding to the video to be processed, each frame extraction result corresponds to a video frame, and then one or more picture quality evaluations such as picture quality evaluation, picture visual evaluation and inter-image motion vector calculation are performed on each video frame.

[0064] In the disclosed embodiment, the first evaluation model can be trained by manually evaluating the clarity, blurriness and other quality evaluation scores of each video frame in advance, so that the first evaluation model can directly evaluate the image quality of each video frame and obtain the quality evaluation result of each video frame. Similarly, the second evaluation model can be trained by manually evaluating the light and shadow, overall aesthetic matching and other aesthetic evaluation scores of each video frame in advance, so that the second evaluation model can directly evaluate the image visually of each video frame and obtain the visual evaluation result of each video frame.

[0065] In the disclosed embodiment, the inter-image motion vector calculation can be understood as calculating the relative motion vector between two images, that is, whether the current video frame is moving or still compared with the previous video frame. Image samples can be acquired in advance, each image sample has a relative motion vector label, and the image samples are trained by a convolutional neural network to obtain a third evaluation model, so that the third evaluation model can directly calculate the inter-image motion vector for each video frame to obtain the relative motion vector of each video frame.

[0066] Therefore, by pre-training one or more evaluation models, the image quality evaluation results of each video frame can be obtained, such as one or more of the quality evaluation results, visual evaluation results and relative motion vectors of each video frame, so that one or more image quality evaluation results can be selected for screening according to the base map of the video cover to further meet the subsequent video cover production needs.

[0067] Based on the description of the foregoing embodiments, it can be understood that after obtaining the visual modality analysis results and image quality evaluation results of each video frame corresponding to the video to be processed, it is necessary to screen each video frame according to the preconfigured visual modality conditions and image quality conditions to determine the target video frame as the base image for generating the cover of the video to be processed.

[0068] In some embodiments, a target video frame whose visual modality analysis result satisfies a preconfigured visual modality condition and whose picture quality evaluation result satisfies a preconfigured picture quality condition is obtained from each video frame, and the target video frame is determined as a base map for generating a cover of the video to be processed, including: obtaining candidate video frames whose first object information, and / or first face information, and / or dialogue information, and / or transition information satisfy the object condition, and / or face condition, and / or dialogue condition, and / or transition condition from each video frame, obtaining a target video frame whose quality evaluation result, and / or visual evaluation result, and / or relative motion vector satisfy the quality evaluation condition, and / or visual evaluation condition, and / or relative motion condition from the candidate video frames, and determining the target video frame as a base map for generating a cover of the video to be processed.

[0069] In the embodiments of the present disclosure, one or more of the first object information, first face information, dialogue information and transition information corresponding to each video frame can be selected as the visual modal analysis result according to actual screening needs, so as to determine one or more of the visual modal conditions such as object conditions, face conditions, dialogue conditions and transition conditions for screening according to the visual modal analysis results.

[0070] As an example, first object information and first face information corresponding to each video frame are obtained, and video frames whose first object information satisfies the corresponding object condition and whose first face information satisfies the face condition are determined as candidate video frames.

[0071] For example, the first object information corresponding to the video frame Z1 is obtained to include actor S1, animal S2 and stool S3, and the first facial information is facial size S4 and facial expression S5; the first object information corresponding to the video frame Z1 is obtained to include actor S1 and water cup S6, and the first facial information is facial size S4 and facial expression S7, and the pre-configured object condition is actor S1, so that both video frame Z1 and video frame Z2 meet the object condition, and the pre-configured facial condition is facial size S4 and facial expression S5, so that video frame Z1 meets the object condition, and video frame Z1 is determined to be a candidate video frame.

[0072] As another example, first face information and transition information corresponding to each video frame are obtained, and video frames whose first face information satisfies the face condition and whose transition information satisfies the transition condition are determined as candidate video frames.

[0073] For example, the first facial information corresponding to video frame Z3 is obtained as face size S8 and face position S9, and the transition information is that video frame Z3 is a transition video frame S10; the first facial information corresponding to video frame Z4 is obtained as face size S8 and face position S9, and the transition information is that video frame Z3 is a transition video frame S11, and the pre-configured face conditions are face size S8 and face position S9, so that video frame Z3 and video frame Z4 both meet the face conditions, and the pre-configured transition condition is a transition video frame S10, so that video frame Z3 meets the transition condition, and video frame Z3 is determined to be a candidate video frame.

[0074] Furthermore, the candidate video frames are further screened to determine the target video frames. According to actual screening requirements, one or more of the quality evaluation results, visual evaluation results and relative motion vectors corresponding to each candidate video frame can be selected as the picture quality evaluation results, thereby determining the picture quality conditions such as one or more of the quality evaluation conditions, visual evaluation conditions and relative motion conditions for screening according to the picture quality evaluation results.

[0075] Continuing with the candidate video frames obtained above, namely video frame Z1 and video frame Z3, the quality evaluation result corresponding to video frame Z1 is clarity S12, the visual evaluation result is light and shadow S13, and the relative motion vector is motion S14; the quality evaluation result corresponding to video frame Z3 is clarity S12, the visual evaluation result is light and shadow S15, and the relative motion vector is still S16; the pre-configured quality evaluation conditions are clarity S12, the visual evaluation condition light and shadow S13, and the relative motion condition is motion S14, so that video frame Z1 simultaneously satisfies the quality evaluation condition, the visual evaluation condition, and the relative motion condition, and the target video frame is determined to be video frame Z1, and video frame Z1 is determined to be the base map for generating the cover of the video to be processed.

[0076] Therefore, the corresponding visual modal conditions and image quality conditions can be determined according to different visual modal analysis results and different image quality evaluation results to screen the target video frames as the base map for generating the cover of the video to be processed, so as to quickly and accurately determine the base map of the cover of the video to be processed, thereby effectively reducing the cost of determining the base map of the video cover and improving the efficiency of determining the base map of the video cover.

[0077] Based on the description of the foregoing embodiment, after determining the base image for generating the cover of the video to be processed, the base image can be further cropped according to the needs according to the corresponding cropping strategy, so as to obtain the target screenshot for subsequent video cover generation; wherein, the number of target screenshots is greater than or equal to the number of base images.

[0078] In practical applications, after selecting the base map, cropping is required to ensure the generation effect of the video cover while improving the generation efficiency of the video cover. The base map can be processed simultaneously through multiple pre-trained image processing services to improve the subsequent cropping efficiency and effect.

[0079] In some embodiments, after the target video frame is determined as the base image for generating the cover of the video to be processed, image icon detection is performed on the base image based on a pre-trained image icon detection service to obtain icon information of the base image, and the first screenshot size and whether the target screenshot contains an icon are obtained based on a pre-configured cropping strategy. The base image is cropped according to the first screenshot size and whether the target screenshot contains an icon based on the icon information of the base image to obtain a first target screenshot.

[0080] In the disclosed embodiments, the image icon detection service is pre-trained, and the image icon detection service can be trained on picture samples through neural networks or other means, so that the image icon detection service can be directly performed on the base map to obtain the icon information of the base map; wherein, the icon information can be understood as information such as whether the base map includes an icon and which specific icon it is.

[0081] Further, based on the preconfigured cropping strategy, the first screenshot size and whether the target screenshot contains an icon are obtained, for example, the target screenshot contains an icon, so that the target image area including the icon is determined in the base image according to the first screenshot size and cropped to obtain the first target screenshot.

[0082] In some embodiments, after the target video frame is determined as the base map for generating the cover of the video to be processed, image object recognition is performed on the base map based on a pre-trained image object recognition service to obtain second object information of the base map, and the second screenshot size and the target object identification and the target object quantity of the target screenshot are obtained based on a preconfigured cropping strategy. The base map is cropped according to the second screenshot size and the target object identification and the target object quantity of the target screenshot based on the second object information of the base map to obtain a second target screenshot.

[0083] In the disclosed embodiment, the image object recognition service is pre-trained, and the image object recognition service can be trained on picture samples through neural networks or other means, so that the image object recognition service can be directly performed on the base map to obtain the second object information of the base map; wherein the second object information can be understood as which objects the base map includes and the quantity of each object, etc. For example, the second object information of a base map includes objects such as two people, a table and a chair.

[0084] Furthermore, based on the preconfigured cropping strategy, the second screenshot size and the target object identifier and the number of target objects of the target screenshot are obtained. For example, the target screenshot contains an object identifier S17, so that the target image area including an object identifier S17 is determined in the base image according to the second screenshot size and cropped to obtain a second target screenshot.

[0085] In some embodiments, after the target video frame is determined as the base image for generating the cover of the video to be processed, face recognition is performed on the base image based on a pre-trained image face recognition service to obtain second face information of the base image, a third screenshot size and target face information of the target screenshot are obtained based on a pre-configured cropping strategy, and the base image is cropped according to the third screenshot size and the target face information of the target screenshot based on the second face information of the base image to obtain a third target screenshot.

[0086] In the disclosed embodiments, the image face recognition service is pre-trained, and the image face recognition service can be trained on picture samples through neural networks or other means, so that the face recognition service can be directly performed on the base image to obtain the second face information of the base image, wherein the second face information can be understood as information such as the face size, expression, and eye opening and closing in the base image.

[0087] Furthermore, based on the preconfigured cropping strategy, the third screenshot size and the target face information of the target screenshot are obtained, for example, the target face information of the target screenshot is the face size S18 and the eyes open S19, so that the target image area including the face size S18 and the eyes open S19 is determined in the base image according to the third screenshot size and cropped to obtain the third target screenshot.

[0088] In some embodiments, after the target video frame is determined as the base image for generating the cover of the video to be processed, optical character recognition is performed on the base image based on a pre-trained image optical character recognition service to obtain optical character information of the base image, a fourth screenshot size and whether the target screenshot contains optical characters are obtained based on a pre-configured cropping strategy, and the base image is cropped according to the fourth screenshot size and whether the target screenshot contains optical characters based on the optical character information of the base image to obtain a fourth target screenshot.

[0089] In the embodiments of the present disclosure, the image optical character recognition service can be pre-trained, and the image optical character recognition service can be trained on picture samples through neural networks or other means, so that the image optical character recognition service can be directly performed on the base map to obtain the optical character information of the base map, wherein the optical character information can be understood as whether the base map includes optical character recognition, and includes information such as the specific content of the optical character recognition.

[0090] Furthermore, based on the preconfigured cropping strategy, the fourth screenshot size and whether the target screenshot contains optical characters are obtained, for example, the target screenshot does not contain the optical character S20, so that the target image area that does not include the optical character S20 is determined in the base image according to the fourth screenshot size and cropped to obtain a fourth target screenshot.

[0091] In the disclosed embodiment, when a base map is obtained, it means that a video to be processed has been split into individual video frames and one or more base maps have been selected from each video frame, and then a target screenshot that meets the requirements (such as image clarity, blurriness, image quality score, etc.) can be further obtained from the selected base map.

[0092] First, the same base image can be processed simultaneously by one or more pre-trained image processing services to obtain the target screenshot. The pre-trained image processing service can be one or more of an image icon detection service, an image object recognition service, an image face recognition service, and an image optical character recognition service, so as to obtain one or more of the icon information, the second object information, the second face information, and the optical character information of the base image.

[0093] For example, by performing image object recognition and image optical character recognition on the base map at the same time, it is possible to obtain which objects (such as people, animals, etc.) and which optical character recognitions (characters such as Chinese, English, and numbers) the base map includes. In the embodiment of the present disclosure, in order to improve the effect of determining the base map of the video cover, multiple pre-trained image processing services can be selected to process the same base map at the same time to obtain one or more of icon information, second object information, second face information, and optical character information.

[0094] It should be noted that, in the process where multiple pre-trained image processing services process the same base map at the same time, the multiple pre-trained image processing services run independently. Therefore, when multiple pre-trained image processing services run independently and process the same base map at the same time, the accuracy and effect of the base map determination of the video cover are guaranteed.

[0095] In the disclosed embodiment, the preconfigured cropping strategy can be selected and set according to actual application needs, such as the screenshot size (i.e., the output image size), whether icons are allowed to appear in the screenshot (output image), the target object identification (e.g., a person or an object, etc.) and the number of target objects allowed to appear, the facial information of the screenshot (output image) (e.g., face position, size, angle, expression, brightness, sharpness, etc.), whether the screenshot (output image) contains optical characters (optical character recognition filtering), etc.

[0096] In the disclosed embodiment, one or more target screenshots (the first target screenshot, the second target screenshot, the third target screenshot and the fourth target screenshot described above, etc.) can be cropped according to the preconfigured cropping strategy and the icon information, the second object information, the second face information and the optical character information of the base map.

[0097] It should be noted that the first screenshot size, the second screenshot size, the third screenshot size and the fourth screenshot size may be the same or different, and one or more of the aforementioned screenshot methods may be selected to perform screenshot processing on the base image.

[0098] For example, the cropping strategy is 2 people and actors F1 and F2, the face size is G, the expression is H, and the body accounts for the proportion of the picture I; the target object identification in the second object information of the base image is actors F1 and F2 and the number of target objects is 2, and the face size in the second face information of the base image is G, and the body accounts for the proportion of the picture I, so the base image is cropped according to the screenshot size to obtain the target screenshot.

[0099] It is understandable that after a base image is cropped, two or more target screenshots can be obtained, so the number of target screenshots is greater than or equal to the number of base images. The cropping process is performed according to the actual application scenario.

[0100] In some embodiments, the method further includes: storing the cover of the video to be processed generated based on the base map to a preset storage server, wherein the terminal obtains and displays the cover of the video to be processed from the storage server through the video server.

[0101] In the disclosed embodiment, the base image can be directly processed to generate the cover of the video to be processed, or the base image can be cropped to obtain a target screenshot, and the target screenshot can be processed to generate the cover of the video to be processed. The specific settings are selected according to the application scenario.

[0102] In the disclosed embodiment, the clarity, blurriness, brightness, etc. of the background image are adjusted to improve the display effect of the cover image, and relevant icons may be added to identify the playback platform to which the video belongs in order to meet application needs.

[0103] Specifically, filtering, denoising and other processing are performed on the base map to make a blurry image clearer, and then the image brightness is enhanced through image enhancement algorithms such as histogram equalization to brighten darker images. Finally, according to pre-configured icon configuration rules (such as icon position, icon size, and alignment, such as alignment of icons with human face lines of sight, etc.), icons are pasted at corresponding positions on the base map to obtain one or more covers.

[0104] Specifically, the cover may be stored in a preset storage server, such as a cloud server, and when the user uses the terminal, the terminal may obtain the stored cover from the storage server through the video server for display.

[0105] It can be understood that when the terminal needs to display the cover of a certain video, the terminal sends a display request to the video server. The video server responds to the request and determines the target cover to send to the terminal based on the popularity of each cover or the preference matching degree of each cover with the user identifier carried in the request. The terminal completes the display, thereby improving the click rate of the video.

[0106] Figure 2 This is a flow chart of another method for determining a base image of a video cover provided by an embodiment of the present disclosure. Based on the above embodiment, this embodiment further optimizes the above method for determining a base image of a video cover. Figure 2 As shown, the method mainly includes the following steps S202 to S210:

[0107] Step S202, obtaining the video to be processed, and obtaining the first object information, first face information, dialogue information and transition information of each video frame corresponding to the video to be processed, as well as the quality evaluation result, visual evaluation result and relative motion vector of each video frame corresponding to the video to be processed.

[0108] Step S204, obtaining candidate video frames whose first object information, first face information, dialogue information and transition information satisfy the object condition, face condition, dialogue condition and transition condition from each video frame.

[0109] Step S206, obtaining quality evaluation results, visual evaluation results, and a target video frame whose relative motion vector satisfies quality evaluation conditions, visual evaluation conditions, and relative motion conditions from the candidate video frames, and determining the target video frame as the base image for generating the cover of the video to be processed.

[0110] Step S208, obtain the icon information, second object information, second face information and optical character information corresponding to the base image, and obtain the target screenshot size, whether the target screenshot contains icons, the target object identification and target object quantity of the target screenshot, the target face information of the target screenshot and whether the target screenshot contains optical characters based on the preconfigured cropping strategy.

[0111] Step S210, based on the icon information, the second object information, the second face information, and the optical character information of the base image, the base image is cropped according to the target screenshot size, whether the target screenshot contains icons, the target object identification and the number of target objects in the target screenshot, the target face information of the target screenshot, and whether the target screenshot contains optical characters to obtain the target screenshot.

[0112] In the above embodiment, there is no need to manually select a base image and there is no need to manually crop the target screenshot. Finally, the base image and / or the target screenshot are processed to obtain a cover image. The above method does not require manual production of the cover. The cover image can be directly selected, screenshoted and processed according to the video, which can greatly reduce the cost of video cover production (time, labor and other costs) and improve the efficiency of video cover production.

[0113] Specifically, after obtaining the video to be processed, multiple video processing services can be executed in parallel to save time, such as Figure 3 As shown in the figure, the visual modality analysis results such as video object recognition, video face analysis, video dialogue recognition, video transition point analysis, and video frame extraction (the frame extraction results are used for picture quality evaluation, picture visual evaluation, and picture quality evaluation results such as motion vector calculation between images, which will also be executed in parallel), and finally the output results of all services can be input into the picture selection algorithm together with the picture selection configuration file (visual modality conditions and picture quality conditions), and the picture selection algorithm can then select pictures to obtain a base map that meets the rules (satisfies the visual modality conditions and picture quality conditions).

[0114] The image selection algorithm can be understood as performing image selection operations based on visual modality conditions and image quality conditions and the visual modality analysis results and image quality evaluation results of each video frame. The visual modality conditions include the number of people and actors allowed to appear in the image, the position, size, score, blurriness, brightness, sharpness, expression, and open or closed eyes of the face. The image quality conditions include the image quality score and aesthetic score, the image similarity deduplication strategy, etc. You can also choose to increase or decrease specific visual modality conditions and image quality conditions according to the actual application scenario.

[0115] In some embodiments, in order to improve the efficiency of determining the background image of the video cover, the visual modal analysis results and image quality evaluation results of each video frame can also be stored on a cloud server. Thus, when the processed video needs to be analyzed again, the visual modal analysis results and image quality evaluation results can be directly obtained, saving time and computing resources to the greatest extent.

[0116] Specifically, after obtaining the base map, multiple image processing services can be executed in parallel to save time, such as Figure 4 As shown, the base image selected by the image selection module is subjected to image logo detection, image object recognition, image face recognition, and image optical character recognition (OCR) recognition, and finally the execution result plus the cropping strategy are input into the cropping algorithm at the same time to obtain the target screenshot.

[0117] The cropping algorithm can be understood as taking screenshots based on the cropping strategy and the image processing results of the base image. The cropping strategy includes the output image size, the number of actors and people allowed to appear, the face position, size, angle, expression, brightness, sharpness, quality, the proportion of the body to the base image, passerby filtering, optical character recognition filtering, etc. The specific settings are selected according to the application scenario.

[0118] In practical applications, after the picture is captured, it needs to be processed. In order to ensure the background image determination effect of the video cover while improving the efficiency of the background image determination of the video cover, the target screenshot can be processed by a preset image processing algorithm (for example, filtering, denoising, etc., and then adding a target icon after image enhancement) to obtain the cover picture.

[0119] It is understandable that in order to improve the display effect of the cover image, it is necessary to perform clarity detection, filter out images whose icon colors are integrated with the background image, and finally obtain a usable high-quality cover image. As an example, obtain the icon pixel coordinates corresponding to the icon in the cover image to be displayed, obtain the background pixel coordinates corresponding to the icon pixel coordinates, perform clarity detection based on the color information of the icon pixel coordinates and the background pixel coordinates, and delete the cover images whose detection results do not meet the preset clarity conditions.

[0120] Specifically, Figure 5 As shown, after image enhancement (filtering, denoising) is performed on the target screenshot, pixel brightness enhancement is performed, and then icons are pasted on the pictures based on the icon pasting algorithm according to the icon configuration rules. Finally, the icon clarity is detected for the pictures after the icons are pasted, and the pictures that do not meet the clarity requirements are filtered out according to the detection results to obtain the final cover picture.

[0121] In order to ensure the operability of determining the background image of the video cover, the first platform can be operated and controlled through the second platform. The method for determining the background image of the video cover provided by the embodiment of the present disclosure also includes: receiving an operation control request sent by the second platform, and adjusting the frequency of obtaining the video to be processed based on the operation control request.

[0122] Specifically, the second platform is the platform for scheduling the entire process, e.g. Figure 6 The system diagram shown includes a first platform and a second platform. The second platform may include functions such as a workflow engine, queue scheduling, monitoring alarm, login authentication, and gateway current limiting. It also includes an operation console. Users can view task status, poster images, production statistics, video coverage and other functions on the second platform, so that they can send operation control requests to the first platform according to actual operation control needs. The first platform determines the base map of the video cover based on the operation control request, such as adjusting the frequency of obtaining the video to be processed, and other operation controls.

[0123] For example, the second platform detects that the number of videos to be processed simultaneously exceeds a certain number, and notifies the second platform to select one or more videos to be processed to suspend processing, etc., to ensure the generation effect.

[0124] Specifically, the first platform selects, crops and processes a video to be processed in sequence, so that a video to be processed can be turned into cover pictures. Among them, picture selection refers to splitting a video into pictures and choosing a suitable picture from multiple pictures; cropping refers to cutting out a cover picture base map that meets the specifications from the selected pictures; processing refers to pasting icons, brightening, image filtering, denoising and other operations on the cut base map.

[0125] Specifically, Figure 6 As shown, when the first platform monitors that a new video is online, it immediately performs video object recognition, video line recognition, video face analysis, video transition point analysis and video scoring (image quality evaluation, image visual evaluation and inter-image motion vector calculation, etc.) to obtain the visual modality analysis results and image quality evaluation results of each video frame, and then selects the image based on the image selection algorithm and the image selection configuration file (visual modality conditions and image quality conditions), and selects the image through Figure 6 The basic image capabilities shown in the figure perform image OCR recognition, image logo detection, image object recognition and image face recognition on the selected image results to obtain the image processing results of the selected image, and then take a screenshot based on the screenshot algorithm and the screenshot configuration file (cropping strategy), and pass Figure 6 The basic capabilities of the picture shown are to perform image enhancement (image filtering, denoising, etc.), brightness enhancement, and icon pasting algorithm. After pasting based on the mapping configuration rules, icon clarity detection is performed to determine the final result (cover image), and the final result is stored so that the result can be obtained immediately when needed, speeding up the operation of the entire process.

[0126] It should be noted that, since each video recognition and analysis service in the first platform runs independently, it is necessary to ensure that the video obtained by each service is the same, so as to avoid frame misalignment in the analysis results.

[0127] in addition, Figure 6 The basic capabilities shown in the picture refer to independently deployed and running services. Each service runs only one task, and each task is independent of each other. This can fully utilize CPU (Central Processing Unit) resources and avoid the CPU context switching problem caused by multi-tasking in parallel. In addition, each service can be horizontally expanded. When there are too many tasks, the system throughput can be improved by adding machines.

[0128] Therefore, through the ability of artificial intelligence to analyze video images frame by frame, select appropriate images, cut out appropriate base images, and finally process the base images to form excellent cover images. The cost of using artificial intelligence to generate cover images is much lower than the cost of manual production, and can achieve the characteristics of high output and wide coverage.

[0129] Corresponding to the above-mentioned method for determining the background image of a video cover, the embodiment of the present disclosure further provides a device for determining the background image of a video cover. Figure 7 This is a schematic diagram of the structure of a device for determining a base image of a video cover provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware and is applied to a first platform. The device includes:

[0130] The video acquisition module 702 is used to acquire the video to be processed.

[0131] The first video processing module 704 is used to obtain visual modality analysis results corresponding to each video frame of the video to be processed.

[0132] The second video processing module 706 is used to obtain the picture quality evaluation result corresponding to each video frame of the video to be processed.

[0133] The image selection module 708 is used to obtain, from the video frames, a target video frame whose visual modality analysis result satisfies a preconfigured visual modality condition and whose image quality evaluation result satisfies a preconfigured image quality condition, and determine the target video frame as a base image for generating a cover of the video to be processed.

[0134] The above-mentioned device provided by the embodiment of the present disclosure first obtains the visual modal analysis results and image quality evaluation results of each video frame corresponding to the video to be processed, and obtains the target video frame whose visual modal analysis results meet the preconfigured visual modal conditions and whose image quality evaluation results meet the preconfigured image quality conditions from each video frame, and determines it as the base map for generating the cover of the video to be processed. The above-mentioned method does not require manual selection of the base map, which can greatly reduce the cost of determining the base map of the video cover, improve the efficiency of determining the base map of the video cover, thereby reducing the subsequent cost of generating the video cover based on the base map.

[0135] In some embodiments, the first video processing module 704 is specifically used to: perform object recognition on the video to be processed based on a pre-trained video object recognition service to obtain first object information of each video frame; and / or, perform face analysis on the video to be processed based on a pre-trained video face analysis service to obtain first face information of each video frame; and / or, perform dialogue recognition on the video to be processed based on a pre-trained video dialogue recognition service to obtain dialogue information of each video frame; and / or, perform video transition point analysis on the video to be processed based on a pre-trained video transition point analysis service to obtain transition information of each video frame.

[0136] In some embodiments, the second video processing module 706 is specifically used to: perform image quality evaluation on each video frame based on a pre-trained first evaluation model to obtain a quality evaluation result of each video frame; and / or perform image visual evaluation on each video frame based on a pre-trained second evaluation model to obtain a visual evaluation result of each video frame; and / or perform inter-image motion vector calculation on each video frame based on a pre-trained third evaluation model to obtain a relative motion vector of each video frame.

[0137] In some embodiments, the image selection module 708 is specifically used to: obtain candidate video frames from the video frames whose first object information, and / or the first face information, and / or the line information, and / or the transition information satisfy the object condition, and / or the face condition, and / or the line condition, and / or the transition condition; obtain the target video frame from the candidate video frames whose quality evaluation result, and / or the visual evaluation result, and / or the relative motion vector satisfy the quality evaluation condition, and / or the visual evaluation condition, and / or the relative motion condition, and determine the target video frame as the base image for generating the cover of the video to be processed.

[0138] In some embodiments, the device also includes: a screenshot module, which is used to: perform image icon detection on the base map based on a pre-trained image icon detection service to obtain icon information of the base map; obtain a first screenshot size and whether the target screenshot contains an icon based on a pre-configured cropping strategy; and crop the base map according to the first screenshot size and whether the target screenshot contains an icon based on the icon information of the base map to obtain a first target screenshot.

[0139] In some embodiments, the screenshot module is further used to: perform image object recognition on the base map based on a pre-trained image object recognition service to obtain second object information of the base map; obtain a second screenshot size and a target object identification and a target object quantity of the target screenshot based on a pre-configured cropping strategy; and crop the base map according to the second screenshot size and the target object identification and the target object quantity of the target screenshot based on the second object information of the base map to obtain a second target screenshot.

[0140] In some embodiments, the screenshot module is further used to: perform face recognition on the base image based on a pre-trained image face recognition service to obtain second face information of the base image; obtain a third screenshot size and target face information of the target screenshot based on a pre-configured cropping strategy; and crop the base image according to the third screenshot size and the target face information of the target screenshot based on the second face information of the base image to obtain a third target screenshot.

[0141] In some embodiments, the screenshot module is further used to: perform optical character recognition on the base map based on a pre-trained image optical character recognition service to obtain optical character information of the base map; obtain a fourth screenshot size and whether the target screenshot contains optical characters based on a pre-configured cropping strategy; and crop the base map according to the fourth screenshot size and whether the target screenshot contains optical characters based on the optical character information of the base map to obtain a fourth target screenshot.

[0142] In some embodiments, the device further includes: a storage module, used to store the cover of the video to be processed generated based on the base map to a preset storage server, wherein the terminal obtains and displays the cover of the video to be processed from the storage server through the video server.

[0143] In some embodiments, the device further includes: a receiving module for receiving an operation control request sent by the second platform; and a control module for adjusting a frequency of acquiring the video to be processed based on the operation control request.

[0144] The device for determining the background image of a video cover provided in the embodiments of the present disclosure can execute the method for determining the background image of a video cover provided in any embodiment of the present disclosure, and has the functional modules and beneficial effects corresponding to the execution method.

[0145] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device embodiment can refer to the corresponding process in the method embodiment, and will not be repeated here.

[0146] An embodiment of the present disclosure provides an electronic device, which includes: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device to implement the steps of any method in the present disclosure.

[0147] Reference below Figure 8 , which shows a schematic diagram of the structure of an electronic device 800 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0148] like Figure 8As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0149] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0150] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0151] In addition to the above-mentioned methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, cause the processor to perform the image processing method provided by the embodiments of the present disclosure. The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present disclosure, the programming languages ​​including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0152] In addition, the embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the processor executes the method for determining the base image of the video cover provided by the embodiment of the present disclosure.

[0153] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0154] The present disclosure embodiment also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method for determining the base image of the video cover in the present disclosure embodiment.

[0155] It is understandable that before using the technical solutions disclosed in the various embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0156] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0157] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0158] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0159] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0160] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining a base image of a video cover, characterized in that: Applied to the first platform, the method comprises: Get the video to be processed; Obtaining visual modality analysis results and picture quality evaluation results for each video frame corresponding to the video to be processed; A target video frame whose visual modality analysis result satisfies a preconfigured visual modality condition and whose picture quality evaluation result satisfies a preconfigured picture quality condition is obtained from each video frame, and the target video frame is determined as a base image for generating a cover of the video to be processed.

2. The method according to claim 1, characterized in that The obtaining of visual modality analysis results of each video frame corresponding to the video to be processed includes: Performing object recognition on the video to be processed based on a pre-trained video object recognition service to obtain first object information of each video frame; and / or, Performing face analysis on the video to be processed based on a pre-trained video face analysis service to obtain first face information of each video frame; and / or, Performing line recognition on the video to be processed based on a pre-trained video line recognition service to obtain line information of each video frame; and / or, Based on the pre-trained video transition point analysis service, video transition point analysis is performed on the video to be processed to obtain transition information of each video frame.

3. The method according to claim 2, characterized in that The obtaining of the picture quality evaluation result of each video frame corresponding to the video to be processed includes: Performing picture quality evaluation on each video frame based on a pre-trained first evaluation model to obtain a quality evaluation result of each video frame; and / or, Performing a visual evaluation of each video frame based on a pre-trained second evaluation model to obtain a visual evaluation result of each video frame; and / or, Based on the pre-trained third evaluation model, inter-image motion vectors of the video frames are calculated to obtain relative motion vectors of the video frames.

4. The method according to claim 3, characterized in that The step of obtaining, from the video frames, a target video frame whose visual modality analysis result satisfies a preconfigured visual modality condition and whose picture quality evaluation result satisfies a preconfigured picture quality condition, and determining the target video frame as a base image for generating a cover of the video to be processed comprises: Acquire, from the video frames, candidate video frames whose first object information, and / or the first face information, and / or the dialogue information, and / or the transition information satisfy object conditions, and / or face conditions, and / or dialogue conditions, and / or transition conditions; Obtain the target video frame whose quality evaluation result, and / or the visual evaluation result, and / or the relative motion vector satisfies the quality evaluation condition, and / or the visual evaluation condition, and / or the relative motion condition from the candidate video frame, and determine the target video frame as the base image for generating the cover of the video to be processed.

5. The method according to claim 1, characterized in that After determining the target video frame as a base image for generating a cover of the video to be processed, the method further includes: Performing image icon detection on the base image based on a pre-trained image icon detection service to obtain icon information of the base image; Obtain the first screenshot size and whether the target screenshot contains an icon based on a preconfigured cropping strategy; Based on the icon information of the base image, the base image is cropped according to the first screenshot size and whether the target screenshot contains an icon to obtain a first target screenshot.

6. The method according to claim 1, characterized in that After determining the target video frame as a base image for generating a cover of the video to be processed, the method further includes: Performing image object recognition on the base map based on a pre-trained image object recognition service to obtain second object information of the base map; Obtaining the second screenshot size and the target object identifier and the number of target objects of the target screenshot based on a preconfigured cropping strategy; Based on the second object information of the base image, the base image is cropped according to the second screenshot size and the target object identifier and the number of target objects of the target screenshot to obtain a second target screenshot.

7. The method according to claim 1, characterized in that After determining the target video frame as a base image for generating a cover of the video to be processed, the method further includes: Performing face recognition on the base image based on a pre-trained image face recognition service to obtain second face information of the base image; Obtaining the third screenshot size and target face information of the target screenshot based on a preconfigured cropping strategy; Based on the second facial information of the base image, the base image is cropped according to the third screenshot size and the target facial information of the target screenshot to obtain a third target screenshot.

8. The method according to claim 1, characterized in that: After determining the target video frame as a base image for generating a cover of the video to be processed, the method further includes: Performing optical character recognition on the base map based on a pre-trained image optical character recognition service to obtain optical character information of the base map; Obtaining a fourth screenshot size and whether the target screenshot contains optical characters based on a preconfigured cropping strategy; Based on the optical character information of the base image, the base image is cropped according to the fourth screenshot size and whether the target screenshot contains optical characters to obtain a fourth target screenshot.

9. The method according to claim 1, characterized in that: The method further comprises: The cover of the video to be processed generated based on the base map is stored in a preset storage server, wherein the terminal obtains the cover of the video to be processed from the storage server through the video server and displays it.

10. The method according to claim 1, characterized in that The method further comprises: Receiving an operation control request sent by the second platform; The frequency of acquiring the to-be-processed video is adjusted based on the operation control request.

11. A device for determining a base image of a video cover, characterized in that: Applied to the first platform, the device comprises: A video acquisition module is used to acquire the video to be processed; A first video processing module, used to obtain a visual modality analysis result of each video frame corresponding to the video to be processed; A second video processing module is used to obtain a picture quality evaluation result corresponding to each video frame of the video to be processed; A picture selection module is used to obtain a target video frame from the video frames whose visual modality analysis result meets the preconfigured visual modality condition and whose picture quality evaluation result meets the preconfigured picture quality condition, and determine the target video frame as the base image for generating the cover of the video to be processed.

12. An electronic device, characterized in that: The electronic device comprises: a storage device having a computer program stored thereon; A processing device, used to execute the computer program in the storage device to implement the steps of the method for determining the base image of the video cover described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the method for determining the base image of the video cover as described in any one of claims 1-10.