Method, medium, device and computing device for determining cover image

Abstract

The embodiments of the present disclosure provide a method, medium, device and computing device for determining a cover image, the method comprising: extracting audio information contained in a target video, determining the correlation between an audio segment of a corresponding time contained in the audio information and video attribute information of the target video, determining the wonderfulness of the audio segment according to the sound attribute information of the audio information, and determining the cover image of the target video according to the wonderfulness and the correlation. The present disclosure can obtain a more attractive cover image, thereby improving the click-through rate of the video; and the processing efficiency is higher when determining the cover image.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of image processing technology. More specifically, embodiments of the present disclosure relate to a method, medium, apparatus, and computing device for determining a cover image. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] With the rapid growth of mobile devices and online media, more and more people like to publish or watch videos online. Videos are usually presented in the form of cover images and text titles on video or streaming websites or applications. The cover image is an important factor in attracting users to click on the video. Therefore, how to create attractive cover images to increase the click-through rate of videos is particularly critical for video publishers and streaming websites.

[0004] In the related art, when determining the cover image of a video, each frame image contained in the video is usually analyzed, low-quality images and images with low aesthetic evaluation are filtered out, and then the most representative frame image is selected as the cover image of the video through cluster analysis.

[0005] However, the cover image of the video obtained in the above manner may have a problem of low video click rate. Summary of the invention

[0006] The present disclosure provides a method, medium, apparatus and computing device for determining a cover image to solve the problem that the cover image of a video obtained by related technologies may have a low click-through rate of the video.

[0007] In a first aspect of the embodiments of the present disclosure, a method for determining a cover image is provided, comprising:

[0008] Extracting audio information contained in the target video;

[0009] Determine the relevance between the audio segment at the corresponding time contained in the audio information and the video attribute information of the target video, where the video attribute information includes at least one of a video title, a video tag, or a video description;

[0010] Determining the wonderfulness of the audio segment according to the sound attribute information of the audio information, where the sound attribute information includes at least one of sound frequency and sound intensity;

[0011] Determine the cover image of the target video based on its excitement and relevance.

[0012] In one possible implementation, determining the correlation between an audio segment of a corresponding time contained in audio information and video attribute information of a target video includes: analyzing and processing the audio information through a preset speech recognition model to obtain text information arranged in time corresponding to the audio information; and analyzing and processing the degree of correlation between the text information and the video attribute information through a preset correlation model to determine the correlation.

[0013] In a possible implementation, determining the wonderfulness of the audio segment according to the sound attribute information of the audio information includes: determining the wonderfulness according to an average value of the sound frequency and / or an average value of the sound intensity corresponding to the audio segment.

[0014] In a possible implementation, a cover image of a target video is determined based on the excitement and relevance, including: determining an intersection time period of an audio clip with the highest excitement and an audio clip with the greatest relevance based on the excitement and relevance; and determining a cover image based on an image of the target video corresponding to the intersection time period.

[0015] In a possible implementation, a cover image is determined based on an image of a target video corresponding to an intersection time period, including: analyzing and processing the image of the target video corresponding to the intersection time period through at least one of a preset aesthetics model and a preset clarity model to determine the cover image.

[0016] In a second aspect, an embodiment of the present disclosure provides a device for determining a cover image, including:

[0017] An extraction module, used for extracting audio information contained in the target video;

[0018] A first determination module is used to determine the relevance between the audio segment at the corresponding time contained in the audio information and the video attribute information of the target video, where the video attribute information includes at least one of a video title, a video tag or a video description;

[0019] A second determination module is used to determine the wonderfulness of the audio segment according to the sound attribute information of the audio information, where the sound attribute information includes at least one of sound frequency and sound intensity;

[0020] The processing module is used to determine the cover image of the target video according to the excitement and relevance.

[0021] In a possible implementation, the first determination module is specifically used to: analyze and process the audio information through a preset speech recognition model to obtain text information arranged in time corresponding to the audio information; analyze and process the degree of correlation between the text information and the video attribute information through a preset correlation model to determine the correlation.

[0022] In a possible implementation manner, the second determination module is specifically configured to determine the wonderfulness according to an average value of sound frequency and / or an average value of sound intensity corresponding to the audio segment.

[0023] In a possible implementation, the processing module is specifically used to: determine the intersection time period of the audio segment with the highest excitement and the audio segment with the greatest correlation based on the excitement and correlation; determine the cover image based on the image of the target video corresponding to the intersection time period.

[0024] In a possible implementation, when the processing module is used to determine the cover image based on the image of the target video corresponding to the intersection time period, it is specifically used to: analyze and process the image of the target video corresponding to the intersection time period through at least one of a preset aesthetics model and a preset clarity model to determine the cover image.

[0025] In a third aspect, an embodiment of the present disclosure provides a computing device, including: a processor, and a memory communicatively connected to the processor;

[0026] Memory stores computer-executable instructions;

[0027] The processor executes the computer-executable instructions stored in the memory to implement the method for determining the cover image as described in the first aspect of the present disclosure.

[0028] In a fourth aspect, an embodiment of the present disclosure provides a storage medium, in which computer program instructions are stored. When the computer program instructions are executed, the method for determining a cover image as described in the first aspect of the present disclosure is implemented.

[0029] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the method for determining a cover image as described in the first aspect of the present disclosure.

[0030] The method, medium, device, and computing device for determining a cover image provided by the embodiments of the present disclosure extract the audio information contained in the target video, determine the correlation between the audio clip at the corresponding time contained in the audio information and the video attribute information of the target video, determine the wonderfulness of the audio clip based on the sound attribute information of the audio information, and determine the cover image of the target video based on the wonderfulness and the correlation. Since the embodiments of the present disclosure take into account the audio information contained in the target video, and determine the cover image of the target video based on the correlation between the audio clip at the corresponding time contained in the audio information and the video attribute information of the target video and the wonderfulness of the audio clip, a more attractive cover image can be obtained, thereby improving the click-through rate of the video; and when determining the cover image, the processing efficiency is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0032] Figure 1 A schematic diagram of an application scenario provided for the disclosed embodiment;

[0033] Figure 2 A flowchart of a method for determining a cover image provided by an embodiment of the present disclosure;

[0034] Figure 3 A schematic diagram of a waveform diagram of a sound frequency provided by an embodiment of the present disclosure;

[0035] Figure 4 A schematic diagram of a waveform diagram of sound intensity provided by an embodiment of the present disclosure;

[0036] Figure 5 A flowchart of a method for determining a cover image provided by another embodiment of the present disclosure;

[0037] Figure 6 A schematic diagram of an intersection time period provided for an embodiment of the present disclosure;

[0038] Figure 7 A schematic diagram of the structure of a device for determining a cover image provided by an embodiment of the present disclosure;

[0039] Figure 8 A schematic diagram of a storage medium provided in an embodiment of the present disclosure;

[0040] Fig. 9 A schematic diagram of the structure of a computing device provided in one embodiment of the present disclosure.

[0041] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0042] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0043] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, device, apparatus, method or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0044] According to an embodiment of the present disclosure, a method, medium, apparatus and computing device for determining a cover image are proposed.

[0045] In this article, it is necessary to understand that the terms involved are: Convolutional Neural Networks (CNN) is a type of feedforward neural network with deep structure that includes convolution calculations, and is one of the representative algorithms of deep learning; Natural Language Processing (NLP) is a discipline that studies language problems in human-computer interaction. According to the difficulty of technical implementation, natural language processing systems can be divided into three types: simple matching, fuzzy matching, and paragraph understanding. In addition, any number of elements in the drawings is for example rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0046] The principle and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure. SUMMARY OF THE INVENTION

[0048] The inventors have found that creating an attractive cover image can increase the click-through rate of a video. However, currently, the cover images of videos are mostly created manually by video creators, for example, manually selecting an image from a video and editing it accordingly as the cover image of the video. The quality of the cover image of the video obtained by the above manual method is not high and the efficiency is low.

[0049] In the related art, when determining the cover image of a video, each frame of the video is usually analyzed, low-quality and low-aesthetic images are filtered out, and then the most representative frame of the video is selected as the cover image of the video through cluster analysis. However, the above method does not use the audio information contained in the video, nor does it use text information such as the text title corresponding to the video. It only analyzes each frame of the video. Therefore, the cover image of the video obtained by the above method is not sufficiently correlated with the text title corresponding to the video, which may lead to the problem of low click-through rate of the video. In addition, when obtaining the cover image of the video by the above method, a multi-module serial filtering method is specifically adopted, which leads to a large amount of calculation, a large number of parameters, and insufficient scalability.

[0050] In addition, in the related art, the cover image of the video can also be determined in the following way: multiple frames of images, such as 19 frames of images, are uniformly extracted from the video, and then the images are scored using a deep learning algorithm, wherein the deep learning algorithm takes into account the influence of facial factors. However, the above method does not utilize the audio information contained in the video, nor does it utilize text information such as the text title corresponding to the video. It only analyzes the 19 frames of images uniformly extracted from the video. Therefore, the cover image of the video obtained by the above method has insufficient correlation with the text title corresponding to the video. Moreover, the above method belongs to an image scoring algorithm, which requires extracting image frames from the video first and then processing the image. Therefore, there are problems of omissions and cumbersome steps, and the video is processed frame by frame, and the calculation efficiency is low.

[0051] In summary, the relevant technologies have the following main problems: (1) The traditional method for quality and aesthetic attribute analysis takes insufficient factors into consideration and has low accuracy; the multi-module serial processing method has many parameters, weak generalization ability, and difficult deployment. (2) In the deep learning method, the feature representation is single and only individual factors are considered; there are many steps, and it is necessary to extract frames first and then process the image.

[0052] Based on the above problems, the present disclosure provides a method, medium, device and computing device for determining a cover image. By analyzing the audio information contained in the video, a more attractive cover image can be obtained, thereby improving the click rate of the video; and when determining the cover image, the processing efficiency is higher.

[0053] Application Scenario Overview

[0054] First reference Figure 1 The application scenarios of the solutions provided in the present disclosure are illustrated by examples. Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure, such as Figure 1 As shown, in this application scenario, the client 101 sends the video whose cover image is to be determined to the server 102, and the server 102 determines the cover image of the video and sends the cover image to the client 101, so that the client 101 displays the cover image. The specific implementation process of the server 102 determining the cover image of the video can refer to the solutions of the following embodiments.

[0055] It should be noted that Figure 1 This is only a schematic diagram of an application scenario provided by the embodiment of the present disclosure. Figure 1 does not limit the equipment included in Figure 1 The positional relationship between the devices in the Figure 1In the application scenario shown, a data storage device may also be included. The data storage device may be an external memory relative to the client 101 or the server 102 , or an internal memory integrated in the client 101 or the server 102 .

[0056] Exemplary Methods

[0057] Combine the following Figure 1 For application scenarios, refer to Figure 2 To describe a method for determining a cover image according to an exemplary embodiment of the present disclosure. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in this regard. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0058] First, a method for determining a cover image is introduced through a specific embodiment.

[0059] Figure 2 A flowchart of a method for determining a cover image provided by an embodiment of the present disclosure. The method of the embodiment of the present disclosure can be applied to a computing device, which can be a server or a server cluster. Figure 2 As shown, the method of the embodiment of the present disclosure includes:

[0060] S201: extracting audio information contained in a target video.

[0061] In the disclosed embodiment, exemplarily, the target video contains images and sounds, and the audio information contained in the target video can be extracted by audio and video processing software. The audio and video processing software is, for example, MKVToolNix, Jianying and other software. Specifically, the target video can be opened by the audio and video processing software, and then saved as Moving Picture Experts Group Audio Layer III (MP3). In addition, the audio information contained in the target video can also be extracted by a pre-written audio and video processing program, and the pre-written audio and video processing program is, for example, a program package such as a Python module (moviepy) for video editing.

[0062] S202: Determine the correlation between the audio segments at the corresponding time contained in the audio information and the video attribute information of the target video.

[0063] The video attribute information includes at least one of a video title, a video tag or a video description.

[0064] In this step, the audio segment at the corresponding time contained in the audio information is, for example, a sentence or sentence segment at the corresponding time contained in the audio information. After the audio information contained in the target video is obtained, the correlation between the audio segment at the corresponding time contained in the audio information and the video attribute information of the target video can be determined based on the audio information. For how to determine the correlation between the audio segment at the corresponding time contained in the audio information and the video attribute information of the target video, please refer to the subsequent embodiments, which will not be repeated here.

[0065] S203: Determine the wonderfulness of the audio segment according to the sound attribute information of the audio information.

[0066] The sound attribute information includes at least one of sound frequency and sound intensity.

[0067] Exemplarily, the sound attribute information of the audio information is obtained by analyzing the audio information using a Python library (PyAudio) for speech processing. Figure 3 A schematic diagram of a waveform diagram of a sound frequency provided by an embodiment of the present disclosure, such as Figure 3 As shown, a waveform diagram of the sound frequency corresponding to different moments is shown. Figure 4 A schematic diagram of a waveform diagram of sound intensity provided by an embodiment of the present disclosure is shown in FIG. Figure 4 As shown, a waveform diagram of sound intensity corresponding to different moments is shown. In this step, after the audio information contained in the target video is obtained, the wonderfulness of the audio clip can be determined according to the sound attribute information of the audio information. For how to determine the wonderfulness of the audio clip according to the sound attribute information of the audio information, reference can be made to the subsequent embodiments, which will not be repeated here.

[0068] S204: Determine a cover image of the target video according to the excitement and relevance.

[0069] In this step, after determining the correlation between the audio clip at the corresponding time contained in the audio information and the video attribute information of the target video and the wonderfulness of the audio clip, the cover image of the target video can be determined according to the wonderfulness and the correlation. For how to determine the cover image of the target video according to the wonderfulness and the correlation, please refer to the subsequent embodiments, which will not be repeated here.

[0070] The method for determining the cover image provided by the embodiment of the present disclosure extracts the audio information contained in the target video, determines the correlation between the audio clip at the corresponding time contained in the audio information and the video attribute information of the target video, determines the wonderfulness of the audio clip based on the sound attribute information of the audio information, and determines the cover image of the target video based on the wonderfulness and the correlation. Since the embodiment of the present disclosure takes into account the audio information contained in the target video, and determines the cover image of the target video based on the correlation between the audio clip at the corresponding time contained in the audio information and the video attribute information of the target video and the wonderfulness of the audio clip, a more attractive cover image can be obtained, thereby improving the click-through rate of the video; and when determining the cover image, the processing efficiency is higher.

[0071] Figure 5 This is a flowchart of a method for determining a cover image provided by another embodiment of the present disclosure. Based on the above embodiment, the present disclosure further describes how to determine a cover image. Figure 5 As shown, the method of the embodiment of the present disclosure may include:

[0072] S501: Extract audio information contained in the target video.

[0073] The detailed description of this step can be found in Figure 2 The relevant description of S201 in the illustrated embodiment will not be repeated here.

[0074] In the present disclosure, Figure 2 The step S202 may further include the following two steps S502 and S503:

[0075] S502: Analyze and process the audio information using a preset speech recognition model to obtain text information arranged in time corresponding to the audio information.

[0076] Exemplarily, the preset speech recognition model is, for example, a speech recognition model obtained by artificial intelligence (AI) speech technology. The speech recognition model can also be obtained by writing a speech recognition program, such as by using the Python speech recognition (SpeechRecognition) software package to obtain the speech recognition model. In this step, after obtaining the audio information contained in the target video, the audio information can be analyzed and processed by the preset speech recognition model to obtain the time-arranged text information corresponding to the audio information, that is, the time-arranged audio clip corresponding to the audio information is obtained. The time-arranged text information corresponding to the audio information is, for example, the first sentence corresponding to the 2nd to 4th seconds, the second sentence corresponding to the 5th to 7th seconds, and so on, and the time-arranged text information can be obtained. It can be understood that there may be some times in the audio information corresponding to the sound, and some times there may be no corresponding sound. Accordingly, the time-arranged text information corresponding to the audio information can be obtained according to the time with sound.

[0077] S503: Analyze and process the correlation between the text information and the video attribute information using a preset correlation model to determine the correlation.

[0078] Exemplarily, the preset relevance model is, for example, an NLP model, and the NLP model is, for example, a Bidirectional Encoder Representations from Transformers (BERT) model. In this step, after obtaining the time-arranged text information corresponding to the audio information, the correlation between the text information and the video attribute information can be analyzed and processed by the preset relevance model to determine the correlation. Exemplarily, the text information is, for example, the first sentence (i.e., the audio clip) corresponding to the 2nd to 4th seconds, and the video attribute information includes, for example, a video title, a video tag, and a video description. The correlation between the first sentence and the video title of the target video, the correlation with the video tag of the target video, and the correlation with the video description of the target video are analyzed and processed by the preset relevance model to determine the correlation.

[0079] In the present disclosure, Figure 2 The step S203 may further include the following step S504:

[0080] S504: Determine the degree of excellence according to an average value of the sound frequency and / or an average value of the sound intensity corresponding to the audio clip.

[0081] In this step, for the audio clips of the corresponding time contained in the audio information, the audio clips can be analyzed by using the PyAudio library, for example, to obtain the sound frequency and sound intensity corresponding to each word contained in the audio clips; according to the sound frequency corresponding to each word, the average value of the sound frequency corresponding to the audio clip can be obtained; according to the sound intensity corresponding to each word, the average value of the sound intensity rate corresponding to the audio clip can be obtained. Therefore, the wonderfulness of the audio clip can be determined according to the average value of the sound frequency corresponding to the audio clip, or the wonderfulness of the audio clip can be determined according to the average value of the sound intensity corresponding to the audio clip, or the wonderfulness of the audio clip can be determined according to the average value of the sound frequency and the average value of the sound intensity corresponding to the audio clip.

[0082] In the present disclosure, Figure 2 The step S204 may further include the following two steps S505 and S506:

[0083] S505: Determine, according to the excitement and relevance, the intersection time period of the audio segment with the highest excitement and the audio segment with the highest relevance.

[0084] In this step, after obtaining the correlation between the audio clips at the corresponding time contained in the audio information and the video attribute information of the target video and the excitement of the audio clips, the intersection time period of the audio clip with the highest excitement and the audio clip with the greatest correlation can be determined based on the excitement and correlation.

[0085] For example, Figure 6 A schematic diagram of an intersection time period provided by an embodiment of the present disclosure, such as Figure 6 As shown, the most exciting audio clip is from 2 minutes 15 seconds to 3 minutes 20 seconds, and the most relevant audio clip is from 2 minutes 40 seconds to 4 minutes 19 seconds. Therefore, it can be determined that the intersection time period of the most exciting audio clip and the most relevant audio clip is from 2 minutes 40 seconds to 3 minutes 20 seconds.

[0086] It can be understood that by adopting speech recognition technology and NLP technology with high maturity and accuracy, the intersection time period of the most exciting audio clip and the most relevant audio clip can be screened out, and step S506 is executed based on the intersection time period, which can greatly reduce the scope of video processing, avoid processing irrelevant content, and improve the efficiency of video processing.

[0087] S506: Determine a cover image according to the image of the target video corresponding to the intersection time period.

[0088] In this step, after determining the intersection time period of the audio segment with the highest degree of excitement and the audio segment with the greatest degree of relevance, the cover image can be determined based on the image of the target video corresponding to the intersection time period.

[0089] Further, optionally, determining the cover image based on the image of the target video corresponding to the intersection time period may include: analyzing and processing the image of the target video corresponding to the intersection time period through at least one of a preset aesthetics model and a preset clarity model to determine the cover image.

[0090] Exemplarily, the preset aesthetics model is, for example, a CNN model. By inputting the image of the target video corresponding to the intersection time period into the preset aesthetics model, a corresponding aesthetics score can be obtained. The preset clarity model is, for example, a CNN model. By inputting the image of the target video corresponding to the intersection time period into the preset clarity model, a corresponding clarity score can be obtained. In one possible implementation, the image of the target video corresponding to the intersection time period can be analyzed and processed by the preset aesthetics model, and the cover image can be determined according to the aesthetics score corresponding to the obtained image. In another possible implementation, the image of the target video corresponding to the intersection time period can be analyzed and processed by the preset clarity model, and the cover image can be determined according to the clarity score corresponding to the obtained image. In yet another possible implementation, a deep CNN model, such as a Neural Image Assessment (NIMA) model, has the functions of both a preset aesthetics model and a preset clarity model, that is, it can simultaneously consider the aesthetics and clarity of the image into one to determine the cover image.

[0091] The method for determining the cover image provided by the embodiment of the present disclosure extracts the audio information contained in the target video, analyzes and processes the audio information through a preset speech recognition model, obtains the text information arranged in time corresponding to the audio information, analyzes and processes the correlation between the text information and the video attribute information through a preset correlation model, and determines the correlation; determines the wonderfulness according to the average value of the sound frequency and / or the average value of the sound intensity corresponding to the audio clip; determines the intersection time period of the audio clip with the highest wonderfulness and the audio clip with the highest correlation according to the wonderfulness and correlation, and determines the cover image according to the image of the target video corresponding to the intersection time period. Since the embodiment of the present disclosure takes into account the audio information contained in the target video, based on the correlation between the audio clip at the corresponding time contained in the audio information and the video attribute information of the target video and the wonderfulness of the audio clip, determines the intersection time period of the audio clip with the highest wonderfulness and the audio clip with the highest correlation, and then determines the cover image of the target video, therefore, it is possible to obtain a cover image that is more relevant to the video theme and more attractive, thereby improving the click-through rate of the video; and when determining the cover image, the processing efficiency is higher.

[0092] Exemplary Devices

[0093] After introducing the medium of the exemplary embodiment of the present disclosure, next, reference is made to Figure 7 The cover image determination device of the exemplary embodiment of the present disclosure is described. The device of the exemplary embodiment of the present disclosure can implement each process in the above-mentioned cover image determination method embodiment and achieve the same function and effect.

[0094] Figure 7 A schematic diagram of a cover image determination device provided by an embodiment of the present disclosure is shown in FIG. Figure 7 As shown, the cover image determination device 700 of the embodiment of the present disclosure includes: an extraction module 701, a first determination module 702, a second determination module 703 and a processing module 704. Among them:

[0095] The extraction module 701 is used to extract the audio information contained in the target video.

[0096] The first determination module 702 is used to determine the correlation between the audio segment of the corresponding time contained in the audio information and the video attribute information of the target video, where the video attribute information includes at least one of a video title, a video tag or a video description.

[0097] The second determination module 703 is used to determine the wonderfulness of the audio segment according to the sound attribute information of the audio information, where the sound attribute information includes at least one of sound frequency and sound intensity.

[0098] The processing module 704 is used to determine the cover image of the target video according to the excitement and relevance.

[0099] In a possible implementation, the first determination module 702 may be specifically used to: analyze and process the audio information through a preset speech recognition model to obtain text information arranged in time corresponding to the audio information; analyze and process the degree of correlation between the text information and the video attribute information through a preset correlation model to determine the correlation.

[0100] In a possible implementation, the second determination module 703 may be specifically configured to determine the wonderfulness according to an average value of sound frequency and / or an average value of sound intensity corresponding to the audio segment.

[0101] In a possible implementation, the processing module 704 can be specifically used to: determine the intersection time period of the audio segment with the highest excitement and the audio segment with the greatest correlation based on the excitement and correlation; determine the cover image based on the image of the target video corresponding to the intersection time period.

[0102] In a possible implementation, when the processing module 704 is used to determine the cover image based on the image of the target video corresponding to the intersection time period, it can be specifically used to: analyze and process the image of the target video corresponding to the intersection time period through at least one of a preset aesthetics model and a preset clarity model to determine the cover image.

[0103] The device of the embodiment of the present disclosure can be used to execute the solution of the method for determining the cover image in any of the above method embodiments. Its implementation principle and technical effect are similar and will not be repeated here.

[0104] Exemplary Media

[0105] After introducing the method of the exemplary embodiment of the present disclosure, next, refer to Figure 8 A storage medium according to an exemplary embodiment of the present disclosure is described.

[0106] Figure 8 A schematic diagram of a storage medium provided by an embodiment of the present disclosure. Figure 8 As shown, the storage medium 800 stores a program product for implementing the above method according to an embodiment of the present disclosure, which can be a portable compact disk read-only memory (CD-ROM) and includes program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto.

[0107] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0108] The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable signal medium may also be any readable medium other than a readable storage medium.

[0109] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user computing device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN).

[0110] Exemplary Computing Devices

[0111] After introducing the method, medium and apparatus of the exemplary embodiments of the present disclosure, next, reference is made to Fig. 9 A computing device according to an exemplary embodiment of the present disclosure is described.

[0112] Fig. 9 The computing device 900 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0113] Fig. 9 A schematic diagram of a computing device according to an embodiment of the present disclosure is shown in FIG. Fig. 9 As shown, the computing device 900 is in the form of a general computing device. The components of the computing device 900 may include, but are not limited to: at least one processing unit 901, at least one storage unit 902, and a bus 903 connecting different system components (including the processing unit 901 and the storage unit 902). Among them, at least one storage unit 902 stores computer-executable instructions; at least one processing unit 901 includes a processor, and the processor executes the computer-executable instructions to implement the method described above.

[0114] The bus 903 includes a data bus, a control bus, and an address bus.

[0115] The storage unit 902 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 9021 and / or a cache memory 9022 , and may further include a readable medium in the form of a non-volatile memory, such as a read-only memory (ROM) 9023 .

[0116] The storage unit 902 may also include a program / utility 9025 having a set (at least one) of program modules 9024, such program modules 9024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0117] The computing device 900 may also communicate with one or more external devices 904 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 905. Furthermore, the computing device 900 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 906. Fig. 9 As shown, the network adapter 906 communicates with other modules of the computing device 900 via the bus 903. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the computing device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0118] It should be noted that, although several units / modules or sub-units / sub-modules of the cover image determination device are mentioned in the above detailed description, such division is merely exemplary and not mandatory. In fact, according to an embodiment of the present disclosure, the features and functions of two or more units / modules described above may be embodied in one unit / module. Conversely, the features and functions of one unit / module described above may be further divided to be embodied by multiple units / modules.

[0119] In addition, although the operations of the disclosed method are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0120] Although the spirit and principle of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.

Claims

1. A method for determining a cover image, comprising: Extracting audio information contained in the target video; Determine the relevance between the audio segment at the corresponding time contained in the audio information and the video attribute information of the target video, where the video attribute information includes at least one of a video title, a video tag, or a video description; Determining the wonderfulness of the audio segment according to the sound attribute information of the audio information, wherein the sound attribute information includes at least one of sound frequency and sound intensity; Determining a cover image of the target video according to the excitement and the relevance; The determining of the correlation between the audio segment of the corresponding time contained in the audio information and the video attribute information of the target video includes: Analyzing and processing the audio information through a preset speech recognition model to obtain text information arranged in time corresponding to the audio information; Analyzing and processing the relevance between the text information and the video attribute information through a preset relevance model to determine the relevance; The step of determining the cover image of the target video according to the excitement and the relevance includes: Determining, according to the wonderfulness and the relevance, an intersection time period of the audio segment with the highest wonderfulness and the audio segment with the greatest relevance; The cover image is determined according to the image of the target video corresponding to the intersection time period.

2. The method for determining a cover image according to claim 1, wherein determining the wonderfulness of the audio segment according to the sound attribute information of the audio information comprises: The wonderfulness is determined according to an average value of the sound frequency and / or an average value of the sound intensity corresponding to the audio segment.

3. The method for determining a cover image according to claim 1, wherein determining the cover image according to the image of the target video corresponding to the intersection time period comprises: The image of the target video corresponding to the intersection time period is analyzed and processed by at least one of a preset aesthetics model and a preset clarity model to determine the cover image.

4. A device for determining a cover image, comprising: An extraction module, used for extracting audio information contained in the target video; A first determination module, configured to determine a correlation between an audio segment at a corresponding time contained in the audio information and video attribute information of the target video, wherein the video attribute information includes at least one of a video title, a video tag, or a video description; A second determination module, configured to determine the wonderfulness of the audio segment according to the sound attribute information of the audio information, wherein the sound attribute information includes at least one of sound frequency and sound intensity; A processing module, used for determining a cover image of the target video according to the excitement and the relevance; The first determining module is specifically configured to: Analyzing and processing the audio information through a preset speech recognition model to obtain text information arranged in time corresponding to the audio information; Analyzing and processing the relevance between the text information and the video attribute information through a preset relevance model to determine the relevance; The processing module is specifically used for: Determining, according to the wonderfulness and the relevance, an intersection time period of the audio segment with the highest wonderfulness and the audio segment with the greatest relevance; The cover image is determined according to the image of the target video corresponding to the intersection time period.

5. The cover image determination device according to claim 4, wherein the second determination module is specifically configured to: The wonderfulness is determined according to an average value of the sound frequency and / or an average value of the sound intensity corresponding to the audio segment.

6. The device for determining a cover image according to claim 4, wherein when the processing module is used to determine the cover image according to the image of the target video corresponding to the intersection time period, it is specifically used to: The image of the target video corresponding to the intersection time period is analyzed and processed by at least one of a preset aesthetics model and a preset clarity model to determine the cover image.

7. A computing device comprising: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method for determining the cover image according to any one of claims 1 to 3.

8. A storage medium storing computer program instructions, wherein the computer program instructions, when executed, implement the method for determining a cover image according to any one of claims 1 to 3.

9. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for determining a cover image according to any one of claims 1 to 3 is implemented.

Images