Cover generation method and device based on artificial intelligence, computer equipment and medium

Abstract

The invention belongs to the field of artificial intelligence and financial science and technology, and relates to an artificial intelligence-based cover generation method and device, computer equipment and a storage medium, and the method comprises the steps: carrying out the frame extraction of an obtained target video, and obtaining an image frame; processing the image frame based on an image encoder in the image-text understanding model to obtain a target image feature; processing the target image features based on a full connection layer in the image-text understanding model to obtain an image quality score; obtaining a query text corresponding to the target video; processing the query text based on a text encoder in the image-text understanding model to obtain target text features; calculating a semantic matching degree between the target image feature and the target text feature; generating a cover score of the image frame based on the image quality score and the semantic matching degree; and screening out the target image frame with the highest cover score from all the image frames and using the target image frame as the cover of the target video. According to the method, the cover generation efficiency and accuracy are improved based on the use of the image-text understanding model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence development technology and the field of financial technology, and in particular to an artificial intelligence-based cover generation method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of AI-generated content technology, the digital creative industry has ushered in unprecedented changes. In this wave, the automated generation of text, image and even video content has become a reality, greatly enriching the multimedia resources on the Internet. It is particularly noteworthy that short videos, as one of the most popular content forms, have shown an explosive growth trend in both creation and consumption. In order to adapt to the fast-paced lifestyle and fragmented information acquisition habits, a large amount of long video content has been re-edited into short videos to meet the needs of users to efficiently obtain information within a limited time.

[0003] However, in the process of video content production and dissemination, the importance of video cover as the first impression that attracts viewers to click and watch is self-evident. The traditional method of generating video cover is highly dependent on manual selection. Specifically, manual selection of cover requires video producers to repeatedly browse the video content and select the most representative or attractive picture as the cover. This process is not only time-consuming and labor-intensive, but also inefficient. It is also easy to cause inconsistency in cover selection due to personal aesthetic differences, and it is difficult to ensure the accuracy of cover selection. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a cover generation method, device, computer equipment and storage medium based on artificial intelligence, so as to solve the technical problem that the existing video cover generation method is highly dependent on manual selection, which is not only time-consuming and labor-intensive, inefficient, but also prone to inconsistency in cover selection due to personal aesthetic differences, and it is difficult to ensure the accuracy of cover selection.

[0005] In order to solve the above technical problems, the embodiment of the present application provides a cover generation method based on artificial intelligence, which adopts the following technical solutions:

[0006] Acquire a target video to be processed, and perform frame extraction processing on the target video to obtain corresponding image frames; wherein the number of the image frames includes multiple;

[0007] Processing the image frame based on an image encoder in a preset image-text understanding model to obtain corresponding target image features;

[0008] Processing the target image features based on the fully connected layer in the image-text understanding model to obtain a corresponding image quality score;

[0009] Acquire text information corresponding to the target video, and construct a query text based on the text information;

[0010] Processing the query text based on the text encoder in the image-text understanding model to obtain corresponding target text features;

[0011] Calculating the semantic matching degree between the target image feature and the target text feature;

[0012] generating a cover score for the image frame based on the image quality score and the semantic matching degree;

[0013] A target image frame with the highest cover score is selected from all the image frames, and the target image frame is used as the cover of the target video.

[0014] Furthermore, the step of processing the target image features based on the fully connected layer in the image-text understanding model to obtain a corresponding image quality score specifically includes:

[0015] Performing prediction processing on the target image features based on the fully connected layer to obtain corresponding definition prediction values ​​and compression ratio prediction values;

[0016] Normalizing the definition prediction value and the compression ratio prediction value to obtain a corresponding specified definition prediction value and a specified compression ratio prediction value;

[0017] Call the preset conversion formula;

[0018] Calculate the specified definition prediction value and the specified compression ratio prediction value based on the conversion formula to obtain a corresponding calculation result;

[0019] The calculation result is used as the image quality score.

[0020] Furthermore, the step of calculating the semantic matching degree between the target image feature and the target text feature specifically includes:

[0021] Call the preset similarity algorithm;

[0022] Calculate the similarity between the target image feature and the target text feature based on the similarity algorithm;

[0023] The similarity is used as the semantic matching degree.

[0024] Furthermore, the step of generating a cover score of the image frame based on the image quality score and the semantic matching degree specifically includes:

[0025] Obtaining a preset weight determination algorithm;

[0026] generating a first weight corresponding to an image quality factor based on the weight determination algorithm, and generating a second weight corresponding to a semantic matching factor;

[0027] Call the preset score calculation formula;

[0028] The image quality score, the semantic matching degree, the first weight, and the second weight are processed based on the score calculation formula to obtain a cover score of the image frame.

[0029] Furthermore, before the step of processing the image frame by the image encoder based on the preset image-text understanding model to obtain the corresponding target image features, the method further includes:

[0030] Get a pre-built pre-trained dataset, and get a pre-built fine-tuning dataset;

[0031] Dividing the pre-training data set into multiple batches of image-text pair data;

[0032] Calling a preset multimodal learning model, and inputting the image-text pair data into the multimodal learning model to obtain corresponding feature data; wherein the multimodal learning model at least includes an image encoder, a text encoder, and a fully connected layer;

[0033] Calculating a regression loss based on the feature data, and calculating a contrast loss based on the feature data;

[0034] Constructing a comprehensive loss based on the regression loss and the contrast loss;

[0035] Based on the comprehensive loss, the multimodal learning model is trained until it meets the preset training batch, thereby obtaining a corresponding first learning model;

[0036] Based on a preset training strategy, the first learning model is fine-tuned using the fine-tuning data set to obtain a second learning model that meets the expected construction conditions;

[0037] The second learning model is used as the image-text comprehension model.

[0038] Furthermore, the step of obtaining a pre-built pre-training data set specifically includes:

[0039] Get a specified amount of image data from the target website;

[0040] Preprocessing the image data based on a preset image degradation algorithm to obtain corresponding designated image data;

[0041] generating a text label corresponding to the specified image data;

[0042] The pre-training dataset is generated based on the specified image data and the text label.

[0043] Furthermore, the step of calculating the regression loss based on the feature data specifically includes:

[0044] Acquire image features from the feature data;

[0045] Generate a prediction value corresponding to the image feature based on the fully connected layer;

[0046] Obtaining a true value corresponding to the image feature;

[0047] Calculating the Euclidean distance between the true value and the predicted value;

[0048] The Euclidean distance is used as the regression loss.

[0049] In order to solve the above technical problems, the embodiment of the present application further provides a cover generation device based on artificial intelligence, which adopts the following technical solution:

[0050] A first processing module is used to obtain a target video to be processed, and perform frame extraction processing on the target video to obtain a corresponding image frame; wherein the number of the image frames includes a plurality;

[0051] A second processing module is used to process the image frame based on an image encoder in a preset image-text understanding model to obtain corresponding target image features;

[0052] A third processing module is used to process the target image features based on the fully connected layer in the image-text understanding model to obtain a corresponding image quality score;

[0053] A first construction module is used to obtain text information corresponding to the target video and construct a query text based on the text information;

[0054] A fourth processing module, configured to process the query text based on the text encoder in the image-text understanding model to obtain corresponding target text features;

[0055] A first calculation module, used for calculating the semantic matching degree between the target image feature and the target text feature;

[0056] A generating module, configured to generate a cover score of the image frame based on the image quality score and the semantic matching degree;

[0057] The first determination module is used to screen out a target image frame with the highest cover score from all the image frames, and use the target image frame as the cover of the target video.

[0058] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0059] Acquire a target video to be processed, and perform frame extraction processing on the target video to obtain corresponding image frames; wherein the number of the image frames includes multiple;

[0060] Processing the image frame based on an image encoder in a preset image-text understanding model to obtain corresponding target image features;

[0061] Processing the target image features based on the fully connected layer in the image-text understanding model to obtain a corresponding image quality score;

[0062] Acquire text information corresponding to the target video, and construct a query text based on the text information;

[0063] Processing the query text based on the text encoder in the image-text understanding model to obtain corresponding target text features;

[0064] Calculating the semantic matching degree between the target image feature and the target text feature;

[0065] generating a cover score for the image frame based on the image quality score and the semantic matching degree;

[0066] A target image frame with the highest cover score is selected from all the image frames, and the target image frame is used as the cover of the target video.

[0067] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0068] Acquire a target video to be processed, and perform frame extraction processing on the target video to obtain corresponding image frames; wherein the number of the image frames includes multiple;

[0069] Processing the image frame based on an image encoder in a preset image-text understanding model to obtain corresponding target image features;

[0070] Processing the target image features based on the fully connected layer in the image-text understanding model to obtain a corresponding image quality score;

[0071] Acquire text information corresponding to the target video, and construct a query text based on the text information;

[0072] Processing the query text based on the text encoder in the image-text understanding model to obtain corresponding target text features;

[0073] Calculating the semantic matching degree between the target image feature and the target text feature;

[0074] generating a cover score for the image frame based on the image quality score and the semantic matching degree;

[0075] A target image frame with the highest cover score is selected from all the image frames, and the target image frame is used as the cover of the target video.

[0076] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0077] The present application first obtains a target video to be processed, and extracts frames from the target video to obtain corresponding image frames; wherein the number of the image frames includes multiple; then the image frames are processed based on an image encoder in a preset image-text understanding model to obtain corresponding target image features; and the target image features are processed based on a fully connected layer in the image-text understanding model to obtain corresponding image quality scores; then text information corresponding to the target video is obtained, and a query text is constructed based on the text information; subsequently, the query text is processed based on a text encoder in the image-text understanding model to obtain corresponding target text features; the semantic matching degree between the target image features and the target text features is further calculated; finally, a cover score of the image frame is generated based on the image quality score and the semantic matching degree, and the target image frame with the highest cover score is screened out from all the image frames, and the target image frame is used as the cover of the video. The present application obtains a target video to be processed, and performs frame extraction processing on the target video to obtain corresponding image frames, then performs image quality analysis on the image frames based on the use of a graphic-text understanding model to obtain a corresponding image quality score, and simultaneously obtains text information corresponding to the target video, and constructs a query text based on the text information, then performs semantic matching processing on the query text and the image frames based on the use of a graphic-text understanding model to obtain a corresponding semantic matching degree, and subsequently generates a cover score for the image frame based on the image quality score and the semantic matching degree, and finally filters out the target image frame with the highest cover score from all the image frames, and uses the target image frame as the cover of the target video, so as to realize automatic, fast and accurate generation of the cover of the target video, avoid the operation of manually selecting the cover, effectively improve the efficiency of cover generation, and ensure the accuracy of cover generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0079] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0080] Figure 2 A flowchart of an embodiment of a cover generation method based on artificial intelligence according to the present application;

[0081] Figure 3 is a structural schematic diagram of an embodiment of a cover generation device based on artificial intelligence according to the present application;

[0082] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0083] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0084] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0085] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0086] like Figure 1As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0087] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0088] The terminal device 101 can be various electronic devices with a display screen and supporting web browsing. In addition to the laptop computer 1011, the tablet computer 1012 or the mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop computer, a desktop computer, etc.

[0089] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0090] It should be noted that the artificial intelligence-based cover generation method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the artificial intelligence-based cover generation device is generally set in the server / terminal device.

[0091] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0092] Continue to refer Figure 2, showing a flowchart of an embodiment of the cover generation method based on artificial intelligence according to the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted. The cover generation method based on artificial intelligence provided in the embodiment of the present application can be applied to any scenario where video cover generation is required, and the cover generation method based on artificial intelligence can be applied to products in these scenarios, for example, the cover generation of insurance videos in the financial insurance field. The cover generation method based on artificial intelligence includes the following steps:

[0093] Step S201, obtaining a target video to be processed, and performing frame extraction processing on the target video to obtain corresponding image frames; wherein the number of the image frames includes multiple.

[0094] In this embodiment, the electronic device (e.g. Figure 1 The target video to be processed can be obtained by a wired connection or a wireless connection. It should be noted that the wireless connection method may include but is not limited to 3G / 4G / 5G connection, Wi-Fi connection, Bluetooth connection, Wi MAX connection, Zigbee connection, UWB (ultrawi deband) connection, and other wireless connection methods currently known or to be developed in the future. In the insurance field, the target video may be a video for which a video cover needs to be generated, such as an insurance product marketing video, an insurance product promotion video, and the like. Among them, the target video to be processed can be read by using a video processing library (such as OpenCV, ffmpeg-python, etc.), and then the frame extraction interval can be set or the target video can be extracted based on specific conditions (such as scene changes, key frame detection) to obtain corresponding multiple image frames.

[0095] Step S202: Process the image frame based on the image encoder in the preset image-text understanding model to obtain corresponding target image features.

[0096] In this embodiment, the image frame can be input into an image encoder in a preset image-text understanding model, and the image frame can be converted into a 512-dimensional feature vector by the image encoder to obtain the corresponding target image features. The above-mentioned image-text understanding model is a pre-trained model. The construction process of the image-text understanding model will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated on here.

[0097] Step S203: Process the target image features based on the fully connected layer in the image-text understanding model to obtain a corresponding image quality score.

[0098] In this embodiment, the above-mentioned specific implementation process of processing the target image features based on the fully connected layer in the image-text understanding model to obtain the corresponding image quality score will be further described in detail in subsequent specific embodiments of the present application and will not be elaborated in detail here.

[0099] Step S204: Acquire text information corresponding to the target video, and construct a query text based on the text information.

[0100] In this embodiment, the text information may include the title of the target video, keywords of the target video, or relevant information about the target video input by the user. If the text information includes only one type of information, the text information is directly used as the query text. If the text information includes multiple types of information (title, keywords, relevant information), all the text information is integrated and the integrated information is used as the query text.

[0101] Step S205: Process the query text based on the text encoder in the image-text understanding model to obtain corresponding target text features.

[0102] In this embodiment, the query text may be input into a text encoder in the above-mentioned image-text understanding model, and the query text may be converted into a 512-dimensional feature vector by the text encoder to obtain corresponding target text features.

[0103] Step S206, calculating the semantic matching degree between the target image feature and the target text feature.

[0104] In this embodiment, the specific implementation process of calculating the semantic matching degree between the target image feature and the target text feature will be described in further detail in subsequent specific embodiments of the present application, and will not be elaborated on here.

[0105] Step S207: generating a cover score for the image frame based on the image quality score and the semantic matching degree.

[0106] In this embodiment, video cover generation includes two aspects: 1. Image quality. Image quality analysis refers to inputting an image and outputting various objective scores of image quality; 2. Image semantic matching, which mainly refers to the semantic matching of the cover content and the video. Specifically, the specific implementation process of generating the cover score of the image frame based on the image quality score and the semantic matching degree will be further described in the subsequent specific embodiments of this application, and will not be elaborated on here.

[0107] Step S208, selecting a target image frame with the highest cover score from all the image frames, and using the target image frame as the cover of the target video.

[0108] In this embodiment, by numerically comparing the cover scores of all the obtained image frames, the target image frame with the highest cover score can be screened out from all the image frames, and then the target image frame can be directly used as the cover of the target video.

[0109] The present application obtains a target video to be processed, and performs frame extraction processing on the target video to obtain corresponding image frames, then performs image quality analysis on the image frames based on the use of a graphic-text understanding model to obtain a corresponding image quality score, and simultaneously obtains text information corresponding to the target video, and constructs a query text based on the text information, then performs semantic matching processing on the query text and the image frames based on the use of a graphic-text understanding model to obtain a corresponding semantic matching degree, and subsequently generates a cover score for the image frame based on the image quality score and the semantic matching degree, and finally filters out the target image frame with the highest cover score from all the image frames, and uses the target image frame as the cover of the target video, so as to realize automatic, fast and accurate generation of the cover of the target video, avoid the operation of manually selecting the cover, effectively improve the efficiency of cover generation, and ensure the accuracy of cover generation.

[0110] In some optional implementations, step S203 includes the following steps:

[0111] The target image features are predicted based on the fully connected layer to obtain corresponding clarity prediction values ​​and compression ratio prediction values.

[0112] In this embodiment, the target image features output by the image encoder in the image-text understanding model after processing the image frame are used as the input of the fully connected layer in the image-text understanding model, so as to predict the clarity and compression ratio of the target image features through the fully connected layer, and output the corresponding clarity prediction value and compression ratio prediction value.

[0113] The definition prediction value and the compression ratio prediction value are normalized to obtain corresponding specified definition prediction value and specified compression ratio prediction value.

[0114] In this embodiment, since the clarity prediction value and the compression ratio prediction value may be in different dimensions and ranges, they need to be normalized to ensure that they have the same contribution when calculating the comprehensive score. Among them, the clarity prediction value and the compression ratio prediction value can be normalized by using a min-max normalization method to obtain the corresponding specified clarity prediction value and the specified compression ratio prediction value.

[0115] Call the preset conversion formula.

[0116] In this embodiment, the above conversion formula may specifically include S=c*E+d*F, where S is the image quality score, E is the definition prediction value, F is the compression ratio prediction value, c is the first preset weight corresponding to the definition, and d is the second preset weight corresponding to the compression ratio. There is no specific limitation on the selection of the above first preset weight and the second preset weight. Specifically, different weights may be assigned to the definition and the compression ratio according to their importance to the image quality. These preset weights may be determined based on experience, user feedback, or experimental data.

[0117] The specified definition prediction value and the specified compression ratio prediction value are calculated based on the conversion formula to obtain corresponding calculation results.

[0118] In this embodiment, the specified definition prediction value and the specified compression ratio prediction value may be substituted into corresponding positions in the above conversion formula for calculation, thereby obtaining a corresponding calculation result.

[0119] The calculation result is used as the image quality score.

[0120] The present application predicts the target image features based on the fully connected layer to obtain the corresponding clarity prediction value and compression ratio prediction value; then normalizes the clarity prediction value and the compression ratio prediction value to obtain the corresponding specified clarity prediction value and the specified compression ratio prediction value; then calls a preset conversion formula, and calculates the specified clarity prediction value and the specified compression ratio prediction value based on the conversion formula to obtain the corresponding calculation result; and subsequently uses the calculation result as the image quality score. The present application predicts the target image features based on the fully connected layer to obtain the corresponding clarity prediction value and the compression ratio prediction value, and then intelligently normalizes the clarity prediction value and the compression ratio prediction value to obtain the corresponding specified clarity prediction value and the specified compression ratio prediction value, and then calculates the specified clarity prediction value and the specified compression ratio prediction value based on the use of the conversion formula, so that the image quality score corresponding to the target image feature can be quickly and accurately generated, effectively ensuring the data accuracy of the obtained image quality score.

[0121] In some optional implementations of this embodiment, step S206 includes the following steps:

[0122] Call the preset similarity algorithm.

[0123] In this embodiment, the above similarity algorithm may specifically adopt a cosine similarity algorithm. The cosine similarity algorithm is a method for evaluating the similarity between two vectors by calculating the cosine value of the angle between them.

[0124] The similarity between the target image feature and the target text feature is calculated based on the similarity algorithm.

[0125] In this embodiment, the cosine similarity between the target image feature and the target text feature may be calculated by using a cosine similarity algorithm.

[0126] The similarity is used as the semantic matching degree.

[0127] In this embodiment, the calculated cosine similarity may be used as the semantic matching degree between the target image feature and the target text feature.

[0128] The present application calls a preset similarity algorithm; then calculates the similarity between the target image feature and the target text feature based on the similarity algorithm; and subsequently uses the similarity as the semantic matching degree. The present application uses a similarity algorithm to calculate the similarity between the target image feature and the target text feature, thereby enabling the rapid and accurate generation of the semantic matching degree between the target image feature and the target text feature, thereby improving the efficiency of generating the semantic matching degree and ensuring the data accuracy of the obtained semantic matching degree.

[0129] In some optional implementations, step S207 includes the following steps:

[0130] Gets the preset weight determination algorithm.

[0131] In this embodiment, there is no specific limitation on the selection of the above-mentioned weight determination algorithm, which can be determined according to actual business needs. Specifically, any one of principal component analysis, factor analysis, AHP hierarchical analysis method, entropy method, standardization method, etc. can be adopted.

[0132] A first weight corresponding to an image quality factor is generated based on the weight determination algorithm, and a second weight corresponding to a semantic matching factor is generated.

[0133] In this embodiment, the image quality factor and the semantic matching factor can be weighted according to the corresponding weight calculation processing steps in the above-mentioned weight determination algorithm, thereby generating a first weight corresponding to the image quality factor and a second weight corresponding to the semantic matching factor.

[0134] Call the preset score calculation formula.

[0135] In this embodiment, the above score calculation formula may specifically include: G=a*M+b*N, where G is the algorithm score, M is the image quality score, a is the first weight, N is the semantic matching degree, and b is the second weight.

[0136] The image quality score, the semantic matching degree, the first weight, and the second weight are processed based on the score calculation formula to obtain a cover score of the image frame.

[0137] In this embodiment, the cover score of the image frame can be obtained by substituting the image quality score, the semantic matching degree, the first weight and the second weight into the corresponding positions in the above score calculation formula for calculation processing.

[0138] The present application obtains a preset weight determination algorithm; then generates a first weight corresponding to the image quality factor based on the weight determination algorithm, and generates a second weight corresponding to the semantic matching factor; then calls a preset score calculation formula; and subsequently processes the image quality score, the semantic matching, the first weight, and the second weight based on the score calculation formula to obtain the cover score of the image frame. The present application generates a first weight corresponding to the image quality factor based on the weight determination algorithm, and generates a second weight corresponding to the semantic matching factor, and then processes the image quality score, the semantic matching, the first weight, and the second weight based on the use of the score calculation formula, thereby realizing fast and accurate generation of the cover score of the image frame, and effectively ensuring the accuracy of the generated cover score. It is beneficial to subsequently screen out the image frame with the highest cover score from all image frames according to the cover scores of each image frame obtained as the final target image frame and used as the cover of the target video, thereby improving the accuracy of the cover of the generated target video.

[0139] In some optional implementations, before step S202, the electronic device may further perform the following steps:

[0140] Get a pre-built pre-trained dataset, and get a pre-built fine-tuned dataset.

[0141] In this embodiment, the construction process of the above-mentioned fine-tuning dataset may include: 1. Video cover collection: Collect diverse video covers from the existing video library, and ensure that the video cover content covers different video types and themes. Image degradation: Similarly, an image degradation algorithm is applied to each video cover to generate versions with different clarity and compression ratios. Text label annotation: Manually label each cover with accurate text descriptions. These labels should accurately reflect the cover content and be as concise and clear as possible. Tag management tools (such as LabelStud io, Datatur ks, etc.) can be used to simplify the annotation process. In addition, the specific implementation process of obtaining the pre-constructed pre-training dataset mentioned above will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated on here.

[0142] The pre-training data set is divided into multiple batches of image-text pair data.

[0143] In this embodiment, in the training cycle, the pre-training data set is loaded into the memory and divided into multiple batches. Each batch contains a certain number of paired image-text pairs. These data are randomly shuffled to ensure that the model is not affected by the order of the data during the training process.

[0144] The preset multimodal learning model is called, and the image-text pair data is input into the multimodal learning model to obtain corresponding feature data. The multimodal learning model at least includes an image encoder, a text encoder, and a fully connected layer.

[0145] In this embodiment, the multimodal learning model can be specifically a model that improves the CLIP model, that is, a fully connected layer is added on the basis of the CLIP model (including an image encoder and a text encoder) for regression prediction. The corresponding feature data is obtained by inputting the image-text pair data into the multimodal learning model. The feature data includes image features and text features corresponding to the image-text pair data respectively. Among them, the image data in the image-text pair data can be processed by using the image encoder in the multimodal learning model to obtain the corresponding image features, and the text data in the image-text pair data can be processed by using the text encoder in the multimodal learning model to obtain the corresponding text features.

[0146] A regression loss is calculated based on the feature data, and a contrast loss is calculated based on the feature data.

[0147] In this embodiment, the process of calculating the contrast loss based on the feature data includes: performing L2 normalization on the image features and text features in the feature data so that they have unit length. Then the cosine similarity (log its) between the standardized image features and text features is calculated, which represents the degree of matching between the image and the text. After that, the first contrast loss (loss_i) from the image to the text and the second contrast loss (loss_t) from the text to the image are calculated using the cross entropy loss (cross_entropy_loss). This usually involves converting the log its into a probability distribution and calculating the cross entropy with the true match (i.e., the same pair of image-text data). Finally, the average of the first contrast loss (loss_i) and the second contrast loss (loss_t) is taken as the final contrast loss. In addition, the specific implementation process of calculating the regression loss based on the feature data and calculating the contrast loss based on the feature data will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated on here.

[0148] A comprehensive loss is constructed based on the regression loss and the contrast loss.

[0149] In this embodiment, the training of the model is determined by two losses, wherein the contrast loss is to facilitate the subsequent search for covers through text, and the regression loss is to train image quality assessment. Specifically, the weighted sum of the regression loss and the contrast loss can be calculated to obtain the corresponding comprehensive loss. The weights of the regression loss and the contrast loss can be adjusted according to the specific task requirements.

[0150] The multimodal learning model is trained based on the comprehensive loss until it meets the preset training batch, thereby obtaining a corresponding first learning model.

[0151] In this embodiment, the entire multimodal learning model is optimized by the back propagation algorithm according to the above-mentioned comprehensive loss, and the parameters of the image encoder, text encoder and fully connected layer are updated at the same time, and the training cycle process is repeated until the model shows good performance on the validation set or reaches the preset training rounds, thereby obtaining the corresponding first learning model. Among them, there is no specific limitation on the selection of the numerical value of the above-mentioned training rounds, and it can be adjusted according to the specific task requirements. In addition, during the model training process, strategies such as learning rate scheduler and early stopping can be used to optimize the training process.

[0152] Based on a preset training strategy, the first learning model is fine-tuned using the fine-tuning data set to obtain a second learning model that meets the expected construction conditions.

[0153] In this embodiment, the above training strategy is the same strategy as the strategy for training the multimodal learning model using a pre-training data set, except that the data used for training is different. The specific process of using the fine-tuning data set to train the first learning model to obtain a second learning model that meets the expected construction conditions can refer to the above-mentioned process of training the multimodal learning model using a pre-training data set, and will not be described in detail here. In addition, the second learning model obtained after the training is completed can not only automatically assign a cover to the video by image quality, but also jointly determine the video cover by the keywords of the video.

[0154] The second learning model is used as the image-text comprehension model.

[0155] In this embodiment, through pre-training on the pre-training dataset and fine-tuning on the fine-tuning dataset, the regression loss of the CLIP model and the contrast loss of the image and text can be optimized at the same time. The final image-text understanding model can not only output the image clarity and compression ratio indicators, but also use the cosine similarity of the representation of the given input text and image to finally determine the video cover through dual indicators.

[0156] The present application obtains a pre-constructed pre-training data set and a pre-constructed fine-tuning data set, and divides the pre-training data set into multiple batches of image-text pair data; then calls a preset multimodal learning model, and inputs the image-text pair data into the multimodal learning model to obtain corresponding feature data; then calculates regression loss based on the feature data, and calculates contrast loss based on the feature data; and constructs a comprehensive loss based on the regression loss and the contrast loss; subsequently, the multimodal learning model is trained based on the comprehensive loss until it meets the preset training batch and obtains the corresponding first learning model; further based on the preset training strategy, the first learning model is fine-tuned using the fine-tuning data set to obtain a second learning model that meets the expected construction conditions; finally, the second learning model is used as the image-text understanding model. In this way, the multimodal learning model is trained and fine-tuned based on the pre-training data set, the fine-tuning data set, and the use of the comprehensive loss constructed by combining regression loss and contrast loss, so that the construction of the image-text understanding model can be completed quickly and intelligently, the construction efficiency of the image-text understanding model is improved, and the model effect of the obtained image-text understanding model is guaranteed.

[0157] In some optional implementations of this embodiment, obtaining a pre-built pre-training data set includes the following steps:

[0158] Gets a specified amount of image data from a target website.

[0159] In this embodiment, the target website is specifically an image website, such as pexe ls. A large number of high-quality images can be downloaded from the target website by using a crawler, and the images can be ensured to cover different themes, styles and scenes to increase the generalization ability of the model. There is no specific limitation on the selection of the value of the specified number, which can be set according to the actual model training requirements, for example, it can be set to 100000.

[0160] The image data is preprocessed based on a preset image degradation algorithm to obtain corresponding designated image data.

[0161] In this embodiment, the above-mentioned image degradation algorithm can specifically adopt Gaussian blur, JPEG compression, noise addition and other algorithms. Degradation levels corresponding to different sharpness and compression ratios are preset, and then the image degradation algorithm is used to process each image data to simulate the different sharpness and compression ratio corresponding to the degradation level, so as to generate corresponding specific image data of different sharpness and compression ratio versions, and finally the image data and the obtained specific image data are constructed to obtain the above-mentioned designated image data.

[0162] A text label corresponding to the specified image data is generated.

[0163] In this embodiment, a text description (capt ion) may be generated for each designated image data by using a BLIP model, and these text descriptions may be used as text labels corresponding to the designated image data during training.

[0164] The pre-training dataset is generated based on the specified image data and the text label.

[0165] In this embodiment, the above-mentioned pre-training data set can be obtained by constructing corresponding training image-text pair data by combining the specified image data with the corresponding text labels.

[0166] The present application obtains a specified amount of image data from a target website; then pre-processes the image data based on a preset image degradation algorithm to obtain the corresponding specified image data; then generates a text label corresponding to the specified image data; and subsequently generates the pre-training data set based on the specified image data and the text label. The present application pre-processes the image data obtained from a target website using an image degradation algorithm to obtain the specified image data, and then generates a text label corresponding to the specified image data. Then, based on the specified image data and the text label, the required pre-training data set can be quickly and intelligently constructed, thereby improving the construction efficiency and construction intelligence of the pre-training data set.

[0167] In some optional implementations of this embodiment, the calculating the regression loss based on the feature data includes the following steps:

[0168] An image feature is obtained from the feature data.

[0169] In this embodiment, the feature data includes image features and text features corresponding to the image-text pair data, respectively. The image data in the image-text pair data can be processed by using an image encoder in a multimodal learning model to obtain corresponding image features, and the text data in the image-text pair data can be processed by using a text encoder in a multimodal learning model to obtain corresponding text features.

[0170] Generate a prediction value corresponding to the image feature based on the fully connected layer.

[0171] In this embodiment, the image features output by the image encoder from the image data in the image-text pair data can be predicted by using the fully connected layer in the multimodal learning model, and then the corresponding image quality prediction value can be output. The image quality prediction value may include a specific definition prediction value and a specific compression ratio prediction value.

[0172] Obtain a true value corresponding to the image feature.

[0173] In this embodiment, the real value may include a real definition value and a real compression ratio value of the image feature.

[0174] The Euclidean distance between the true value and the predicted value is calculated.

[0175] In this embodiment, the Euclidean distance between the true value and the predicted value may be calculated by calling a calculation formula of the Euclidean distance.

[0176] The Euclidean distance is used as the regression loss.

[0177] The present application obtains image features from the feature data; then generates a predicted value corresponding to the image feature based on the fully connected layer; then obtains the true value corresponding to the image feature; subsequently calculates the Euclidean distance between the true value and the predicted value, and uses the Euclidean distance as the regression loss. The present application obtains image features from the feature data, generates a predicted value corresponding to the image feature based on the fully connected layer, and subsequently calculates the Euclidean distance between the predicted value and the true value corresponding to the image feature as the corresponding regression loss, thereby effectively improving the computational intelligence of the regression loss and ensuring the accuracy of the obtained regression loss.

[0178] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0179] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned target image frame, the above-mentioned target image frame can also be stored in a blockchain node.

[0180] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0181] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0182] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0183] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0184] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0185] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a cover generation device based on artificial intelligence, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0186] like Figure 3 As shown, the cover generation device 300 based on artificial intelligence described in this embodiment includes: a first processing module 301, a second processing module 302, a third processing module 303, a first construction module 304, a fourth processing module 305, a first calculation module 306, a generation module 307 and a first determination module 308. Among them:

[0187] The first processing module 301 is used to obtain a target video to be processed, and perform frame extraction processing on the target video to obtain a corresponding image frame; wherein the number of the image frames includes multiple;

[0188] A second processing module 302 is used to process the image frame based on an image encoder in a preset image-text understanding model to obtain corresponding target image features;

[0189] The third processing module 303 is used to process the target image features based on the fully connected layer in the image-text understanding model to obtain a corresponding image quality score;

[0190] A first construction module 304 is used to obtain text information corresponding to the target video and construct a query text based on the text information;

[0191] A fourth processing module 305 is used to process the query text based on the text encoder in the image-text understanding model to obtain corresponding target text features;

[0192] A first calculation module 306, used to calculate the semantic matching degree between the target image feature and the target text feature;

[0193] A generating module 307, configured to generate a cover score of the image frame based on the image quality score and the semantic matching degree;

[0194] The first determination module 308 is used to select a target image frame with the highest cover score from all the image frames, and use the target image frame as the cover of the target video.

[0195] In some optional implementations of this embodiment, the third processing module 303 includes:

[0196] A prediction submodule, used for performing prediction processing on the target image features based on the fully connected layer to obtain corresponding definition prediction values ​​and compression ratio prediction values;

[0197] A first processing submodule is used to perform normalization processing on the definition prediction value and the compression ratio prediction value to obtain a corresponding specified definition prediction value and a specified compression ratio prediction value;

[0198] The first calling submodule is used to call a preset conversion formula;

[0199] A first calculation submodule, configured to calculate the specified definition prediction value and the specified compression ratio prediction value based on the conversion formula to obtain a corresponding calculation result;

[0200] The first determination submodule is used to use the calculation result as the image quality score.

[0201] In some optional implementations of this embodiment, the first calculation module 306 includes:

[0202] The second calling submodule is used to call a preset similarity algorithm;

[0203] A second calculation submodule, used for calculating the similarity between the target image feature and the target text feature based on the similarity algorithm;

[0204] The second determining submodule is used to use the similarity as the semantic matching degree.

[0205] In some optional implementations of this embodiment, the generating module 307 includes:

[0206] A first acquisition submodule, used to acquire a preset weight determination algorithm;

[0207] A first generating submodule, configured to generate a first weight corresponding to an image quality factor based on the weight determination algorithm, and to generate a second weight corresponding to a semantic matching factor;

[0208] The third calling submodule is used to call a preset score calculation formula;

[0209] The third calculation submodule is used to process the image quality score, the semantic matching degree, the first weight and the second weight based on the score calculation formula to obtain the cover score of the image frame.

[0210] In some optional implementations of this embodiment, the cover generation device based on artificial intelligence further includes:

[0211] An acquisition module, used to acquire a pre-built pre-training dataset and to acquire a pre-built fine-tuning dataset;

[0212] A division module, used for dividing the pre-training data set into multiple batches of image-text pair data;

[0213] An input module, used to call a preset multimodal learning model, and input the image-text pair data into the multimodal learning model to obtain corresponding feature data; wherein the multimodal learning model at least includes an image encoder, a text encoder and a fully connected layer;

[0214] A second calculation module, used to calculate the regression loss based on the feature data, and to calculate the contrast loss based on the feature data;

[0215] A second construction module is used to construct a comprehensive loss based on the regression loss and the contrast loss;

[0216] A training module, used for training the multimodal learning model based on the comprehensive loss until it meets the preset training batch and obtains the corresponding first learning model;

[0217] A fine-tuning module, configured to fine-tune the first learning model using the fine-tuning data set based on a preset training strategy to obtain a second learning model that meets the expected construction conditions;

[0218] The second determining module is used to use the second learning model as the image-text understanding model.

[0219] In some optional implementations of this embodiment, the acquisition module includes:

[0220] The second acquisition submodule is used to acquire a specified amount of image data from a target website;

[0221] A preprocessing submodule, used to preprocess the image data based on a preset image degradation algorithm to obtain corresponding specified image data;

[0222] A second generating submodule, used for generating a text label corresponding to the specified image data;

[0223] The third generating submodule is used to generate the pre-training data set based on the specified image data and the text label.

[0224] In some optional implementations of this embodiment, the second calculation module includes:

[0225] A third acquisition submodule, used to acquire image features from the feature data;

[0226] a fourth generating submodule, configured to generate a prediction value corresponding to the image feature based on the fully connected layer;

[0227] A fourth acquisition submodule, used to acquire a true value corresponding to the image feature;

[0228] A fourth calculation submodule, used for calculating the Euclidean distance between the true value and the predicted value;

[0229] The third determination submodule is used to use the Euclidean distance as the regression loss.

[0230] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0231] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital processors (DSPs), embedded devices, etc.

[0232] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0233] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Med ia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the cover generation method based on artificial intelligence, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0234] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the computer-readable instructions stored in the memory 41 or process data, such as running the computer-readable instructions of the cover generation method based on artificial intelligence.

[0235] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0236] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned artificial intelligence-based cover generation method.

[0237] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0238] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A cover generation method based on artificial intelligence, characterized in that: The steps include: Acquire a target video to be processed, and perform frame extraction processing on the target video to obtain corresponding image frames; wherein the number of the image frames includes multiple; Processing the image frame based on an image encoder in a preset image-text understanding model to obtain corresponding target image features; Processing the target image features based on the fully connected layer in the image-text understanding model to obtain a corresponding image quality score; Acquire text information corresponding to the target video, and construct a query text based on the text information; Processing the query text based on the text encoder in the image-text understanding model to obtain corresponding target text features; Calculating the semantic matching degree between the target image feature and the target text feature; generating a cover score for the image frame based on the image quality score and the semantic matching degree; A target image frame with the highest cover score is selected from all the image frames, and the target image frame is used as the cover of the target video.

2. The method for generating a cover based on artificial intelligence according to claim 1, characterized in that: The step of processing the target image features based on the fully connected layer in the image-text understanding model to obtain a corresponding image quality score specifically includes: Performing prediction processing on the target image features based on the fully connected layer to obtain corresponding definition prediction values ​​and compression ratio prediction values; Normalizing the definition prediction value and the compression ratio prediction value to obtain a corresponding specified definition prediction value and a specified compression ratio prediction value; Call the preset conversion formula; Calculate the specified definition prediction value and the specified compression ratio prediction value based on the conversion formula to obtain a corresponding calculation result; The calculation result is used as the image quality score.

3. The cover generation method based on artificial intelligence according to claim 1 is characterized in that: The step of calculating the semantic matching degree between the target image feature and the target text feature specifically includes: Call the preset similarity algorithm; Calculate the similarity between the target image feature and the target text feature based on the similarity algorithm; The similarity is used as the semantic matching degree.

4. The cover generation method based on artificial intelligence according to claim 1 is characterized in that: The step of generating a cover score for the image frame based on the image quality score and the semantic matching degree specifically includes: Obtaining a preset weight determination algorithm; generating a first weight corresponding to an image quality factor based on the weight determination algorithm, and generating a second weight corresponding to a semantic matching factor; Call the preset score calculation formula; The image quality score, the semantic matching degree, the first weight, and the second weight are processed based on the score calculation formula to obtain a cover score of the image frame.

5. The method for generating a cover based on artificial intelligence according to claim 1, characterized in that: Before the step of processing the image frame by the image encoder based on the preset image-text understanding model to obtain the corresponding target image features, the step further includes: Get a pre-built pre-trained dataset, and get a pre-built fine-tuning dataset; Dividing the pre-training data set into multiple batches of image-text pair data; Calling a preset multimodal learning model, and inputting the image-text pair data into the multimodal learning model to obtain corresponding feature data; wherein the multimodal learning model at least includes an image encoder, a text encoder, and a fully connected layer; Calculating a regression loss based on the feature data, and calculating a contrast loss based on the feature data; Constructing a comprehensive loss based on the regression loss and the contrast loss; Based on the comprehensive loss, the multimodal learning model is trained until it meets the preset training batch, thereby obtaining a corresponding first learning model; Based on a preset training strategy, the first learning model is fine-tuned using the fine-tuning data set to obtain a second learning model that meets the expected construction conditions; The second learning model is used as the image-text comprehension model.

6. The method for generating a cover based on artificial intelligence according to claim 5, characterized in that: The step of obtaining a pre-built pre-training data set specifically includes: Get a specified amount of image data from the target website; Preprocessing the image data based on a preset image degradation algorithm to obtain corresponding designated image data; generating a text label corresponding to the specified image data; The pre-training dataset is generated based on the specified image data and the text label.

7. The method for generating a cover based on artificial intelligence according to claim 5, characterized in that: The step of calculating the regression loss based on the feature data specifically includes: Acquire image features from the feature data; Generate a prediction value corresponding to the image feature based on the fully connected layer; Obtaining a true value corresponding to the image feature; Calculating the Euclidean distance between the true value and the predicted value; The Euclidean distance is used as the regression loss.

8. A cover generation device based on artificial intelligence, characterized in that: include: A first processing module is used to obtain a target video to be processed, and perform frame extraction processing on the target video to obtain a corresponding image frame; wherein the number of the image frames includes a plurality; A second processing module is used to process the image frame based on an image encoder in a preset image-text understanding model to obtain corresponding target image features; A third processing module is used to process the target image features based on the fully connected layer in the image-text understanding model to obtain a corresponding image quality score; A first construction module is used to obtain text information corresponding to the target video and construct a query text based on the text information; A fourth processing module, configured to process the query text based on the text encoder in the image-text understanding model to obtain corresponding target text features; A first calculation module, used for calculating the semantic matching degree between the target image feature and the target text feature; A generating module, configured to generate a cover score of the image frame based on the image quality score and the semantic matching degree; The first determination module is used to screen out a target image frame with the highest cover score from all the image frames, and use the target image frame as the cover of the target video.

9. A computer device, characterized in that: It includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the cover generation method based on artificial intelligence as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the artificial intelligence-based cover generation method as described in any one of claims 1 to 7.

Images