Video processing method and device, computer device, storage medium and program product

Abstract

The application provides a video processing method and device, computer equipment, a storage medium and a program product, and relates to the field of video processing. The video processing method comprises: acquiring a basic feature of a target video; the basic feature is determined based on at least one of a source object identifier, label information and video length of the target video; predicting the basic feature to obtain a basic feature influence; acquiring a multi-modal feature of the target video; the multi-modal feature is determined based on at least one of a keyword feature, a video cover feature and a video content feature of the target video; predicting the multi-modal feature to obtain a multi-modal feature influence; and determining a target prediction influence of the target video based on the basic feature influence and the multi-modal feature influence. The embodiments of the application can be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation and auxiliary driving. The accuracy of the influence prediction of the target video can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and specifically to a video processing method, apparatus, computer equipment, storage medium, and program product. Background Technology

[0002] Video influence represents the quality level of a video; a higher video influence value indicates higher video quality. Videos with high influence can increase view counts, and predicting video influence in advance plays an important role in the field of internet data mining.

[0003] The platform's video push process includes two stages: video exposure and video viewing. However, currently, predicting video influence solely based on video content before its release has a low accuracy rate. Summary of the Invention

[0004] This application provides a video processing method, apparatus, computer equipment, storage medium, and program product, which can solve the problem of low accuracy in video influence prediction methods in related technologies. The technical solution is as follows:

[0005] On the one hand, a video processing method is provided, the method including:

[0006] Obtain the basic features of the target video; the basic features are determined based on at least one of the target video's source object identifier, tag information, and video length;

[0007] The basic features are predicted by the first influence prediction model to obtain the basic feature influence of the target video.

[0008] Obtain the multimodal features of the target video; the multimodal features are determined based on at least one of the target video's keyword features, video cover features, and video content features;

[0009] The influence of multimodal features on the target video is obtained by predicting the multimodal features using the second influence prediction model.

[0010] The target predictive influence of the target video is determined based on the influence of basic features and the influence of multimodal features.

[0011] In one possible implementation, the first and second influence prediction models are trained as follows:

[0012] Acquire multiple sample videos;

[0013] Calculate the true impact of each sample video;

[0014] Obtain the basic features and multimodal features of each video sample;

[0015] Input the basic features of each sample into the first initial prediction model to obtain the corresponding prediction influence of each first sample.

[0016] The multimodal features of each sample are input into the second initial prediction model to obtain the corresponding prediction influence of each second sample.

[0017] Based on the predicted influence of each first sample and the predicted influence of each second sample, the influence of each sample is determined.

[0018] Based on the influence of each sample and the corresponding real influence, the first initial prediction model and the second initial prediction model are trained to obtain the first influence prediction model and the second influence prediction model.

[0019] In one possible implementation, the true impact of each sample video is calculated, including:

[0020] For each sample video, obtain the unique user data for exposure and the unique user data for clicks.

[0021] Click-through rate is determined based on unique user click data and unique user impression data.

[0022] The true influence is determined based on unique user exposure data, unique user click data, and user click-through rate.

[0023] In one possible implementation, true influence is determined based on unique user exposure data, unique user click data, and user click-through rate, including:

[0024] Determine the first weight for unique user exposure data, the second weight for unique user click data, and the third weight for user click-through rate;

[0025] The true influence is obtained by weighting the unique user data for exposure, unique user data for clicks, and user click-through rate based on the first, second, and third weights.

[0026] In one possible implementation, the basic features of the target video are obtained, including:

[0027] The source object identifier is converted into an object identifier feature vector, and the tag information is converted into a tag feature vector.

[0028] The video length is converted into an initial length vector, and the initial length vector is normalized to obtain the video length vector;

[0029] The basic features are obtained by concatenating the object identifier feature vector, the label feature vector, and the video length vector.

[0030] In one possible implementation, the multimodal features of the target video are obtained, including:

[0031] Obtain the video cover image of the target video, and extract features from the video cover image to obtain the video cover features;

[0032] Obtain the key text of the target video; the key text includes at least one of the target video's title and summary;

[0033] Extract key text features to obtain initial text features, then segment the initial text features to obtain keyword features;

[0034] Perform frame extraction on the target video to obtain at least one target video frame;

[0035] For each target video frame, feature extraction is performed to obtain video frame features;

[0036] Multimodal features are obtained based on video cover features, keyword features, and video frame features of at least one target video frame.

[0037] In one possible implementation, multimodal features are obtained based on video cover features, keyword features, and video frame features of at least one target video frame, including:

[0038] The similarity between keyword features and features of each video frame is determined to obtain the first video weighted features;

[0039] Based on the video cover features and the first video weighted features, the second video weighted features of each target video frame are determined;

[0040] Based on the second video weighted features of each target video frame, determine the video content features;

[0041] Multimodal features are obtained by fusing video cover features, keyword features, and video content features.

[0042] In one possible implementation, video cover features, keyword features, and video content features are fused to obtain multimodal features, including:

[0043] The video cover features, keyword features, and video content features are mapped to vectors of preset dimensions to obtain video cover features, keyword features, and video content features of preset dimensions.

[0044] The video cover features, keyword features, and video content features of preset dimensions are concatenated to obtain the concatenated features;

[0045] The splicing features are normalized to determine the weights of the video cover features, video cover features, and video content features respectively.

[0046] The weighted sum of the video cover features, video cover features, and video content features is determined based on their weights, resulting in multimodal features.

[0047] On the other hand, a video processing apparatus is provided, the apparatus comprising:

[0048] The first acquisition module is used to acquire the basic features of the target video; the basic features are determined based on at least one of the source object identifier, tag information and video length of the target video.

[0049] The first prediction module is used to predict the basic features using the first influence prediction model to obtain the basic feature influence of the target video.

[0050] The second acquisition module is used to acquire the multimodal features of the target video; the multimodal features are determined based on at least one of the keyword features, video cover features, and video content features of the target video.

[0051] The second prediction module is used to predict the multimodal features through the second influence prediction model to obtain the multimodal feature influence of the target video.

[0052] The determination module is used to determine the target prediction influence of the target video based on the influence of basic features and the influence of multimodal features.

[0053] In one possible implementation, a training module is also included, used for:

[0054] Acquire multiple sample videos;

[0055] Calculate the true impact of each sample video;

[0056] Obtain the basic features and multimodal features of each video sample;

[0057] Input the basic features of each sample into the first initial prediction model to obtain the corresponding prediction influence of each first sample.

[0058] The multimodal features of each sample are input into the second initial prediction model to obtain the corresponding prediction influence of each second sample.

[0059] Based on the predicted influence of each first sample and the predicted influence of each second sample, the influence of each sample is determined.

[0060] Based on the influence of each sample and the corresponding real influence, the first initial prediction model and the second initial prediction model are trained to obtain the first influence prediction model and the second influence prediction model.

[0061] In one possible implementation, the training module, when calculating the true influence of each sample video, is specifically used for:

[0062] For each sample video, obtain the unique user data for exposure and the unique user data for clicks.

[0063] Click-through rate is determined based on unique user click data and unique user impression data.

[0064] The true influence is determined based on unique user exposure data, unique user click data, and user click-through rate.

[0065] In one possible implementation, the training module, when determining true influence based on impression-based unique user data, click-based unique user data, and user click-through rate, is specifically used for:

[0066] Determine the first weight for unique user exposure data, the second weight for unique user click data, and the third weight for user click-through rate;

[0067] The true influence is obtained by weighting the unique user data for exposure, unique user data for clicks, and user click-through rate based on the first, second, and third weights.

[0068] In one possible implementation, the first acquisition module, when acquiring the basic features of the target video, specifically performs the following:

[0069] The source object identifier is converted into an object identifier feature vector, and the tag information is converted into a tag feature vector.

[0070] The video length is converted into an initial length vector, and the initial length vector is normalized to obtain the video length vector;

[0071] The basic features are obtained by concatenating the object identifier feature vector, the label feature vector, and the video length vector.

[0072] In one possible implementation, the second acquisition module, when acquiring the multimodal features of the target video, is specifically used for:

[0073] Obtain the video cover image of the target video, and extract features from the video cover image to obtain the video cover features;

[0074] Obtain the key text of the target video; the key text includes at least one of the target video's title and summary;

[0075] Extract key text features to obtain initial text features, then segment the initial text features to obtain keyword features;

[0076] Perform frame extraction on the target video to obtain at least one target video frame;

[0077] For each target video frame, feature extraction is performed to obtain video frame features;

[0078] Multimodal features are obtained based on video cover features, keyword features, and video frame features of at least one target video frame.

[0079] In one possible implementation, when the second acquisition module acquires multimodal features based on video cover features, keyword features, and video frame features of at least one target video frame, it is specifically used for:

[0080] The similarity between keyword features and features of each video frame is determined to obtain the first video weighted features;

[0081] Based on the video cover features and the first video weighted features, the second video weighted features of each target video frame are determined;

[0082] Based on the second video weighted features of each target video frame, determine the video content features;

[0083] Multimodal features are obtained by fusing video cover features, keyword features, and video content features.

[0084] In one possible implementation, when the second acquisition module fuses the video cover features, keyword features, and video content features to obtain multimodal features, it is specifically used for:

[0085] The video cover features, keyword features, and video content features are mapped to vectors of preset dimensions to obtain video cover features, keyword features, and video content features of preset dimensions.

[0086] The video cover features, keyword features, and video content features of preset dimensions are concatenated to obtain the concatenated features;

[0087] The splicing features are normalized to determine the weights of the video cover features, video cover features, and video content features respectively.

[0088] The weighted sum of the video cover features, video cover features, and video content features is determined based on their weights, resulting in multimodal features.

[0089] On the other hand, a computer device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the video processing method described above.

[0090] On the other hand, a computer-readable storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the video processing method described above.

[0091] On the other hand, a computer program product is provided, including a computer program that, when executed by a processor, implements the video processing method described above.

[0092] The beneficial effects of the technical solution provided in this application are:

[0093] This method not only determines the basic feature influence of a target video based on at least one of the source object identifier, tag information, and video length, but also determines the multimodal feature influence of the target video through at least one of the keyword features, video cover features, and video content features. Then, based on the basic feature influence and the multimodal feature influence, the target predicted influence of the target video is determined. This allows for an objective evaluation of the target video from multiple perspectives, thereby effectively improving the accuracy of the influence prediction of the target video.

[0094] Furthermore, by using a cumulative attention mechanism to determine the weighted features of the second video, important frame data in the target video can be effectively filtered out, thereby more accurately analyzing the influence of the target video.

[0095] Furthermore, by using an adaptive weighting calculation method, weights can be adjusted based on video cover features, video cover features, and video content features, thereby further improving the accuracy of influence determination. Attached Figure Description

[0096] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0097] Figure 1a This is a diagram illustrating video ranking and recommendation based on influence in an example.

[0098] Figure 1b A schematic diagram illustrating the implementation environment of a video processing method provided in this application embodiment;

[0099] Figure 2 A flowchart illustrating a video processing method provided in an embodiment of this application;

[0100] Figure 3 A schematic diagram illustrating the video processing scheme provided as an example in this application;

[0101] Figure 4 A schematic flowchart illustrating a model training method provided in an embodiment of this application;

[0102] Figure 5 A schematic diagram illustrating the video processing scheme provided as an example in this application;

[0103] Figure 6 This is a schematic diagram of the structure of a video processing apparatus provided in an embodiment of this application;

[0104] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0105] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0106] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” indicates implementation as “A,” or implementation as “A,” or implementation as “A and B.”

[0107] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0108] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0109] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0110] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, and inductive learning.

[0111] Video influence represents the quality level of a video; a higher video influence value indicates higher video quality. Videos with high influence can increase view counts, and predicting video influence in advance plays an important role in the field of internet data mining.

[0112] like Figure 1a As shown, taking a game video published on the XX video platform as an example, influence analysis aims to score the influence of video content data. The game video shown in the figure has an influence score of 7. The platform can sort and recommend videos based on their influence level, as indicated by the bottom right ranking of each video to be recommended in the figure. Influence scores are used to increase the exposure of high-quality videos and improve the user experience of the video platform.

[0113] Figure 1b This is a schematic diagram of the implementation environment of a video processing method provided in an embodiment of the present invention. See also... Figure 1bThe implementation environment includes: terminal device 101 and server 102.

[0114] Specifically, server 102 obtains multiple target videos to be played from terminal device 101. For each target video, it obtains the basic features of the target video; predicts the basic features using a first influence prediction model to obtain the basic feature influence of the target video; obtains the multimodal features of the target video; predicts the multimodal features using a second influence prediction model to obtain the multimodal feature influence of the target video; and determines the target predicted influence of the target video based on the basic feature influence and the multimodal feature influence. Server 102 returns the target predicted influence corresponding to each target video to terminal device 101, and terminal device 101 can sort, recommend, and play the target videos according to the target predicted influence corresponding to each target video.

[0115] Understandable, Figure 1b This describes an example application scenario and does not limit the application scenario of the video processing method of this application. In the above scenario, the server determines the target prediction influence of the target video. In other scenarios, the terminal device 101 can also directly determine the target prediction influence of the target video. This application embodiment does not limit this.

[0116] Those skilled in the art will understand that a server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Terminals can include, but are not limited to, smartphones (such as Android phones, iOS phones, etc.), tablets, laptops, smart voice interaction devices, digital broadcast receivers, MIDs (Mobile Internet Devices), PDAs (Personal Digital Assistants), desktop computers, smart home appliances, in-vehicle terminals (such as in-vehicle navigation terminals, in-vehicle computers, etc.), smart speakers, smartwatches, and aircraft. Terminals and servers can be directly or indirectly connected via wired or wireless communication, but are not limited thereto. Embodiments of this invention can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving. Specific applications can be determined based on actual application needs and are not limited here.

[0117] Figure 2This is a flowchart illustrating a video processing method provided in an embodiment of this application. The method can be executed by a computer device. Figure 2 As shown, the method may include the following steps:

[0118] Step 201: Obtain the basic features of the target video.

[0119] The basic features are determined based on at least one of the source object identifier, tag information, and video length of the target video.

[0120] The source object identifier of the target video can refer to the name of the object that uploaded the target video.

[0121] Specifically, the tag information can be determined by the video category to which the target video is uploaded, or it can be included in the target video during the upload process.

[0122] The specific process for obtaining the basic features of the target video will be described in more detail below.

[0123] Step 202: Predict the basic features using the first influence prediction model to obtain the basic feature influence of the target video.

[0124] Among them, the influence of basic features can be the influence predicted for the basic features of the target video; influence can represent the video quality level, and a higher video influence value means a higher video quality.

[0125] The first influence prediction model can be pre-trained. By inputting basic features into the first influence prediction model, the basic feature influence can be output. The specific training process of the first influence prediction model will be described in detail below.

[0126] Step 203: Obtain the multimodal features of the target video.

[0127] Among them, the multimodal features are determined based on at least one of the target video's keyword features, video cover features, and video content features.

[0128] In the specific implementation process, the keyword features, video cover features, and video content features of the target video can be fused to obtain the multimodal features of the target video. The specific process of obtaining multimodal features will be further elaborated below.

[0129] Step S204: The multimodal features are predicted using the second influence prediction model to obtain the multimodal feature influence of the target video.

[0130] Among them, the multimodal feature influence can be the influence predicted by the multimodal features of the target video.

[0131] The second influence prediction model can also be pre-trained. By inputting multimodal features into the second influence prediction model, the multimodal feature influence can be output. The specific training process for the second influence prediction model will be described in detail below.

[0132] Step S205: Determine the target prediction influence of the target video based on the influence of basic features and the influence of multimodal features.

[0133] Specifically, the influence of basic features and the influence of multimodal features can be fused to obtain the target prediction influence.

[0134] In some implementations, the influence of basic features and the influence of multimodal features can be directly added together to obtain the target prediction influence.

[0135] like Figure 3 As shown in the example, the basic feature influence is first predicted for the target video to obtain the basic feature influence; then, the multimodal feature influence is predicted for the target video, where the multimodal features may include the title and summary, the cover image of the target video and the video content of the target video to obtain the multimodal feature influence; then, the multimodal feature influence and the basic feature influence are fused to obtain the target predicted influence, which is the overall video influence shown in the figure.

[0136] In the above implementation, the basic feature influence of the target video is determined not only based on at least one of the source object identifier, tag information and video length, but also based on at least one of the keyword features, video cover features and video content features. Then, the target predicted influence of the target video is determined based on the basic feature influence and the multimodal feature influence. This allows for an objective evaluation of the target video from multiple perspectives, thereby effectively improving the accuracy of the influence prediction of the target video.

[0137] The process of obtaining the first influence prediction model and the second influence prediction model will be further explained below with reference to the accompanying drawings and specific embodiments.

[0138] In some possible implementations, such as Figure 4 As shown, the first influence prediction model and the second influence prediction model can be trained in the following way:

[0139] Step S401: Obtain multiple sample videos.

[0140] Among them, the sample videos can be those that have already been uploaded and played, and whose exposure data and click data can be used to determine their true influence.

[0141] Step S402: Calculate the true influence of each sample video.

[0142] Specifically, the true impact of a sample video can be calculated based on the number of unique users who viewed it and the number of unique users who clicked on it. The calculation process for true impact will be explained in more detail below.

[0143] Step S403: Obtain the basic features and multimodal features of each sample video.

[0144] Among them, the basic features of the sample can be at least one of the source object identifier, tag information and video length of the sample video; the multimodal features of the sample can be at least one of the keyword features, video cover features and video content features of the sample video.

[0145] In the specific implementation process, the source object identifier of the sample video can be converted into the corresponding object identifier feature vector, the label information of the sample video can be converted into the label feature vector, the video length of the sample video can be converted into the initial length vector, and the initial length vector can be normalized to obtain the video length vector of the sample video; finally, the object identifier feature vector, label feature vector and video length vector of the sample video are concatenated to obtain the basic features of the sample video.

[0146] In practice, the multimodal features of a sample video can be obtained based on the video cover features, keyword features, and video frame features of the sample video.

[0147] Specifically, the process of obtaining the basic features and multimodal features of the sample video can be the same as the process of obtaining the basic features and multimodal features of the target video. The process of obtaining the basic features and multimodal features of the target video will be described in more detail below.

[0148] Step S404: Input the basic features of each sample into the first initial prediction model to obtain the corresponding prediction influence of each first sample; input the multimodal features of each sample into the second initial prediction model to obtain the corresponding prediction influence of each second sample.

[0149] Specifically, the first and second initial prediction models can be the same type of model, but with different parameters.

[0150] The first and second initial prediction models can be of different types, but both are used to predict the influence of input features, namely the basic features of the sample and the multimodal features of the sample.

[0151] Step S405: Based on the predicted influence of each first sample and the predicted influence of each second sample, determine the influence of each corresponding sample.

[0152] Specifically, for each sample video, the first sample predicted influence and the second sample predicted influence of that sample video can be fused to obtain the sample influence of that sample video.

[0153] In some possible implementations, the first and second sample prediction influences of the sample video can be directly added together to obtain the sample influence of the sample video.

[0154] Step S406: Based on the influence of each sample and the corresponding real influence, train the first initial prediction model and the second initial prediction model to obtain the first influence prediction model and the second influence prediction model.

[0155] Specifically, based on the sample influence and corresponding real influence of each sample video, the loss value for each sample video can be calculated. Based on the loss value of each sample video, the total loss for each sample video can be calculated. The parameters of the first initial prediction model and the second initial prediction model can be adjusted until the training termination condition is met, such as the total loss being less than a preset threshold, the total loss converging, or the number of training iterations reaching a preset number, thus obtaining the first influence prediction model and the second influence prediction model.

[0156] The process of calculating the true impact will be further explained below with reference to the accompanying drawings and embodiments.

[0157] In the specific implementation process, step S402, calculating the true influence of each sample video, may include:

[0158] (1) For each sample video, obtain the exposure unique user data and click unique user data of the sample video.

[0159] Among them, the unique visitor (UV) data and the unique visitor (UV) data can be data within a preset number of days after the sample video is launched, such as the UV data of exposure and the UV data of clicks within 7 days after launch.

[0160] (2) Determine the click-through rate based on click-based unique user data and impression-based unique user data.

[0161] Specifically, the ratio of click UV data to exposure UV data can be set as the user click-through rate.

[0162] (3) Determine the true influence based on exposure unique user data, click unique user data and user click-through rate.

[0163] In some implementations, determining true influence based on unique user exposure data, unique user click data, and user click-through rate may include:

[0164] a. Determine the first weight of unique user exposure data, the second weight of unique user click data, and the third weight of user click-through rate;

[0165] b. Based on the first weight, second weight, and third weight, determine the weighted sum between the exposure unique user data, click unique user data, and user click-through rate to obtain the true influence.

[0166] Specifically, firstly, based on exposure UV data, click UV data, and the distribution of click UV data / exposure UV data, the value range corresponding to different influence levels is determined, and the corresponding influence level is calculated. Then, the influence levels of the three are weighted and summed to finally obtain the true influence of the sample video.

[0167] The video's influence level L is calculated as follows:

[0168] L=α×[f _d (Click on UV data) +f _b (UV exposure data)]+(1-α)×[f _r (Click on UV data / exposure UV data)] (1)

[0169] In the formula, α is the weight; f _d (), f _b () and f _r () is a function for calculating the level of influence.

[0170] In one example, α can be set to 0.2, and the mapping relationship between the numerical value and the level of influence can be shown in Table 1 below.

[0171]

[0172] As shown in the table above, different exposure UV data, click UV data, and user click-through rate correspond to different levels of influence. The true influence of the sample video can be calculated based on the pre-set correspondence.

[0173] The above embodiments illustrate the specific process of determining the true influence of sample videos. The following will further illustrate the specific process of obtaining the basic features of target videos in conjunction with the embodiments.

[0174] In some possible implementations, step S201, acquiring the basic features of the target video, may include:

[0175] (1) Convert the source object identifier into an object identifier feature vector and the tag information into a tag feature vector;

[0176] (2) Convert the video length into an initial length vector and normalize the initial length vector to obtain the video length vector;

[0177] (3) The object identification feature vector, the tag feature vector and the video length vector are concatenated to obtain the basic features.

[0178] Specifically, the source object identifier can be treated as discrete information and transformed into an object identifier feature vector with a first preset dimension. For example, the source object identifier can be transformed into a 4-dimensional vector; the video length can be set as a continuous feature, and the initial length vector can be normalized to between 0 and 1 to obtain the video length vector; then the object identifier feature vector, the tag feature vector, and the video length vector are concatenated.

[0179] The above embodiments illustrate the specific process of obtaining the basic features of the target video. The following will further illustrate the specific process of obtaining the multimodal features of the target video with reference to the embodiments.

[0180] In some possible implementations, step 203, acquiring the multimodal features of the target video, may include:

[0181] (1) Obtain the video cover image of the target video and extract the video cover features from the video cover image.

[0182] Specifically, you can first obtain the initial cover image of the target video, then scale the initial cover image to a preset size to obtain the video cover image.

[0183] For example, the initial cover image can be scaled to 224×224 to obtain the video cover image.

[0184] In practice, the video cover image can be input into a neural network to extract video cover features.

[0185] In some possible implementations, the video cover image can be input into a deep residual network (ResNet), and the output of the last fully connected layer of the ResNet can be extracted to obtain a 2048-dimensional feature vector, which is the video cover feature.

[0186] (2) Obtain the key text of the target video.

[0187] The key text includes at least one of the title and summary of the target video.

[0188] (3) Extract the features of key texts to obtain initial text features, and segment the initial text features to obtain keyword features.

[0189] In some possible implementations, the Bidirectional Encoder Representations from Transformers (BERT) model can be used to extract features from the video title and summary to obtain initial text features. Then, the initial text features can be segmented to obtain 768-dimensional keyword features.

[0190] (4) Perform frame extraction on the target video to obtain at least one target video frame.

[0191] Specifically, a target video frame can be extracted at preset time intervals.

[0192] For example, one target video frame can be retained every second.

[0193] (5) For each target video frame, feature extraction is performed on the target video frame to obtain video frame features.

[0194] In some possible implementations, the same extraction method as for video cover features can be used: input the target video frame into ResNet, extract the output of the last fully connected layer of ResNet, and thus obtain the video frame features.

[0195] (6) Obtain multimodal features based on video cover features, keyword features and video frame features of at least one target video frame.

[0196] In some possible implementations, obtaining multimodal features based on video cover features, keyword features, and video frame features of at least one target video frame may include:

[0197] a. Determine the similarity between keyword features and features of each video frame to obtain the first video weighted features.

[0198] Specifically, the similarity between keyword features and features of each video frame is calculated to obtain the first video-weighted features based on text:

[0199] out1 = f(v) t v vid (2)

[0200] In the formula, out1 represents the first video weighted feature; v t Indicates keyword features; v vid f represents the video frame features; f() represents the input features of the attention mechanism.

[0201] b. Based on the video cover features and the first video weighted features, determine the second video weighted features for each target video frame.

[0202] Specifically, the weighted features of the second video can be calculated based on the following formula:

[0203] out2 = f(v) i ,out1) (3)

[0204] In the formula, out2 represents the second video weighted feature; v i represents the video cover feature; out1 represents the weighted feature of the first video; f() represents the input feature of the attention mechanism.

[0205] c. Determine video content features based on the second video weighted features of each target video frame.

[0206] Specifically, the second video weighted features of each target video frame are summed to obtain the video content features.

[0207] v v =sum(out2)∈R 2048 (4)

[0208] In the formula, out2 represents the second video weighted feature of each target video frame; v v This represents the characteristics of the video content; the sum() function represents the function that sums the input vector.

[0209] In one example, taking the attention mechanism used in formula (2) as an example, the single-step attention mechanism is formally represented as f(x,y), with input x = v t y = v vid First, map x and y to the same dimension 512 to obtain x. 1 ∈R 512 ,y 1 ∈n*R 512 Then x 1 Extended n times, we get x 2 ∈n*R 512 x is calculated using the dot product. 2 Similarity to y: e = x 2 ⊙y 1 ,e∈R n*512 By using a linear transformation, e is mapped to one dimension, resulting in e²∈R. n Normalize e² using the softmax function: a i =e2i / sum(e2), where each dimension a in a is the final value. i This represents the importance of each frame in y calculated from x. Using a to weight y, we obtain the output vector of this attention mechanism: out1 = a⊙y∈R n*2048 .

[0210] d. Integrate video cover features, keyword features, and video content features to obtain multimodal features.

[0211] In the above embodiments, by using a cumulative attention mechanism to determine the second video weighted features, important frame data in the target video can be effectively filtered out, thereby more accurately analyzing the influence of the target video.

[0212] In some possible implementations, video cover features, keyword features, and video content features are fused to obtain multimodal features, which may include:

[0213] The video cover features, keyword features, and video content features are mapped to vectors of preset dimensions to obtain video cover features, keyword features, and video content features of preset dimensions.

[0214] The video cover features, keyword features, and video content features of preset dimensions are concatenated to obtain the concatenated features;

[0215] The splicing features are normalized to determine the weights of the video cover features, video cover features, and video content features respectively.

[0216] The weighted sum of the video cover features, video cover features, and video content features is determined based on their weights, resulting in multimodal features.

[0217] In one example, the previously obtained video cover feature v is first... i ∈R 2048 Keyword features v t ∈R 768 Video content features v v ∈R 2048 Each is mapped to 1-dimensional (with v) using a multilayer perceptron (MLP). i For example, you only need to use a 2048×1 matrix in the network, along with v i Multiplication, i.e., v i (Convert to a 1D vector). Concatenate the three resulting 1D vectors to obtain a 3D vector. Normalize this vector using a softmax operation to obtain weight values ​​for different features. Use these weight values ​​to... i v t v v We perform weighted summation to obtain multimodal features, and then calculate the influence of the multimodal features.

[0218] In the above embodiments, by using an adaptive weighting calculation method, weights can be adjusted based on video cover features, video cover features, and video content features, thereby further improving the accuracy of influence determination.

[0219] To more clearly illustrate the video processing method of this application, the following will further explain the video processing method of this application with examples.

[0220] like Figure 5 As shown, in one example, the video processing method of this application may include the following steps:

[0221] Obtain the target video's video length, source object identifier, and tag information (i.e., the tags shown in the image);

[0222] Extract the basic features of the target video using the xDeepFM model (a feature extraction model);

[0223] The basic features are predicted by the first influence prediction model to obtain the basic feature influence of the target video.

[0224] Obtain the video cover image of the target video, as shown in the figure. Extract the video cover features of the target video using ResNet, as shown in the figure, namely the image features v. i ;

[0225] Obtain the title and summary of the target video, and extract keyword features from the target video using BERT, i.e., the text features v shown in the figure. t ;

[0226] Frame extraction is performed on the target video to obtain at least one target video frame; features are extracted from each target video frame using ResNet to obtain video frame features of at least one target video frame.

[0227] The first video weighted feature is obtained based on the keyword features and the features of each video frame, which is attention 1 (Attn-1) in the cumulative attention shown in the figure;

[0228] Based on the video cover features and the first video weighted features, the second video weighted features of each target video frame are determined, which is Attention 2 (Attn-2) in the cumulative attention shown in the figure;

[0229] The video content features, i.e., the video features v shown in the figure, are determined by using the second video weighted features based on each target video frame. v ;

[0230] The video cover feature (i.e., the image feature v shown in the figure) is obtained through MLP. i Keyword features (i.e., text features v shown in the figure)t ) and video content features (i.e., the video features v shown in the figure) v These are mapped to vectors of preset dimensions to obtain video cover features, keyword features, and video content features of preset dimensions.

[0231] The video cover features, keyword features, and video content features of preset dimensions are concatenated to obtain the concatenated features;

[0232] The spliced ​​features are normalized using softmax to determine the weights of the video cover feature, video cover feature, and video content feature respectively.

[0233] The weighted sum of the video cover features, video cover features, and video content features is determined based on their weights to obtain the multimodal features.

[0234] The influence of multimodal features on the target video is obtained by predicting the multimodal features using the second influence prediction model.

[0235] The target predicted influence of the target video is determined based on the influence of basic features and the influence of multimodal features, i.e., the predicted value of the video influence level shown in the figure.

[0236] The video processing method described above not only determines the basic feature influence of the target video based on at least one of the source object identifier, tag information, and video length, but also determines the multimodal feature influence of the target video through at least one of the keyword features, video cover features, and video content features. Then, based on the basic feature influence and the multimodal feature influence, the target predicted influence of the target video is determined. This allows for an objective evaluation of the target video from multiple perspectives, thereby effectively improving the accuracy of the influence prediction of the target video.

[0237] Furthermore, by using a cumulative attention mechanism to determine the weighted features of the second video, important frame data in the target video can be effectively filtered out, thereby more accurately analyzing the influence of the target video.

[0238] Furthermore, by using an adaptive weighting calculation method, weights can be adjusted based on video cover features, video cover features, and video content features, thereby further improving the accuracy of influence determination.

[0239] Figure 6 This is a schematic diagram of the structure of a video processing apparatus provided in an embodiment of this application. Figure 6 As shown, the device includes:

[0240] The first acquisition module 601 is used to acquire the basic features of the target video; the basic features are determined based on at least one of the source object identifier, tag information and video length of the target video;

[0241] The first prediction module 602 is used to predict the basic features through the first influence prediction model to obtain the basic feature influence of the target video.

[0242] The second acquisition module 603 is used to acquire the multimodal features of the target video; the multimodal features are determined based on at least one of the keyword features, video cover features, and video content features of the target video.

[0243] The second prediction module 604 is used to predict the multimodal features through the second influence prediction model to obtain the multimodal feature influence of the target video.

[0244] The determination module 605 is used to determine the target prediction influence of the target video based on the influence of basic features and the influence of multimodal features.

[0245] In one possible implementation, a training module is also included, used for:

[0246] Acquire multiple sample videos;

[0247] Calculate the true impact of each sample video;

[0248] Obtain the basic features and multimodal features of each video sample;

[0249] Input the basic features of each sample into the first initial prediction model to obtain the corresponding prediction influence of each first sample.

[0250] The multimodal features of each sample are input into the second initial prediction model to obtain the corresponding prediction influence of each second sample.

[0251] Based on the predicted influence of each first sample and the predicted influence of each second sample, the influence of each sample is determined.

[0252] Based on the influence of each sample and the corresponding real influence, the first initial prediction model and the second initial prediction model are trained to obtain the first influence prediction model and the second influence prediction model.

[0253] In one possible implementation, the training module, when calculating the true influence of each sample video, is specifically used for:

[0254] For each sample video, obtain the unique user data for exposure and the unique user data for clicks.

[0255] Click-through rate is determined based on unique user click data and unique user impression data.

[0256] The true influence is determined based on unique user exposure data, unique user click data, and user click-through rate.

[0257] In one possible implementation, the training module, when determining true influence based on impression-based unique user data, click-based unique user data, and user click-through rate, is specifically used for:

[0258] Determine the first weight for unique user exposure data, the second weight for unique user click data, and the third weight for user click-through rate;

[0259] The true influence is obtained by weighting the unique user data for exposure, unique user data for clicks, and user click-through rate based on the first, second, and third weights.

[0260] In one possible implementation, the first acquisition module 601, when acquiring the basic features of the target video, is specifically used for:

[0261] The source object identifier is converted into an object identifier feature vector, and the tag information is converted into a tag feature vector.

[0262] The video length is converted into an initial length vector, and the initial length vector is normalized to obtain the video length vector;

[0263] The basic features are obtained by concatenating the object identifier feature vector, the label feature vector, and the video length vector.

[0264] In one possible implementation, the second acquisition module 603, when acquiring the multimodal features of the target video, is specifically used for:

[0265] Obtain the video cover image of the target video, and extract features from the video cover image to obtain the video cover features;

[0266] Obtain the key text of the target video; the key text includes at least one of the target video's title and summary;

[0267] Extract key text features to obtain initial text features, then segment the initial text features to obtain keyword features;

[0268] Perform frame extraction on the target video to obtain at least one target video frame;

[0269] For each target video frame, feature extraction is performed to obtain video frame features;

[0270] Multimodal features are obtained based on video cover features, keyword features, and video frame features of at least one target video frame.

[0271] In one possible implementation, when the second acquisition module 603 acquires multimodal features based on video cover features, keyword features, and video frame features of at least one target video frame, it is specifically used for:

[0272] The similarity between keyword features and features of each video frame is determined to obtain the first video weighted features;

[0273] Based on the video cover features and the first video weighted features, the second video weighted features of each target video frame are determined;

[0274] Based on the second video weighted features of each target video frame, determine the video content features;

[0275] Multimodal features are obtained by fusing video cover features, keyword features, and video content features.

[0276] In one possible implementation, when the second acquisition module 603 fuses the video cover features, keyword features, and video content features to obtain multimodal features, it is specifically used for:

[0277] The video cover features, keyword features, and video content features are mapped to vectors of preset dimensions to obtain video cover features, keyword features, and video content features of preset dimensions.

[0278] The video cover features, keyword features, and video content features of preset dimensions are concatenated to obtain the concatenated features;

[0279] The splicing features are normalized to determine the weights of the video cover features, video cover features, and video content features respectively.

[0280] The weighted sum of the video cover features, video cover features, and video content features is determined based on their weights, resulting in multimodal features.

[0281] The aforementioned video processing device not only determines the basic feature influence of the target video based on at least one of the source object identifier, tag information, and video length, but also determines the multimodal feature influence of the target video through at least one of the keyword features, video cover features, and video content features. Then, based on the basic feature influence and the multimodal feature influence, it determines the target predicted influence of the target video. This allows for an objective evaluation of the target video from multiple perspectives, thereby effectively improving the accuracy of the influence prediction of the target video.

[0282] Furthermore, by using a cumulative attention mechanism to determine the weighted features of the second video, important frame data in the target video can be effectively filtered out, thereby more accurately analyzing the influence of the target video.

[0283] Furthermore, by using an adaptive weighting calculation method, weights can be adjusted based on video cover features, video cover features, and video content features, thereby further improving the accuracy of influence determination.

[0284] The video processing apparatus of this embodiment can execute the video processing method shown in the above embodiments of this application, and its implementation principle is similar, so it will not be described again here.

[0285] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. For example... Figure 7 As shown, the computer device includes: a memory and a processor; at least one program, stored in the memory, which, when executed by the processor, can achieve the following compared to the prior art:

[0286] This method not only determines the basic feature influence of a target video based on at least one of the source object identifier, tag information, and video length, but also determines the multimodal feature influence of the target video through at least one of the keyword features, video cover features, and video content features. Then, based on the basic feature influence and the multimodal feature influence, the target predicted influence of the target video is determined. This allows for an objective evaluation of the target video from multiple perspectives, thereby effectively improving the accuracy of the influence prediction of the target video.

[0287] Furthermore, by using a cumulative attention mechanism to determine the weighted features of the second video, important frame data in the target video can be effectively filtered out, thereby more accurately analyzing the influence of the target video.

[0288] Furthermore, by using an adaptive weighting calculation method, weights can be adjusted based on video cover features, video cover features, and video content features, thereby further improving the accuracy of influence determination.

[0289] In one alternative embodiment, a computer device is provided, such as Figure 7 As shown, Figure 7The computer device 700 shown includes a processor 701 and a memory 703. The processor 701 and the memory 703 are connected, for example, via a bus 702. Optionally, the computer device 700 may further include a transceiver 704, which can be used for data interaction between the computer device and other computer devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 704 is not limited to one type, and the structure of the computer device 700 does not constitute a limitation on the embodiments of this application.

[0290] Processor 701 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 701 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0291] Bus 702 may include a pathway for transmitting information between the aforementioned components. Bus 702 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 702 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0292] The memory 703 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0293] The memory 703 stores application code (computer program) that executes the solution of this application, and its execution is controlled by the processor 701. The processor 701 executes the application code stored in the memory 703 to implement the content shown in the foregoing method embodiments.

[0294] Computer equipment includes, but is not limited to: virtualized computer equipment, virtual machines, servers, service clusters, and user terminals.

[0295] This application provides a computer-readable storage medium storing a computer program that, when run on a computer, enables the computer to execute the corresponding content of the video processing method described in the foregoing method embodiments.

[0296] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the video processing method described above.

[0297] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0298] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0299] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0300] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0301] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0302] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0303] The modules described in the embodiments of this disclosure can be implemented in software or in hardware. The names of the modules are not necessarily limiting in certain circumstances; for example, the first acquisition module can also be described as "a module for acquiring basic features of a target video".

[0304] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A video processing method, characterized in that, The method includes: Obtain the basic features of the target video; the basic features are determined based on at least one of the target video's source object identifier, tag information, and video length; The basic features are predicted using the first influence prediction model to obtain the basic feature influence for the target video; The multimodal features of the target video are obtained by fusing the keyword features, video cover features, and video frame features of the target video. The multimodal features are predicted using a second influence prediction model to obtain the multimodal feature influence for the target video. The target prediction influence of the target video is determined based on the influence of the basic features and the influence of the multimodal features.

2. The video processing method according to claim 1, characterized in that, The first influence prediction model and the second influence prediction model were trained in the following manner: Acquire multiple sample videos; Calculate the true impact of each of the sample videos; Obtain the basic features and multimodal features of each sample video; Input the basic features of each sample into the first initial prediction model to obtain the corresponding prediction influence of each first sample. The multimodal features of each sample are input into the second initial prediction model to obtain the corresponding prediction influence of each second sample; Based on the predicted influence of each of the first samples and the predicted influence of each of the second samples, the influence of each corresponding sample is determined; Based on the influence of each sample and the corresponding real influence, the first initial prediction model and the second initial prediction model are trained to obtain the first influence prediction model and the second influence prediction model.

3. The video processing method according to claim 2, characterized in that, The calculation of the true influence of each of the sample videos includes: For each of the sample videos, obtain the exposure-independent user data and click-independent user data for that sample video; The user click-through rate is determined based on the aforementioned click-based unique user data and impression-based unique user data. The true influence is determined based on the exposure data of unique users, the click data of unique users, and the click-through rate.

4. The video processing method according to claim 3, characterized in that, The determination of the true influence based on the exposure-based unique user data, the click-based unique user data, and the user click-through rate includes: A first weight is determined for the exposure-specific user data, a second weight is determined for the click-specific user data, and a third weight is determined for the user click-through rate; The weighted sum of the exposure unique user data, the click unique user data, and the user click-through rate is determined based on the first weight, the second weight, and the third weight to obtain the true influence.

5. The video processing method according to claim 1, characterized in that, The basic features of the target video obtained include: The source object identifier is converted into an object identifier feature vector, and the tag information is converted into a tag feature vector; The video length is converted into an initial length vector, and the initial length vector is normalized to obtain the video length vector; The basic features are obtained by concatenating the object identifier feature vector, the tag feature vector, and the video length vector.

6. The video processing method according to claim 1, characterized in that, The acquisition of the multimodal features of the target video includes: Obtain the video cover image of the target video, and perform feature extraction on the video cover image to obtain the video cover features; Obtain key text from the target video; the key text includes at least one of the target video's title and summary; Extract features from the key text to obtain initial text features, and segment the initial text features to obtain the keyword features; Perform frame extraction on the target video to obtain at least one target video frame; For each target video frame, feature extraction is performed on the target video frame to obtain video frame features; The multimodal features are obtained based on the video cover features, the keyword features, and the video frame features of at least one target video frame.

7. The video processing method according to claim 6, characterized in that, The multimodal features are obtained based on the video cover features, the keyword features, and the video frame features of at least one target video frame, including: Determine the similarity between the keyword features and the features of each video frame to obtain the first video weighted features; Based on the video cover feature and the first video weighted feature, a second video weighted feature is determined for each of the target video frames; Based on the second video weighted features of each target video frame, video content features are determined; The multimodal features are obtained by fusing the video cover features, the keyword features, and the video content features.

8. The video processing method according to claim 7, characterized in that, The process of fusing the video cover feature, the keyword feature, and the video content feature to obtain the multimodal feature includes: The video cover feature, the keyword feature, and the video content feature are mapped to vectors of preset dimensions to obtain video cover features, keyword features, and video content features of preset dimensions. The video cover features, keyword features, and video content features of the preset dimensions are concatenated to obtain the concatenated features; The splicing features are normalized to determine the weights of the video cover feature, the video cover feature, and the video content feature, respectively. The weighted sum of the video cover feature, the video cover feature, and the video content feature is determined based on their respective weights to obtain the multimodal feature.

9. A video processing apparatus, characterized in that, The device includes: The first acquisition module is used to acquire the basic features of the target video; the basic features are determined based on at least one of the source object identifier, tag information and video length of the target video; The first prediction module is used to predict the basic features using the first influence prediction model to obtain the basic feature influence of the target video. The second acquisition module is used to acquire the multimodal features of the target video; the multimodal features are obtained by fusing the keyword features, video cover features and video frame features of the target video; The second prediction module is used to predict the multimodal features using the second influence prediction model to obtain the multimodal feature influence of the target video. The determination module is used to determine the target prediction influence of the target video based on the basic feature influence and the multimodal feature influence.

10. The video processing apparatus according to claim 9, characterized in that, The first influence prediction model and the second influence prediction model were trained in the following manner: Acquire multiple sample videos; Calculate the true impact of each of the sample videos; Obtain the basic features and multimodal features of each sample video; Input the basic features of each sample into the first initial prediction model to obtain the corresponding prediction influence of each first sample. The multimodal features of each sample are input into the second initial prediction model to obtain the corresponding prediction influence of each second sample; Based on the predicted influence of each of the first samples and the predicted influence of each of the second samples, the influence of each corresponding sample is determined; Based on the influence of each sample and the corresponding real influence, the first initial prediction model and the second initial prediction model are trained to obtain the first influence prediction model and the second influence prediction model.

11. The video processing apparatus according to claim 10, characterized in that, The true impact of the sample videos was calculated using the following method: For each of the sample videos, obtain the exposure-independent user data and click-independent user data for that sample video; The user click-through rate is determined based on the aforementioned click-based unique user data and impression-based unique user data. The true influence is determined based on the exposure data of unique users, the click data of unique users, and the click-through rate.

12. The video processing apparatus according to claim 11, characterized in that, The true impact of the sample videos was calculated using the following method: A first weight is determined for the exposure-specific user data, a second weight is determined for the click-specific user data, and a third weight is determined for the user click-through rate; The weighted sum of the exposure unique user data, the click unique user data, and the user click-through rate is determined based on the first weight, the second weight, and the third weight to obtain the true influence.

13. The video processing apparatus according to claim 9, characterized in that, The first acquisition module is used for: The source object identifier is converted into an object identifier feature vector, and the tag information is converted into a tag feature vector; The video length is converted into an initial length vector, and the initial length vector is normalized to obtain the video length vector; The basic features are obtained by concatenating the object identifier feature vector, the tag feature vector, and the video length vector.

14. The video processing apparatus according to claim 9, characterized in that, The second acquisition module is used for: Obtain the video cover image of the target video, and perform feature extraction on the video cover image to obtain the video cover features; Obtain key text from the target video; the key text includes at least one of the target video's title and summary; Extract features from the key text to obtain initial text features, and segment the initial text features to obtain the keyword features; Perform frame extraction on the target video to obtain at least one target video frame; For each target video frame, feature extraction is performed on the target video frame to obtain video frame features; The multimodal features are obtained based on the video cover features, the keyword features, and the video frame features of at least one target video frame.

15. The video processing apparatus according to claim 14, characterized in that, The second acquisition module is used for: Determine the similarity between the keyword features and the features of each video frame to obtain the first video weighted features; Based on the video cover feature and the first video weighted feature, a second video weighted feature is determined for each of the target video frames; Based on the second video weighted features of each target video frame, video content features are determined; The multimodal features are obtained by fusing the video cover features, the keyword features, and the video content features.

16. The video processing apparatus according to claim 14, characterized in that, The second acquisition module is used for: The video cover feature, the keyword feature, and the video content feature are mapped to vectors of preset dimensions to obtain video cover features, keyword features, and video content features of preset dimensions. The video cover features, keyword features, and video content features of the preset dimensions are concatenated to obtain the concatenated features; The splicing features are normalized to determine the weights of the video cover feature, the video cover feature, and the video content feature, respectively. The weighted sum of the video cover feature, the video cover feature, and the video content feature is determined based on their respective weights to obtain the multimodal feature.

17. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the video processing method according to any one of claims 1 to 8.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the video processing method according to any one of claims 1 to 8.

19. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the video processing method according to any one of claims 1 to 8.

Images