Video cover recommendation method and device and storage medium
By obtaining and analyzing user video interaction information, combining video cover elements and semantic vectors, weighting the recommended video cover based on interest information, the problem of mismatch in video cover recommendations in the prior art is solved, and the accuracy and personalization of recommendations are improved.
Patent Information
- Application Number
- CN202310666165.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-06
- Publication Date
- 2024-12-06
AI Technical Summary
The existing video cover recommendation method is used to mismatch recommendations due to fixed video dimension selection in multi-user scenarios, which affects the accuracy of video cover recommendations.
By obtaining the video interaction information of the target object, determining its associated interest information, and vector representation of the image data in the preset cover, combining the target element and semantic vector, weighting based on interest information, a matching video cover is recommended.
It improves the adaptability of the video cover and the target object, improves the accuracy of video cover recommendations, and realizes personalized video cover recommendations.
Smart Images

Figure CN119094848A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a method, device and storage medium for recommending a video cover. Background Art
[0002] With the rapid development of Internet technology, people have higher and higher requirements for media content interaction. In the process of video media content interaction, video covers are often required to be displayed to encourage users to interact with videos.
[0003] Generally, the determination of a video cover can be performed by identifying the video content, thereby generating a video cover that is representative of the video content.
[0004] However, the image selection based on the video dimension is fixed, that is, the same cover image is displayed to different users. In scenarios with a large number of users, due to different user needs, using a fixed cover image may result in mismatches, affecting the accuracy of video cover recommendations. Summary of the invention
[0005] In view of this, the present application provides a method for recommending video covers, which can effectively improve the accuracy of video cover recommendations.
[0006] The first aspect of the present application provides a video cover recommendation method, which can be applied to a system or program including a video cover recommendation function in a terminal device, and specifically includes:
[0007] Acquire video interaction information corresponding to the target object, so as to determine interest information associated with the target object based on the video interaction information;
[0008] Performing vector representation on the image data in the preset cover associated with the target object to obtain a cover image semantic vector;
[0009] Determine a target element in the preset cover, so as to determine a cover image element vector corresponding to the preset cover according to the target element;
[0010] Obtaining a cover image target vector corresponding to the preset cover according to the cover image element vector and the cover image semantic vector;
[0011] Weighting the cover image target vector based on the interest information to obtain an object representation vector corresponding to the target object;
[0012] In response to the input of the candidate cover, a candidate cover vector corresponding to the candidate cover is determined, and a target cover recommended to the target object is determined based on the candidate cover vector and the object representation vector.
[0013] Optionally, in some possible implementations of the present application, acquiring video interaction information corresponding to the target object to determine interest information associated with the target object based on the video interaction information includes:
[0014] Acquire video interaction information corresponding to the target object, so as to determine an interest video corresponding to the target object based on an interaction action in the video interaction information;
[0015] Determine a video vector corresponding to each video in the video set based on the video interaction information;
[0016] Determine the cosine similarity between the videos according to the video vectors, so as to determine the interest set corresponding to the target object according to the cosine similarity;
[0017] Determining the interest level of the target object for each video based on the interest set, so as to determine a corresponding predicted video according to the interest level;
[0018] The predicted video is combined with the interest video to obtain interest information associated with the target object.
[0019] Optionally, in some possible implementations of the present application, obtaining the video interaction information corresponding to the target object to determine the interest video corresponding to the target object based on the interaction action in the video interaction information includes:
[0020] Obtaining video interaction information corresponding to the target object;
[0021] Extracting search records according to the interactive actions in the video interaction information;
[0022] Determine the search video information associated with the search record to obtain the interest video indicated by the search video information.
[0023] Optionally, in some possible implementations of the present application, the method further includes:
[0024] Obtaining object portrait information corresponding to the target object;
[0025] Determining a video tag of the target object associated with the video content according to the object portrait information;
[0026] The video content associated with the video tag is called to expand the interest video.
[0027] Optionally, in some possible implementations of the present application, calling the video content associated with the video tag to expand the interest video includes:
[0028] Calling the video content associated with the video tag to obtain the called video;
[0029] Acquire the interactive time period information corresponding to the video tag, and filter the called video based on the interactive time period information to obtain the extended video;
[0030] The interest video is expanded based on the expanded video.
[0031] Optionally, in some possible implementations of the present application, the vector representation of the image data in the preset cover associated with the target object to obtain the cover image semantic vector includes:
[0032] Acquire training data of the preset cover indication;
[0033] Inputting the training data into a prediction network for vector representation to obtain a prediction representation vector;
[0034] The predicted representation vector is transmitted to the target network for prediction to obtain an output representation vector, wherein the parameter weight information of the target network is an exponential moving average of the parameter weight information of the predicted network;
[0035] Determine loss information between the output representation vector and the predicted representation vector, the loss information comprising cosine similarity loss and mean square error loss;
[0036] Performing self-supervisory training on the prediction network according to the loss information;
[0037] Based on the trained prediction network, the image data in the preset cover associated with the target object is represented by vector to obtain a cover image semantic vector.
[0038] Optionally, in some possible implementations of the present application, determining a target element in the preset cover to determine a cover image element vector corresponding to the preset cover according to the target element includes:
[0039] Detecting the human face in the preset cover based on a target detection network to determine multiple target human faces indicated by the target element;
[0040] Extracting features of the target face according to a feature extraction network to obtain facial features;
[0041] The facial features corresponding to different target faces are averaged to obtain a cover image element vector corresponding to the preset cover.
[0042] Optionally, in some possible implementations of the present application, the detecting the face in the preset cover based on the target detection network to determine the multiple target faces indicated by the target element includes:
[0043] Detecting the face in the preset cover based on the target detection network to obtain a candidate face frame;
[0044] Screening the candidate face frames to obtain screened face frames;
[0045] Detecting key points of the face in the screened face frame to determine position information and confidence information of the face indication based on the key points of the face;
[0046] A plurality of target faces indicated by the target element are determined based on the position information and the confidence information.
[0047] Optionally, in some possible implementations of the present application, extracting features from the target face according to a feature extraction network to obtain facial features includes:
[0048] Get face training data;
[0049] Determining facial similarity information corresponding to training faces in the facial training data;
[0050] Configuring a face loss function based on the face similarity information so that similar faces in the training faces are closely spaced in the feature space, and dissimilar faces in the training faces are clearly spaced in the feature space;
[0051] Training a preset extraction network according to the face loss function to obtain the feature extraction network;
[0052] Feature extraction is performed on the target face according to the feature extraction network to obtain the facial features.
[0053] Optionally, in some possible implementations of the present application, averaging the facial features corresponding to different target faces to obtain a cover image element vector corresponding to the preset cover includes:
[0054] The facial features corresponding to different target faces are averaged to obtain the average features corresponding to the different target faces;
[0055] Obtain priority information corresponding to different target faces;
[0056] The average feature is weighted based on the priority information to obtain a cover image element vector corresponding to the preset cover.
[0057] Optionally, in some possible implementations of the present application, weighting the cover image target vector based on the interest information to obtain an object representation vector corresponding to the target object includes:
[0058] Determine, based on the video interaction information, a video interaction sequence of the video indicated by the interest information;
[0059] Determining time information corresponding to the video interaction sequence;
[0060] The cover image target vector is weighted based on the interest information and the time information to obtain an object representation vector corresponding to the target object.
[0061] Optionally, in some possible implementations of the present application, in response to the input of the candidate cover, determining a candidate cover vector corresponding to the candidate cover, and determining a target cover recommended to the target object according to the candidate cover vector and the object representation vector, includes:
[0062] In response to the target object's interactive operation on the video display interface, obtaining the video to be displayed;
[0063] Determining the candidate cover of the video configuration to be displayed;
[0064] Determine a candidate cover vector corresponding to the candidate cover, and obtain a recommendation score based on an inner product calculation between the candidate cover vector and the object representation vector;
[0065] The target cover recommended to the target object is determined based on the recommendation score.
[0066] A second aspect of the present application provides a video cover recommendation device, comprising: an acquisition unit, configured to acquire video interaction information corresponding to a target object, so as to determine interest information associated with the target object based on the video interaction information;
[0067] A processing unit, used for performing vector representation on the image data in the preset cover associated with the target object to obtain a semantic vector of the cover image;
[0068] The processing unit is further used to determine a target element in the preset cover, so as to determine a cover image element vector corresponding to the preset cover according to the target element;
[0069] The processing unit is further used to combine the cover image element vector with the cover image semantic vector to obtain a cover image target vector corresponding to the preset cover;
[0070] The processing unit is further used to weight the cover image target vector based on the interest information to obtain an object representation vector corresponding to the target object;
[0071] The recommendation unit is used to determine a candidate cover vector corresponding to the candidate cover in response to an input of a candidate cover, and determine a target cover recommended to the target object based on the candidate cover vector and the object representation vector.
[0072] Optionally, in some possible implementations of the present application, the acquisition unit is specifically used to acquire video interaction information corresponding to the target object, so as to determine the interest video corresponding to the target object based on the interaction action in the video interaction information;
[0073] The acquisition unit is specifically configured to determine a video vector corresponding to each video in the video set based on the video interaction information;
[0074] The acquisition unit is specifically used to determine the cosine similarity between videos according to the video vectors, so as to determine the interest set corresponding to the target object according to the cosine similarity;
[0075] The acquisition unit is specifically configured to determine the interest level of the target object for each video based on the interest set, so as to determine the corresponding predicted video according to the interest level;
[0076] The acquisition unit is specifically used to combine the predicted video with the interest video to obtain the interest information associated with the target object.
[0077] Optionally, in some possible implementations of the present application, the acquisition unit is specifically used to acquire video interaction information corresponding to the target object;
[0078] The acquisition unit is specifically configured to extract search records according to the interaction actions in the video interaction information;
[0079] The acquisition unit is specifically configured to determine the search video information associated with the search record to acquire the video of interest indicated by the search video information.
[0080] Optionally, in some possible implementations of the present application, the acquisition unit is specifically used to acquire object portrait information corresponding to the target object;
[0081] The acquisition unit is specifically used to determine the video tag of the target object associated with the video content according to the object portrait information;
[0082] The acquisition unit is specifically used to call the video content associated with the video tag to expand the interest video.
[0083] Optionally, in some possible implementations of the present application, the acquisition unit is specifically used to call the video content associated with the video tag to obtain the called video;
[0084] The acquisition unit is specifically used to acquire the interaction period information corresponding to the video tag, so as to filter the called video with extension based on the interaction period information;
[0085] The acquisition unit is specifically configured to expand the interest video based on the extended video.
[0086] Optionally, in some possible implementations of the present application, the processing unit is specifically used to obtain training data of the preset cover indication;
[0087] The processing unit is specifically used to input the training data into the prediction network for vector representation to obtain a prediction representation vector;
[0088] The processing unit is specifically used to transfer the predicted representation vector to the target network for prediction to obtain an output representation vector, wherein the parameter weight information of the target network is an exponential moving average of the parameter weight information of the predicted network;
[0089] The processing unit is specifically used to determine loss information between the output representation vector and the predicted representation vector, wherein the loss information includes cosine similarity loss and mean square error loss;
[0090] The processing unit is specifically configured to perform self-supervisory training on the prediction network according to the loss information;
[0091] The processing unit is specifically used to perform vector representation on the image data in the preset cover associated with the target object based on the trained prediction network to obtain a cover image semantic vector.
[0092] Optionally, in some possible implementations of the present application, the processing unit is specifically configured to detect a face in the preset cover based on a target detection network to determine a plurality of target faces indicated by the target element;
[0093] The processing unit is specifically used to extract features of the target face according to the feature extraction network to obtain facial features;
[0094] The processing unit is specifically used to average the facial features corresponding to different target faces to obtain the cover image element vector corresponding to the preset cover.
[0095] Optionally, in some possible implementations of the present application, the processing unit is specifically configured to detect a face in the preset cover based on the object detection network to obtain a candidate face frame;
[0096] The processing unit is specifically used to screen the candidate face frames to obtain screened face frames;
[0097] The processing unit is specifically used to detect the key points of the face in the screened face frame to determine the position information and confidence information of the face indication based on the key points of the face;
[0098] The processing unit is specifically configured to determine multiple target faces indicated by the target element based on the position information and the confidence information.
[0099] Optionally, in some possible implementations of the present application, the processing unit is specifically used to obtain face training data;
[0100] The processing unit is specifically used to determine the face similarity information corresponding to the training faces in the face training data;
[0101] The processing unit is specifically used to configure a face loss function based on the face similarity information, so that similar faces in the training faces are closely spaced in the feature space, and dissimilar faces in the training faces are clearly spaced in the feature space;
[0102] The processing unit is specifically used to train a preset extraction network according to the face loss function to obtain the feature extraction network;
[0103] The processing unit is specifically used to extract features of the target face according to the feature extraction network to obtain the facial features.
[0104] Optionally, in some possible implementations of the present application, the processing unit is specifically configured to average the facial features corresponding to different target faces to obtain average features corresponding to the different target faces;
[0105] The processing unit is specifically used to obtain priority information corresponding to different target faces;
[0106] The processing unit is specifically used to weight the average feature based on the priority information to obtain a cover image element vector corresponding to the preset cover.
[0107] Optionally, in some possible implementations of the present application, the processing unit is specifically configured to determine a video interaction sequence of the video indicated by the interest information based on the video interaction information;
[0108] The processing unit is specifically used to determine the time information corresponding to the video interaction sequence;
[0109] The processing unit is specifically used to weight the cover image target vector based on the interest information and the time information to obtain an object representation vector corresponding to the target object.
[0110] Optionally, in some possible implementations of the present application, the recommendation unit is specifically configured to obtain the video to be displayed in response to the interactive operation of the target object on the video display interface;
[0111] The recommendation unit is specifically used to determine the candidate cover of the video configuration to be displayed;
[0112] The recommendation unit is specifically used to determine the candidate cover vector corresponding to the candidate cover, and obtain a recommendation score by performing an inner product calculation on the candidate cover vector and the object representation vector;
[0113] The recommendation unit is specifically configured to determine the target cover to be recommended to the target object based on the recommendation score.
[0114] The third aspect of the present application provides a computer device, including a processor and a memory, the memory is used to store a computer program; the processor is used to execute the video cover recommendation method described in the first aspect or any one of the first aspects according to the instructions in the computer program.
[0115] The fourth aspect of the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is executed on a computer device, the computer device executes the method for recommending a video cover as described in the first aspect or any one of the first aspects.
[0116] According to one aspect of the present application, a computer program product is provided, including a computer program, wherein the computer program is stored in a computer-readable storage medium, and wherein when the computer program in the computer-readable storage medium is executed by a processor, the steps of the method for recommending a video cover as described in the first aspect or any one of the first aspects are implemented.
[0117] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:
[0118] By obtaining the video interaction information corresponding to the target object, the interest information associated with the target object is determined based on the video interaction information; then the image data in the preset cover associated with the target object is represented by a vector to obtain a cover image semantic vector; and the target element in the preset cover is determined to determine the cover image element vector corresponding to the preset cover according to the target element; the cover image element vector is further combined with the cover image semantic vector to obtain the cover image target vector corresponding to the preset cover; then the cover image target vector is weighted based on the interest information to obtain the object representation vector corresponding to the target object; and then in response to the input of the candidate cover, the candidate cover vector corresponding to the candidate cover is determined to determine the target cover recommended to the target object according to the inner product of the candidate cover vector and the object representation vector. Thus, the video cover recommendation process matching the object is realized. Since the interest information is determined by using the object's interaction information, and the object is targeted based on the interest information, the adaptability of the video cover to the target object is improved, and the accuracy of the video cover recommendation is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0119] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0120] Figure 1 A diagram of the network architecture that runs the video cover recommendation system;
[0121] Figure 2 A process architecture diagram for recommending a video cover provided in an embodiment of the present application;
[0122] Figure 3 A flowchart of a method for recommending a video cover provided in an embodiment of the present application;
[0123] Figure 4 A schematic diagram of a scenario of a method for recommending a video cover provided in an embodiment of the present application;
[0124] Figure 5 A schematic diagram of another method for recommending a video cover provided in an embodiment of the present application;
[0125] Figure 6 A flowchart of another method for recommending a video cover provided in an embodiment of the present application;
[0126] Figure 7 A schematic diagram of another method for recommending a video cover provided in an embodiment of the present application;
[0127] Figure 8 A schematic diagram of the structure of a video cover recommendation device provided in an embodiment of the present application;
[0128] Fig. 9 A schematic diagram of the structure of a terminal provided in an embodiment of the present application;
[0129] Fig.10 A schematic diagram of the structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION
[0130] The embodiment of the present application provides a method for recommending a video cover and a related device, which can be applied to a system or program in a terminal that includes a video cover recommendation function, by obtaining video interaction information corresponding to a target object, so as to determine interest information associated with the target object based on the video interaction information; then the image data in the preset cover associated with the target object is represented by a vector to obtain a cover image semantic vector; and the target element in the preset cover is determined to determine the cover image element vector corresponding to the preset cover according to the target element; the cover image element vector is further combined with the cover image semantic vector to obtain a cover image target vector corresponding to the preset cover; then the cover image target vector is weighted based on the interest information to obtain an object representation vector corresponding to the target object; and then in response to the input of the candidate cover, the candidate cover vector corresponding to the candidate cover is determined to determine the target cover recommended to the target object according to the inner product of the candidate cover vector and the object representation vector. Thus, a video cover recommendation process matching the object is realized, and since the interest information is determined by using the object's interaction information, and the object is targeted based on the interest information, the adaptability of the video cover to the target object is improved, and the accuracy of the video cover recommendation is improved.
[0131] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0132] It should be understood that the video cover recommendation method provided in the present application can be applied to a system or program in a terminal that includes a video cover recommendation function, such as a video platform. Specifically, the video cover recommendation system can be run on a system such as Figure 1 In the network architecture shown in Figure 1 As shown in FIG. 1 , it is a network architecture diagram of the video cover recommendation system. As can be seen from the figure, the video cover recommendation system can provide a video cover recommendation process with multiple information sources, that is, the interactive operation on the terminal side triggers the server to determine the video cover and provide feedback to the terminal; it can be understood that Figure 1 A variety of terminals are shown, and the terminal may be a computer device. In actual scenarios, there may be more or fewer types of terminals participating in the process of recommending video covers. The specific number and type depend on the actual scenario and are not limited here. In addition, Figure 1 One server is shown in the figure, but in an actual scenario, multiple servers may be involved, and the specific number of servers depends on the actual scenario.
[0133] In this embodiment, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a mobile phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, smart voice interaction device, smart home appliance, car terminal, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected by wired or wireless communication, and the terminal and the server can be connected to form a blockchain network, which is not limited in this application.
[0134] It can be understood that the above-mentioned video cover recommendation system can run on a terminal, can also run on a server, and can also run on a third-party device to provide video cover recommendations to obtain the recommendation processing results of the video cover of the information source; the specific video cover recommendation system can be run in the above-mentioned device in the form of a program, can also be run as a system component in the above-mentioned device, and can also be used as a cloud service program. This embodiment can be applied to cloud technology, audio and video, autonomous driving and other scenarios. The specific operation mode depends on the actual scenario and is not limited here.
[0135] With the rapid development of Internet technology, people have higher and higher requirements for media content interaction. In the process of video media content interaction, video covers are often required to be displayed to encourage users to interact with videos.
[0136] Generally, the determination of a video cover can be performed by identifying the video content, thereby generating a video cover that is representative of the video content.
[0137] However, the image selection based on the video dimension is fixed, that is, the same cover image is displayed to different users. In scenarios with a large number of users, due to different user needs, using a fixed cover image may result in mismatches, affecting the accuracy of video cover recommendations.
[0138] In order to solve the above problems, this application proposes a video cover recommendation method, which is applied to Figure 2 In the recommended process framework of the video cover shown, Figure 2 As shown, it is a process architecture diagram for recommending a video cover provided in an embodiment of the present application. The user interacts with the video through the terminal, triggering the server to mine the user's potential interests based on the user's viewing behavior, and then obtains the cover image representation through self-supervised training based on the stock video cover images, and then calculates the character representation appearing in the cover image. Finally, the user's representation vector is calculated based on the cover image representation, and the user representation vector and the inner product of the candidate cover image representation are scored, so as to select the cover image that best suits the user for display.
[0139] It is understandable that the method provided by the present application can be written as a program to serve as a processing logic in a hardware system, or as a video cover recommendation device, which implements the above processing logic in an integrated or external manner. As an implementation method, the video cover recommendation device obtains the video interaction information corresponding to the target object to determine the interest information associated with the target object based on the video interaction information; then the image data in the preset cover associated with the target object is represented by a vector to obtain a cover image semantic vector; and the target element in the preset cover is determined to determine the cover image element vector corresponding to the preset cover according to the target element; the cover image element vector is further combined with the cover image semantic vector to obtain the cover image target vector corresponding to the preset cover; then the cover image target vector is weighted based on the interest information to obtain the object representation vector corresponding to the target object; and then in response to the input of the candidate cover, the candidate cover vector corresponding to the candidate cover is determined to determine the target cover recommended to the target object according to the inner product of the candidate cover vector and the object representation vector. This realizes the process of recommending video covers that match the objects. Since the interest information is determined by using the interactive information of the objects, and the objects are targetedly recommended based on the interest information, the adaptability of the video cover to the target object is improved, and the accuracy of the video cover recommendation is improved.
[0140] The solution provided in the embodiments of the present application involves artificial intelligence computer vision technology, which is specifically described by the following embodiments:
[0141] Combined with the above process architecture, the following will introduce the recommended method for the video cover in this application. Please refer to Figure 3 , Figure 3A flowchart of a method for recommending a video cover provided in an embodiment of the present application. The method can be executed by a server or a terminal, or by both a server and a terminal. The embodiment of the present application includes at least the following steps:
[0142] 301. Obtain video interaction information corresponding to a target object to determine interest information associated with the target object based on the video interaction information.
[0143] In this embodiment, the target object may be a user, a terminal, or other object with an indicative identifier. In this embodiment, the user is used as an example for description and is not limited thereto.
[0144] In addition, the video interaction information is the information generated by the user performing interactive operations on the video, which may specifically include watching, clicking, searching, collecting, etc. This embodiment takes the watching operation as an example for explanation and is not limited thereto.
[0145] It should be noted that in the specific implementation of this application, related data such as user information and interaction information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.
[0146] It is understandable that the interest information associated with the target object is determined through video interaction information, that is, the content that the user is interested in is identified, so as to facilitate the subsequent targeted cover recommendation process; however, there is a disadvantage of only considering the user's viewing history to select pictures, which will limit the content that the user browses, that is, the user often sees a few familiar stars, that is, there is an "information cocoon" problem. Therefore, this embodiment should consider the user's past viewing history on the one hand, and explore new content that the user may be interested in on the other hand, so as to expand the interest information.
[0147] Specifically, the expansion process of interest information can be performed through collaborative filtering algorithms. Collaborative filtering algorithms are a type of recommendation system algorithm. The basic idea of collaborative filtering algorithms is to use historical behavior data of users (such as browsing records, purchase records, etc.) to predict the user's preference for future items, so as to recommend related items to the user. That is, recommendations are made by observing the similarity between videos. Assuming that the interests of videos that users like are similar to those of other similar videos, other videos similar to videos that a user already likes can be recommended to a user based on the similarity between videos.
[0148] For the processing process of the collaborative filtering algorithm, first obtain the video interaction information corresponding to the target object, and determine the interest video corresponding to the target object based on the interactive actions in the video interaction information; then determine the video vector corresponding to each video in the video set based on the video interaction information; specifically, the video vector wi can be defined, and the dimension of vector wi is the number of all users, and there are two cases: if user u watches video i, then w iu =1; user u has not watched video i, then w iu =0. Since the number of videos watched by users is limited, the video vector wi is extremely sparse. After defining the video vector, the similarity between videos can be calculated. Define the video similarity matrix S. Since ItemCF is also a type of recommendation algorithm based on object behavior data, the object behavior record can be used here to calculate the similarity of two videos. Specifically, the cosine similarity between videos can be determined based on the video vector, and the interest set corresponding to the target object can be determined based on the cosine similarity; that is, the cosine similarity is used to measure the similarity between video i and video i', and the formula is as follows:
[0149]
[0150] For determining the interest set, a similarity threshold can be set according to the similarity between videos, or the K videos most similar to the current video can be retained, which are recorded as set Ii.
[0151] Furthermore, the interest level of the target object for each video can be determined based on the interest set, so as to determine the corresponding predicted video according to the interest level; that is, the interest level of user u for a certain item i can be obtained based on the item set Ii similar to video i, and the formula is as follows:
[0152]
[0153] Among them, y ui′ It is an element in the user-video matrix Y. If user u has watched video i, then yui=1; if user u has not watched video i, then yui=0.
[0154] Then we can add the predicted result of ItemCF to the user's real user-video matrix to get the output of this module:
[0155]
[0156] in
[0157]
[0158] Among them, y ui′ is the element in the user-video matrix Y, y uiIt is an extended predicted video. By extending the potential interests, this embodiment takes into account both the user's viewing behavior and potential interests, thereby improving the comprehensiveness of the user's interests.
[0159] Optionally, in addition to determining the videos of interest through user viewing behavior, the search records can also be used, that is, first obtaining the video interaction information corresponding to the target object; then extracting the search records based on the interactive actions in the video interaction information; and then determining the search video information associated with the search records to obtain the videos of interest indicated by the search video information, thereby improving the comprehensiveness of the user's interest description.
[0160] In addition, the expansion of interest videos can also be done through object portraits, that is, first obtain the object portrait information corresponding to the target object; then determine the video tag associated with the video content of the target object (such as pet, star A, etc.) based on the object portrait information; then call the video content associated with the video tag to expand the interest video.
[0161] In a possible scenario, considering that users may have different interests at different time periods, for example, when working (working period), they prefer video covers with technical descriptions, while when resting (rest period), they prefer video covers with celebrity descriptions. Specifically, the video content associated with the video tag can be first called to obtain the called video; then the interactive period information corresponding to the video tag can be obtained to filter the called video with an extended video based on the interactive period information; and the interest video can be expanded based on the extended video, so as to achieve targeted interest descriptions for different time periods, perform targeted cover configurations for different time periods, and improve the accuracy of subsequent push.
[0162] 302. Perform vector representation on the image data in the preset cover associated with the target object to obtain a cover image semantic vector.
[0163] In this embodiment, in order to be able to select personalized cover images according to different users, the cover image of the video needs to be converted into a feature vector. The feature vector is a summary of the semantic information of the cover image, and the distance between the feature vectors reflects the similarity between the corresponding cover images.
[0164] Specifically, for the representation of the cover image content, a self-supervised image representation network can be used to learn useful representations from a large amount of unlabeled cover image data. The self-supervised network can be BYOL (Bootstrap Your Own Latent). This algorithm transforms unlabeled data into useful information by learning its own latent representation, thereby improving the generalization ability of the model. The processing process of the BYOL algorithm is as follows: Figure 4 As shown, Figure 4A scenario diagram of a method for recommending a video cover provided in an embodiment of the present application; that is, the prediction network is defined by a set of weights θ and consists of three stages: encoder fθ, projector gθ and predictor qθ. The target network has the same architecture as the prediction network, but uses different weights ξ, that is, the encoder of the target network is fξ, the projector is gξ and the predictor is sg. The target network provides a regression target for training the prediction network, and its parameter ξ is the exponential moving average of the online parameter θ. During the training process, for each input sample, the online network generates a representation vector and passes it to the target network for prediction. The output representation vector of the target network is compared with the true representation vector of the online network to calculate the loss function. The loss function includes two parts: one is the cosine similarity loss between the online network and the target network, and the other is the mean square error loss between the online network and the target network. After each batch, gradient descent is used to update the weights of the online network, and the exponential moving average is used to update the weights of the target network.
[0165] Therefore, for the processing of the BYOL algorithm, the training data indicated by the preset cover can be first obtained; then the training data can be input into the prediction network for vector representation to obtain a prediction representation vector; and the prediction representation vector can be passed to the target network for prediction to obtain an output representation vector, and the parameter weight information of the target network is the exponential moving average of the parameter weight information of the prediction network; then the loss information between the output representation vector and the prediction representation vector is determined, and the loss information includes cosine similarity loss and mean square error loss; and then the prediction network is self-supervisedly trained according to the loss information; thereby, the image data in the preset cover associated with the target object is vectorized based on the trained prediction network to obtain a cover image semantic vector.
[0166] In addition, the BYOL algorithm also uses some techniques to enhance the stability and effect of training, such as data enhancement, L2 normalization, and progressive training.
[0167] This embodiment uses two neural networks for self-supervised learning, training one network to predict the representation of the other network, and then using these predicted representations to update its own representation. The weights of the prediction network and the target network are shared, but the weight of the target network is calculated by sliding average and does not directly participate in the gradient update. Thus, the prediction network can gradually improve its own performance by learning the potential representation of the target network.
[0168] 303. Determine a target element in the preset cover, and determine a cover image element vector corresponding to the preset cover according to the target element.
[0169] In this embodiment, the target element is the element that the user is interested in on the cover, such as cats, flowers, characters, etc.; this embodiment is illustrated by taking the character category of celebrities as an example. This is because video works, especially film and television works, revolve around celebrities, and different celebrities have different appeals to users. In addition to the semantic representation of the cover image obtained in the previous module, in order to select images based on different celebrity information, the corresponding representations of different celebrity characters in the cover image can also be extracted.
[0170] Specifically, for each cover image, the faces in the preset cover are first detected based on the target detection network to determine the multiple target faces indicated by the target elements; then the target faces are feature extracted according to the feature extraction network to obtain facial features; and then the facial features corresponding to different target faces are averaged to obtain the cover image element vector corresponding to the preset cover.
[0171] In a possible scenario, the target detection network can be a multi-task cascaded convolutional network (MTCNN). MTCNN is a deep learning algorithm for face detection and facial key point detection. The MTCNN algorithm adopts a cascade approach and divides face detection and facial key point detection into three stages: Proposal Network (P-Net): used to generate candidate face frames; Refine Network (R-Net): used to screen candidate face frames and remove unqualified frames; Output Network (O-Net): used to detect facial key points, and finally predict the position and confidence of the face frame through regression model and classification model. Therefore, for the process of determining the target face, the face in the preset cover can be first detected based on the target detection network to obtain a candidate face frame; then the candidate face frame is screened through P-Net to obtain a screened face frame; the facial key points in the screened face frame are detected through R-Net to determine the position information and confidence information indicated by the face based on the facial key points; and then the multiple target faces indicated by the target element are determined based on the position information and confidence information through O-Net.
[0172] In another possible scenario, the feature extraction network can use the ArcFace algorithm, which is a deep learning algorithm for face recognition. The loss function of the ArcFace algorithm adds a cosine similarity constraint on the basis of the Softmax classifier, so that faces of the same category are closer in the feature space and the distance between faces of different categories is more obvious. This can improve the accuracy and robustness of face recognition. Therefore, for the training process of the feature extraction network, first obtain face training data; then determine the face similarity information corresponding to the training faces in the face training data; and configure the face loss function based on the face similarity information, so that similar faces in the training faces are close in the feature space, and dissimilar faces in the training faces are clearly spaced in the feature space; then train the preset extraction network according to the face loss function to obtain a feature extraction network; and then extract features of the target face according to the feature extraction network to obtain face features.
[0173] Optionally, considering that the user may be interested in different characters and have different degrees of interest in different tasks, the present embodiment simulates the scene through priority information, that is, firstly averaging the facial features corresponding to different target faces to obtain the average features corresponding to the different target faces; then obtaining the priority information corresponding to the different target faces (for example, star A: 0.5; star B: 0.3; star C: 0.2); and then weighting the average features based on the priority information to obtain the cover image element vector corresponding to the preset cover. Through weighted processing of different characters, the user's recommendation for multiple character scenes can be simulated to improve the accuracy of multi-element recommendations.
[0174] This embodiment jointly determines the selection of the video cover image based on the user's historical viewing behavior and the viewing behavior of similar users. For example, if a user has watched many videos about stars A and B, and through collaborative filtering, it is found that similar users are very interested in star C, for a certain video, if it contains stars A, B, C, D, and E, then in the image selection algorithm, it will be more inclined to select the cover image containing stars A, B, and C. In this way, for the user, the cover image contains the stars that he is interested in or is likely to be interested in, and the user's interest value in the video will be higher, which improves the user's retention on the video platform as a whole.
[0175] 304. Combine the cover image element vector with the cover image semantic vector to obtain a cover image target vector corresponding to the preset cover.
[0176] In this embodiment, for each cover image i, a cover image semantic vector indicating the semantic representation of the cover image and a cover image element vector indicating the character representation of the cover image can be obtained. After these two vectors are standardized (L2Normalize) respectively, they are spliced to obtain a cover image target vector vi indicating the overall representation of the cover image.
[0177] 305. Weight the cover image target vector based on the interest information to obtain an object representation vector corresponding to the target object.
[0178] In this embodiment, the cover image target vector is weighted by the interest information to obtain an object representation vector describing the video cover that the user (target object) is interested in.
[0179] Optionally, since the user's interests may change over time, the cover image representation may also be weighted according to the object behavior interest and time, that is, first determine the video interaction sequence of the video indicated by the interest information based on the video interaction information; then determine the time information corresponding to the video interaction sequence; and then weight the cover image target vector based on the interest information and time information to obtain the object representation vector corresponding to the target object. The formula for obtaining the weighted vector that describes the user representation is as follows:
[0180]
[0181] Among them, vi represents the image representation of the i-th video, zui is the output of the user's potential interest module, is the time-weighted value, Δt ui Represents the difference between the current time and the time when user u watched video i, in days.
[0182] This embodiment can emphasize the videos that the user has watched recently through time weighting, so that the cover image selected by the user can better reflect the user's recent interests.
[0183] 306. In response to the input of the candidate cover, determine a candidate cover vector corresponding to the candidate cover, and determine a target cover recommended to the target object based on the candidate cover vector and the object representation vector.
[0184] In this embodiment, for several candidate cover images, their corresponding image representations are extracted, and inner products are performed with the user vectors respectively, and the cover image with the largest value after the inner product is selected as the cover image finally displayed to the user.
[0185] It is understandable that the input of candidate covers can be triggered by user operations on the video, or it can be performed in the background when configuring the video. The specific input method depends on the actual scenario.
[0186] In another possible scenario, this embodiment may adopt Figure 5The module division is as follows: Figure 5 A scenario diagram of another method for recommending video covers provided in an embodiment of the present application; that is, step 301 of this embodiment can be performed by a potential interest calculation module, step 302 can be performed by a cover image semantic representation extraction module, step 303 can be performed by a cover image element representation extraction module, and steps 304-306 can be performed by a cover image selection module.
[0187] Specifically, first, the potential interest calculation module is used to mine the user's potential interests based on the user's viewing behavior, and then the cover image semantic representation extraction module is used to obtain the cover image representation based on the stock video cover images through self-supervised training. Then, the cover image character representation extraction module is used to calculate the character representation appearing in the cover image. Finally, the cover image selection module is used to calculate the user's representation vector based on the cover image representation, and the user representation vector and the inner product of the candidate cover image representation are scored to select the cover image that best suits the user for display.
[0188] Through this embodiment scenario, a personalized cover image selection algorithm based on viewing behavior and potential interests is proposed. For a given number of candidate cover images, the user's potential interests are first mined based on the user's viewing behavior, and then the semantic representation and character representation of the cover image are extracted to obtain the cover image representation. Finally, the user's representation vector is calculated based on the cover image representation, and the user's representation vector and the inner product of the candidate cover image representation are scored, so as to select the cover image that best suits the user for display. Compared with the current work of fixed cover images, the present invention can select cover images according to user interests; and can mine user's potential interests through collaborative filtering to avoid information cocoons.
[0189] In combination with the above embodiments, it can be known that by obtaining the video interaction information corresponding to the target object, the interest information associated with the target object is determined based on the video interaction information; then the image data in the preset cover associated with the target object is represented by a vector to obtain a cover image semantic vector; and the target element in the preset cover is determined to determine the cover image element vector corresponding to the preset cover according to the target element; the cover image element vector is further combined with the cover image semantic vector to obtain the cover image target vector corresponding to the preset cover; then the cover image target vector is weighted based on the interest information to obtain the object representation vector corresponding to the target object; and then in response to the input of the candidate cover, the candidate cover vector corresponding to the candidate cover is determined to determine the target cover recommended to the target object according to the inner product of the candidate cover vector and the object representation vector. Thus, the video cover recommendation process matching the object is realized. Since the interest information is determined by using the object's interaction information, and the object is targeted based on the interest information, the adaptability of the video cover to the target object is improved, and the accuracy of the video cover recommendation is improved.
[0190] The above embodiment introduces the process of determining the target cover. In the video cover recommendation scenario, the recommendation process generally occurs during the video interaction. The following describes this scenario. Figure 6 , Figure 6 A flowchart of another method for recommending a video cover provided in an embodiment of the present application. The method can be executed by a server or a terminal, or by both a server and a terminal. The embodiment of the present application includes at least the following steps:
[0191] 601. In response to an interactive operation of a target object on a video display interface, a video to be displayed is obtained.
[0192] In this embodiment, the interactive operation of the target object in the video display interface can be a click operation, such as clicking on a video in a certain column; it can also be a sliding operation, such as sliding in the video list to switch the video list. The specific interactive operation depends on the actual scenario and is not described here.
[0193] Specifically, Figure 7 As shown, Figure 7 A schematic diagram of another method for recommending video covers provided in an embodiment of the present application; the user obtains video 3 and video 4 to be displayed by sliding on the video display interface A1. That is, when recommending a video to a user, the cover image that best matches the user will be selected based on this embodiment, and then the cover image will be displayed to the user. In this way, different cover images will be displayed to different users, which improves the overall attractiveness of the platform to users and increases user retention.
[0194] 602. Determine candidate covers for the video configuration to be displayed.
[0195] In this embodiment, the candidate cover configured for the video to be displayed can be the cover displayed in the historical record, or it can be a representative cover obtained by content recognition of the video to be displayed, or it can be a cover configured by relevant personnel in the background. The specific determination process depends on the actual scenario.
[0196] 603. Determine a candidate cover vector corresponding to the candidate cover, and obtain a recommendation score by performing an inner product calculation on the candidate cover vector and the object representation vector.
[0197] In this embodiment, the candidate cover vector can be Figure 3 Steps 302 - 305 of the illustrated embodiment are not described in detail here.
[0198] 604. Determine a target cover to be recommended to the target object based on the recommendation score.
[0199] In this embodiment, the largest candidate cover can be determined as the target cover according to the size of the recommendation score; or the first candidate cover whose recommendation score exceeds the threshold can be used as the target cover, thereby eliminating the need to calculate all candidate covers and improving the efficiency of cover configuration.
[0200] It is understandable that the above-mentioned process of determining the target cover can be cached in the background when the video application is started, that is, when the user starts the video platform, the background of the video platform automatically caches the covers of the relevant videos.
[0201] In addition, the target cover determination process can also be performed in real time during the user browsing process, that is, the video to be displayed is determined during the user interaction operation, and the cover is determined at this time. Through the cover recommendation process, different cover images can be displayed to different users in a personalized way to enhance the user experience.
[0202] In order to better implement the above solution of the embodiment of the present application, the following also provides related devices for implementing the above solution. Figure 8 , Figure 8 The structure diagram of a video cover recommendation device provided in an embodiment of the present application is as follows. The recommendation device 800 includes:
[0203] An acquisition unit 801 is configured to acquire video interaction information corresponding to a target object, so as to determine interest information associated with the target object based on the video interaction information;
[0204] The processing unit 802 is used to perform vector representation on the image data in the preset cover associated with the target object to obtain a cover image semantic vector;
[0205] The processing unit 802 is further configured to determine a target element in the preset cover, so as to determine a cover image element vector corresponding to the preset cover according to the target element;
[0206] The processing unit 802 is further configured to combine the cover image element vector with the cover image semantic vector to obtain a cover image target vector corresponding to the preset cover;
[0207] The processing unit 802 is further configured to weight the cover image target vector based on the interest information to obtain an object representation vector corresponding to the target object;
[0208] The recommendation unit 803 is used to determine a candidate cover vector corresponding to the candidate cover in response to an input of a candidate cover, and determine a target cover recommended to the target object based on the candidate cover vector and the object representation vector.
[0209] Optionally, in some possible implementations of the present application, the acquisition unit 801 is specifically configured to acquire video interaction information corresponding to the target object, so as to determine an interest video corresponding to the target object based on an interaction action in the video interaction information;
[0210] The acquisition unit 801 is specifically configured to determine a video vector corresponding to each video in the video set based on the video interaction information;
[0211] The acquisition unit 801 is specifically configured to determine the cosine similarity between videos according to the video vectors, so as to determine the interest set corresponding to the target object according to the cosine similarity;
[0212] The acquisition unit 801 is specifically configured to determine the interest level of the target object for each video based on the interest set, so as to determine the corresponding predicted video according to the interest level;
[0213] The acquisition unit 801 is specifically configured to combine the predicted video with the interest video to obtain the interest information associated with the target object.
[0214] Optionally, in some possible implementations of the present application, the acquisition unit 801 is specifically configured to acquire video interaction information corresponding to the target object;
[0215] The acquisition unit 801 is specifically configured to extract search records according to the interaction actions in the video interaction information;
[0216] The acquisition unit 801 is specifically configured to determine the search video information associated with the search record, so as to acquire the interest video indicated by the search video information.
[0217] Optionally, in some possible implementations of the present application, the acquisition unit 801 is specifically used to acquire object portrait information corresponding to the target object;
[0218] The acquisition unit 801 is specifically configured to determine a video tag of the target object associated with the video content according to the object portrait information;
[0219] The acquisition unit 801 is specifically configured to call the video content associated with the video tag to expand the interest video.
[0220] Optionally, in some possible implementations of the present application, the acquisition unit 801 is specifically configured to call the video content associated with the video tag to obtain the called video;
[0221] The acquisition unit 801 is specifically configured to acquire the interaction period information corresponding to the video tag, so as to filter the called video with extension based on the interaction period information;
[0222] The acquisition unit 801 is specifically configured to expand the interest video based on the extended video.
[0223] Optionally, in some possible implementations of the present application, the processing unit 802 is specifically configured to obtain training data of the preset cover indication;
[0224] The processing unit 802 is specifically used to input the training data into the prediction network for vector representation to obtain a prediction representation vector;
[0225] The processing unit 802 is specifically configured to transmit the predicted representation vector to the target network for prediction to obtain an output representation vector, wherein the parameter weight information of the target network is an exponential moving average of the parameter weight information of the predicted network;
[0226] The processing unit 802 is specifically configured to determine loss information between the output representation vector and the predicted representation vector, wherein the loss information includes a cosine similarity loss and a mean square error loss;
[0227] The processing unit 802 is specifically configured to perform self-supervisory training on the prediction network according to the loss information;
[0228] The processing unit 802 is specifically used to perform vector representation on the image data in the preset cover associated with the target object based on the trained prediction network to obtain a cover image semantic vector.
[0229] Optionally, in some possible implementations of the present application, the processing unit 802 is specifically configured to detect a face in the preset cover based on a target detection network to determine a plurality of target faces indicated by the target element;
[0230] The processing unit 802 is specifically configured to extract features of the target face according to a feature extraction network to obtain facial features;
[0231] The processing unit 802 is specifically used to average the facial features corresponding to different target faces to obtain the cover image element vector corresponding to the preset cover.
[0232] Optionally, in some possible implementations of the present application, the processing unit 802 is specifically configured to detect a face in the preset cover based on the object detection network to obtain a candidate face frame;
[0233] The processing unit 802 is specifically configured to filter the candidate face frames to obtain a filtered face frame;
[0234] The processing unit 802 is specifically configured to detect key points of the face in the screened face frame, so as to determine position information and confidence information of the face indication based on the key points of the face;
[0235] The processing unit 802 is specifically configured to determine multiple target faces indicated by the target element based on the position information and the confidence information.
[0236] Optionally, in some possible implementations of the present application, the processing unit 802 is specifically used to obtain face training data;
[0237] The processing unit 802 is specifically configured to determine the face similarity information corresponding to the training faces in the face training data;
[0238] The processing unit 802 is specifically configured to configure a face loss function based on the face similarity information, so that similar faces in the training faces are closely spaced in the feature space, and dissimilar faces in the training faces are clearly spaced in the feature space;
[0239] The processing unit 802 is specifically configured to train a preset extraction network according to the face loss function to obtain the feature extraction network;
[0240] The processing unit 802 is specifically configured to perform feature extraction on the target face according to the feature extraction network to obtain the facial features.
[0241] Optionally, in some possible implementations of the present application, the processing unit 802 is specifically configured to average the facial features corresponding to different target faces to obtain average features corresponding to the different target faces;
[0242] The processing unit 802 is specifically used to obtain priority information corresponding to different target faces;
[0243] The processing unit 802 is specifically used to weight the average feature based on the priority information to obtain a cover image element vector corresponding to the preset cover.
[0244] Optionally, in some possible implementations of the present application, the processing unit 802 is specifically configured to determine a video interaction sequence of the video indicated by the interest information based on the video interaction information;
[0245] The processing unit 802 is specifically configured to determine time information corresponding to the video interaction sequence;
[0246] The processing unit 802 is specifically used to weight the cover image target vector based on the interest information and the time information to obtain an object representation vector corresponding to the target object.
[0247] Optionally, in some possible implementations of the present application, the recommendation unit 803 is specifically configured to obtain the video to be displayed in response to the interactive operation of the target object on the video display interface;
[0248] The recommendation unit 803 is specifically used to determine the candidate cover of the video configuration to be displayed;
[0249] The recommendation unit 803 is specifically used to determine the candidate cover vector corresponding to the candidate cover, and obtain a recommendation score by performing an inner product calculation on the candidate cover vector and the object representation vector;
[0250] The recommendation unit 803 is specifically configured to determine the target cover to be recommended to the target object based on the recommendation score.
[0251] By obtaining the video interaction information corresponding to the target object, the interest information associated with the target object is determined based on the video interaction information; then the image data in the preset cover associated with the target object is represented by a vector to obtain a cover image semantic vector; and the target element in the preset cover is determined to determine the cover image element vector corresponding to the preset cover according to the target element; the cover image element vector is further combined with the cover image semantic vector to obtain the cover image target vector corresponding to the preset cover; then the cover image target vector is weighted based on the interest information to obtain the object representation vector corresponding to the target object; and then in response to the input of the candidate cover, the candidate cover vector corresponding to the candidate cover is determined to determine the target cover recommended to the target object according to the inner product of the candidate cover vector and the object representation vector. Thus, the video cover recommendation process matching the object is realized. Since the interest information is determined by using the object's interaction information, and the object is targeted based on the interest information, the adaptability of the video cover to the target object is improved, and the accuracy of the video cover recommendation is improved.
[0252] The present application also provides a computer device, which may be a terminal. Fig. 9 As shown, taking the terminal as a mobile phone as an example:
[0253] Fig. 9 The block diagram shows a partial structure of a mobile phone provided in an embodiment of the present application. Fig. 9The mobile phone includes: a radio frequency (RF) circuit 910, a memory 920, an input unit 930, a display unit 940, a sensor 950, an audio circuit 960, a wireless fidelity (WiFi) module 970, a processor 980, and a power supply 990. Those skilled in the art will understand that Fig. 9 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0254] Combine the following Fig. 9 A detailed introduction to the various components of the mobile phone:
[0255] The RF circuit 910 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 980 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 910 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 910 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to the global system of mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc.
[0256] The memory 920 can be used to store software programs and modules. The processor 980 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 920. The memory 920 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 920 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0257] The input unit 930 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 930 may include a touch panel 931 and other input devices 932. The touch panel 931, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 931 or near the touch panel 931 using any suitable object or accessory such as a finger, stylus, etc., and the air touch operation within a certain range on the touch panel 931), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 931 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 980, and can receive and execute commands sent by the processor 980. In addition, the touch panel 931 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 931, the input unit 930 may further include other input devices 932. Specifically, the other input devices 932 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.
[0258] The display unit 940 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 940 may include a display panel 941. Optionally, the display panel 941 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 931 may cover the display panel 941. When the touch panel 931 detects a touch operation on or near it, it is transmitted to the processor 980 to determine the type of touch event. Subsequently, the processor 980 provides a corresponding visual output on the display panel 941 according to the type of touch event. Although in Fig. 9 In the embodiment, the touch panel 931 and the display panel 941 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 931 and the display panel 941 can be integrated to realize the input and output functions of the mobile phone.
[0259] The processor 980 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 920, and calling data stored in the memory 920, it executes various functions of the mobile phone and processes data, thereby performing overall detection of the mobile phone. Optionally, the processor 980 may include one or more processing units; optionally, the processor 980 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 980.
[0260] The mobile phone also includes a power supply 990 (such as a battery) for supplying power to various components. Optionally, the power supply can be logically connected to the processor 980 through a power management system, thereby managing functions such as charging, discharging, and power consumption management through the power management system.
[0261] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.
[0262] In the embodiment of the present application, the processor 980 included in the mobile phone also has the function of executing each step of the above-mentioned page processing method.
[0263] The present application also provides a computer device, which may be a server. Fig.10 , Fig.101 is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server 1000 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1022 (for example, one or more processors) and a memory 1032, and one or more storage media 1030 (for example, one or more mass storage devices) storing application programs 1042 or data 1044. Among them, the memory 1032 and the storage medium 1030 can be short-term storage or permanent storage. The program stored in the storage medium 1030 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 1022 can be configured to communicate with the storage medium 1030 to execute a series of instruction operations in the storage medium 1030 on the server 1000.
[0264] The server 1000 may also include one or more power supplies 1026, one or more wired or wireless network interfaces 1050, one or more input and output interfaces 1058, and / or one or more operating systems 1041, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
[0265] The steps performed by the management device in the above embodiment can be based on the Fig.10 The server structure shown.
[0266] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a computer program, which, when executed on a computer device, enables the computer device to execute the method described in the foregoing embodiment.
[0267] The embodiments of the present application also provide a computer program product including a computer program, which, when executed on a computer device, enables the computer device to execute the method described in the aforementioned embodiments.
[0268] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0269] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0270] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0271] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0272] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a terminal or a server, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.
[0273] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A video cover recommendation method, It is characterized in that include: Acquire video interaction information corresponding to the target object, so as to determine interest information associated with the target object based on the video interaction information; Performing vector representation on the image data in the preset cover associated with the target object to obtain a cover image semantic vector; Determine a target element in the preset cover, so as to determine a cover image element vector corresponding to the preset cover according to the target element; Obtaining a cover image target vector corresponding to the preset cover according to the cover image element vector and the cover image semantic vector; Weighting the cover image target vector based on the interest information to obtain an object representation vector corresponding to the target object; In response to the input of the candidate cover, a candidate cover vector corresponding to the candidate cover is determined, and a target cover recommended to the target object is determined based on the candidate cover vector and the object representation vector.
2. The method according to claim 1, It is characterized in that The acquiring the video interaction information corresponding to the target object to determine the interest information associated with the target object based on the video interaction information includes: Acquire video interaction information corresponding to the target object, so as to determine an interest video corresponding to the target object based on an interaction action in the video interaction information; Determine a video vector corresponding to each video in the video set based on the video interaction information; Determine the cosine similarity between the videos according to the video vectors, so as to determine the interest set corresponding to the target object according to the cosine similarity; Determining the interest level of the target object for each video based on the interest set, so as to determine a corresponding predicted video according to the interest level; The predicted video is combined with the interest video to obtain interest information associated with the target object.
3. The method according to claim 2, It is characterized in that The acquiring the video interaction information corresponding to the target object, so as to determine the interest video corresponding to the target object based on the interaction action in the video interaction information, includes: Obtaining video interaction information corresponding to the target object; Extracting search records according to the interactive actions in the video interaction information; Determine the search video information associated with the search record to obtain the interest video indicated by the search video information.
4. The method according to claim 2, It is characterized in that The method further comprises: Obtaining object portrait information corresponding to the target object; Determining a video tag of the target object associated with the video content according to the object portrait information; The video content associated with the video tag is called to expand the interest video.
5. The method according to claim 4, It is characterized in that The calling of the video content associated with the video tag to expand the interest video includes: Calling the video content associated with the video tag to obtain the called video; Acquire the interactive time period information corresponding to the video tag, and filter the called video based on the interactive time period information to obtain the extended video; The interest video is expanded based on the expanded video.
6. The method according to claim 1, It is characterized in that The step of performing vector representation on the image data in the preset cover associated with the target object to obtain a cover image semantic vector includes: Acquire training data of the preset cover indication; Inputting the training data into a prediction network for vector representation to obtain a prediction representation vector; The predicted representation vector is transmitted to the target network for prediction to obtain an output representation vector, wherein the parameter weight information of the target network is an exponential moving average of the parameter weight information of the predicted network; Determine loss information between the output representation vector and the predicted representation vector, the loss information comprising cosine similarity loss and mean square error loss; Performing self-supervisory training on the prediction network according to the loss information; Based on the trained prediction network, the image data in the preset cover associated with the target object is represented by vector to obtain a cover image semantic vector.
7. The method according to claim 1, It is characterized in that The step of determining a target element in the preset cover, so as to determine a cover image element vector corresponding to the preset cover according to the target element, includes: Detecting the human face in the preset cover based on a target detection network to determine multiple target human faces indicated by the target element; Extracting features of the target face according to a feature extraction network to obtain facial features; The facial features corresponding to different target faces are averaged to obtain a cover image element vector corresponding to the preset cover.
8. The method according to claim 7, It is characterized in that The detecting the human face in the preset cover based on the target detection network to determine the multiple target human faces indicated by the target element includes: Detecting the face in the preset cover based on the target detection network to obtain a candidate face frame; Screening the candidate face frames to obtain screened face frames; Detecting key points of the face in the screened face frame to determine position information and confidence information of the face indication based on the key points of the face; A plurality of target faces indicated by the target element are determined based on the position information and the confidence information.
9. The method according to claim 7, It is characterized in that The step of extracting features from the target face according to the feature extraction network to obtain facial features includes: Get face training data; Determining facial similarity information corresponding to training faces in the facial training data; Configuring a face loss function based on the face similarity information so that similar faces in the training faces are closely spaced in the feature space, and dissimilar faces in the training faces are clearly spaced in the feature space; Training a preset extraction network according to the face loss function to obtain the feature extraction network; Feature extraction is performed on the target face according to the feature extraction network to obtain the facial features.
10. The method according to claim 7, It is characterized in that The averaging of facial features corresponding to different target faces to obtain a cover image element vector corresponding to the preset cover includes: The facial features corresponding to different target faces are averaged to obtain the average features corresponding to the different target faces; Obtain priority information corresponding to different target faces; The average feature is weighted based on the priority information to obtain a cover image element vector corresponding to the preset cover.
11. The method according to claim 1, It is characterized in that The step of weighting the cover image target vector based on the interest information to obtain an object representation vector corresponding to the target object includes: Determine, based on the video interaction information, a video interaction sequence of the video indicated by the interest information; Determining time information corresponding to the video interaction sequence; The cover image target vector is weighted based on the interest information and the time information to obtain an object representation vector corresponding to the target object.
12. The method according to claim 1, It is characterized in that The step of determining, in response to the input of the candidate cover, a candidate cover vector corresponding to the candidate cover, and determining a target cover recommended to the target object according to the candidate cover vector and the object representation vector includes: In response to the target object's interactive operation on the video display interface, obtaining the video to be displayed; Determining the candidate cover of the video configuration to be displayed; Determine a candidate cover vector corresponding to the candidate cover, and obtain a recommendation score based on an inner product calculation between the candidate cover vector and the object representation vector; The target cover recommended to the target object is determined based on the recommendation score.
13. A device for recommending video cover, It is characterized in that include: An acquisition unit, configured to acquire video interaction information corresponding to a target object, so as to determine interest information associated with the target object based on the video interaction information; A processing unit, used for performing vector representation on the image data in the preset cover associated with the target object to obtain a semantic vector of the cover image; The processing unit is further used to determine a target element in the preset cover, so as to determine a cover image element vector corresponding to the preset cover according to the target element; The processing unit is further used to obtain a cover image target vector corresponding to the preset cover according to the cover image element vector and the cover image semantic vector; The processing unit is further used to weight the cover image target vector based on the interest information to obtain an object representation vector corresponding to the target object; The recommendation unit is used to determine a candidate cover vector corresponding to the candidate cover in response to an input of a candidate cover, and determine a target cover recommended to the target object based on the candidate cover vector and the object representation vector.
14. A computer device, It is characterized in that The computer device comprises a processor and a memory: The memory is used to store a computer program; the processor is used to execute the video cover recommendation method according to any one of claims 1 to 12 according to the instructions in the computer program.
15. A computer program product, comprising a computer program, wherein the computer program is stored in a computer-readable storage medium. It is characterized in that When the computer program in the computer-readable storage medium is executed by a processor, the steps of the method for recommending a video cover as described in any one of claims 1 to 12 are implemented.
