Video recommendation method and model training method and device for video recommendation

Abstract

The invention provides a video recommendation method, a model training method and device for video recommendation, electronic equipment and a storage medium, relates to the field of artificial intelligence, in particular to the fields of deep learning, natural language processing and image processing, and can be used for application scenes such as generative search, document intelligent editing, intelligent assistants, virtual assistants and intelligent e-commerce. The specific implementation scheme is as follows: extracting target feature information of a satisfactory video of a target object and target feature information of a candidate video from a pre-configured target feature information set; the target feature information set comprises target feature information of a plurality of videos in a video library, and the target feature information is obtained by performing dimensionality reduction on the basis of multi-modal semantic information of the corresponding videos; determining the score of the candidate video based on the target feature information of the satisfactory video and the target feature information of the candidate video; and based on the score of the candidate video, determining a recommended video for the target object.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and in particular to the fields of deep learning, natural language processing and image processing. Background Technology

[0002] With the rapid development and widespread adoption of the internet, the number of videos uploaded to video platforms is gradually increasing, and users' demand for watching videos is also rising continuously. Personalized video recommendations have become an important means of improving the user viewing experience. Therefore, it is necessary to consider how to predict the video content that users may be interested in. Summary of the Invention

[0003] This disclosure provides a video recommendation method, a model training method for video recommendation, and an apparatus.

[0004] According to one aspect of this disclosure, a video recommendation method is provided, comprising:

[0005] In response to receiving a video recommendation request for a target object, the system extracts the target feature information of the target object's satisfactory videos and the target feature information of candidate videos from a pre-configured target feature information set. The target feature information set includes the target feature information of multiple videos in the video library, and the target feature information is obtained by dimensionality reduction based on the multimodal semantic information of its corresponding video.

[0006] The score of the candidate video is determined based on the target feature information of the satisfactory video and the target feature information of the candidate video.

[0007] Based on the ratings of candidate videos, recommended videos are determined for the target audience.

[0008] According to another aspect of this disclosure, a method for training a model for video recommendation is provided, comprising:

[0009] Based on the first sample video and its corresponding score in the first sample dataset, a target model is obtained through a preset number of training rounds. The target model includes a multilayer perceptron and a ranking model. The multilayer perceptron is used to reduce the dimensionality of the multimodal semantic vector of the target video to obtain the target feature information of the target video. The ranking model is used to determine the score of the target video based on the target feature information of the target video.

[0010] A multilayer perceptron is used to determine the target feature information of the second sample video in the second sample dataset;

[0011] The ranking model is trained online based on the target feature information of the second sample video and the corresponding score of the second sample video.

[0012] According to another aspect of this disclosure, a video recommendation device is provided, comprising:

[0013] The feature information extraction module is used to respond to a received video recommendation request for a target object by extracting the target feature information of the target object's satisfactory videos and the target feature information of the candidate videos from a pre-configured target feature information set. The target feature information set includes the target feature information of multiple videos in the video library, and the target feature information is obtained by dimensionality reduction based on the multimodal semantic information of its corresponding video.

[0014] The video scoring module is used to determine the score of candidate videos based on the target feature information of satisfactory videos and the target feature information of candidate videos.

[0015] The recommended video determination module is used to determine recommended videos for the target audience based on the scores of candidate videos.

[0016] According to another aspect of this disclosure, a model training apparatus for video recommendation is provided, comprising:

[0017] The first model training module is used to train the target model for a preset number of rounds based on the first sample video and the corresponding score in the first sample dataset. The target model includes a multilayer perceptron and a ranking model. The multilayer perceptron is used to reduce the dimensionality of the multimodal semantic vector of the target video to obtain the target feature information of the target video. The ranking model is used to determine the score of the target video based on the target feature information of the target video.

[0018] The feature information determination module is used to determine the target feature information of the second sample video in the second sample dataset using a multilayer perceptron.

[0019] The second model training module is used to train the ranking model online based on the target feature information of the second sample video and the corresponding score of the second sample video.

[0020] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0021] At least one processor; and

[0022] The memory is communicatively connected to the at least one processor; wherein,

[0023] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0024] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0025] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0026] The method provided in this disclosure reduces the computational and storage overhead during online inference by pre-reducing the dimensionality of the multimodal semantic information of videos in the video library to obtain a target feature information set. Upon receiving a video recommendation request for a target object, the method utilizes the target feature information of satisfactory videos and candidate videos in the target feature information set to determine the score of the candidate videos. Based on this score, a recommended video is determined. This means that upon receiving a video recommendation request for a target object, it is not necessary to perform large-scale computation to obtain multimodal semantic information to acquire target feature information. Instead, it can fully utilize the multimodal information of the videos to determine recommended videos that fit the target object, thereby improving both the efficiency and effectiveness of video recommendation.

[0027] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0028] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0029] Figure 1 This is a schematic flowchart of a video recommendation method provided in an embodiment of this disclosure;

[0030] Figure 2 This is a schematic flowchart of a model training method for video recommendation provided in an embodiment of this disclosure;

[0031] Figure 3 This is a schematic diagram illustrating an application example of the model training method for video recommendation provided in this disclosure.

[0032] Figure 4 This is a schematic block diagram of a video recommendation device provided in one embodiment of the present disclosure;

[0033] Figure 5 This is a schematic block diagram of a video recommendation device provided in another embodiment of this disclosure;

[0034] Figure 6 This is a schematic block diagram of a video recommendation device provided in another embodiment of this disclosure;

[0035] Figure 7 This is a schematic block diagram of a model training apparatus for video recommendation provided in an embodiment of the present disclosure;

[0036] Figure 8 This is a block diagram of an electronic device used to implement the video recommendation method and / or the model training method for video recommendation according to the embodiments of this disclosure. Detailed Implementation

[0037] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0038] Figure 1 This disclosure illustrates a video recommendation method according to an embodiment. This method can be applied to a video recommendation device, which can be deployed in an electronic device. The electronic device can be a single or multi-device terminal, server, or other processing device. The terminal can be a mobile device, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, or other user equipment (UE); the server can be a single-device server or a server cluster. In some possible implementations, the method can also be implemented by a processor calling computer-readable instructions stored in memory. Figure 1 As shown, the method may include the following steps S110 to S130.

[0039] Step S110: In response to receiving a video recommendation request for the target object, extract the target feature information of the satisfactory video and the target feature information of the candidate video from the pre-configured target feature information set; wherein, the target feature information set includes the target feature information of multiple videos in the video library, and the target feature information is obtained by dimensionality reduction based on the multimodal semantic information of its corresponding video.

[0040] In this embodiment of the disclosure, the target object can be a user who needs to watch videos. When the electronic device of the target object detects actions such as swiping the screen or clicking on the video recommendation list, it can send a video recommendation request to the video recommendation device.

[0041] In this embodiment, the video recommendation device can pre-degrade the multimodal semantic information of each video to obtain target feature information for each video, and store the target feature information for each video to obtain a target feature information set. Subsequently, when a video recommendation request for a target object is received, the stored target feature information set can be read, and the target feature information of the target object's satisfactory videos and candidate videos can be extracted from the target feature information set.

[0042] In some examples, the multimodal semantic information of a video can be vector information, which can be obtained by fusing the semantic vectors of the video frame and the semantic vectors of the text.

[0043] In some examples, the target audience's satisfactory videos are those videos in the video library that the target audience has already watched and that exhibit positive feedback behavior. For example, positive feedback behavior may include one or more actions such as the target audience watching (e.g., watching for a duration exceeding a preset threshold), liking, or saving.

[0044] Optionally, the number of satisfactory videos can be one or more, and the target feature information of each satisfactory video can be different.

[0045] In this embodiment of the disclosure, target feature information of each satisfactory video and target feature information of candidate videos of a target object can be extracted from a pre-configured set of target feature information. The satisfactory videos of each target object can be different.

[0046] In this embodiment of the disclosure, candidate videos can be understood as videos to be recommended. Any one or more videos in the video library other than the satisfactory videos of the target object can be used as candidate videos, and the target feature information of each candidate video can be different.

[0047] Step S120: Determine the score of the candidate video based on the target feature information of the satisfactory video and the target feature information of the candidate video.

[0048] Optionally, the rating of a candidate video is used to characterize the degree to which the candidate video matches the viewing interests of the target audience. The target feature information of a satisfactory video can characterize the viewing interests of the target audience; therefore, by utilizing the target feature information of both the satisfactory video and the candidate video, the rating can be accurately determined.

[0049] For example, the rating of a candidate video can be related to the similarity between the satisfactory video and the candidate video. For instance, the similarity between the two can be proportional to the rating of the candidate video, or the rating of the candidate video can be determined based on the similarity between the satisfactory video and the candidate video.

[0050] Optionally, when there are multiple satisfactory videos, the score of the candidate video can be determined based on the target feature information of each satisfactory video and the target feature information of the candidate video.

[0051] Optionally, the score for each candidate video can be determined based on the target feature information of the satisfactory video and the target feature information of each candidate video, and the score for each candidate video can be different.

[0052] Optionally, a ranking model can be used to process the target feature information of each satisfactory video and the target feature information of each candidate video to obtain a score for the candidate video.

[0053] Step S130: Based on the ratings of the candidate videos, determine the recommended videos for the target audience.

[0054] In this embodiment of the disclosure, the recommended video can be the video displayed to the target object on the target object's electronic device after receiving a video recommendation request from the target object. Since each target object may have a different satisfactory video, the recommended video may also be different for different target objects.

[0055] Optionally, the recommended videos may include the next video displayed to the target object after the target object swipes the screen, or videos displayed in the video recommendation list after the target object clicks on the video recommendation list.

[0056] Optionally, one or more videos with higher ratings (e.g., the K highest-rated videos, where K is a positive integer) can be selected as recommended videos for the target audience.

[0057] Optionally, a threshold for the rating of candidate videos can be preset. If the rating of a candidate video is higher than the preset threshold, the candidate video is determined to be a recommended video for the target audience.

[0058] In related technologies, models typically model user preferences using ID (identifier) ​​and statistical features. Specifically, they train an embedding (vector) based on the content ID, source ID (e.g., publisher ID), and category ID of content the user has historically consumed (e.g., videos watched or liked). This process does not utilize multimodal signals of the content, such as cover image vectors or title semantic vectors, because multimodal signals generally have high dimensionality (128-1024 dimensions), while conventional ID-type feature vectors are around 10 dimensions. Therefore, using multimodal signals would lead to significant computational and storage overhead during online inference. However, if only ID-type features are included, when new content is published, the ID-type feature vectors are all default values ​​(or lack training), easily leading to inaccurate predictions of new content. The method provided in this disclosure reduces the computational and storage overhead during online inference by pre-reducing the dimensionality of the multimodal semantic information of videos in the video library to obtain a target feature information set. When a video recommendation request for a target object is received, the target feature information of the target object's satisfactory videos and the target feature information of candidate videos in the target feature information set is used to determine the score of the candidate videos. Based on the score, the recommended video is determined. That is, when a video recommendation request for a target object is received, there is no need to perform large-scale calculations to obtain multimodal semantic information to obtain target feature information. The multimodal information of the video can be fully utilized to determine the recommended video that fits the target object, thereby improving the video recommendation efficiency and the recommendation effect.

[0059] In some embodiments, the video recommendation method may further include:

[0060] A multimodal model is used to obtain the multimodal semantic vectors of the videos to be processed in the video library;

[0061] Multilayer Perceptron (MLP) is used to reduce the dimensionality of multimodal semantic vectors to obtain the target feature information of the video to be processed.

[0062] Add the target feature information of the video to be processed to the target feature information set.

[0063] In this embodiment of the disclosure, the video to be processed can be a video in the video library for which the target feature information has not been determined. For example, the video to be processed can be a video newly published to the video library.

[0064] In this embodiment of the disclosure, the multimodal semantic vector of the video to be processed may include the semantic vector of the image and the semantic vector of the text of the video to be processed. The semantic vector of the image and the semantic vector of the text can be fused to obtain the multimodal semantic vector of the video.

[0065] Optionally, the multimodal semantic vector can be a high-dimensional vector, such as a 768*3=2304-dimensional vector.

[0066] Optionally, the multimodal model may include the CLIP (Contrastive Language–Image Pre-training) model. The CLIP model can extract semantic vectors from images (e.g., video cover, keyframes, etc.) and text (e.g., video summary, title, etc.) of the video to be processed, and fuse the semantic vectors of the images and the text to obtain the multimodal semantic information of the video.

[0067] In this embodiment of the disclosure, vector features in multimodal semantic vectors can be extracted through one layer of MLP training, and the vector features can be dimensionality reduced to obtain target feature information of the video to be processed. That is, the target feature information can be vector features with low dimension, and the low-dimensional target feature information can be stored in the target feature information set.

[0068] In this embodiment of the disclosure, a multilayer perceptron can be used to reduce the dimensionality of the multimodal semantic vectors of multiple videos to be processed, thereby obtaining the target feature information of each video to be processed. The target feature information of each video to be processed can be fixed, and the target feature information of each video to be processed can have a low-dimensional vector of the same dimension.

[0069] In one example, MLP can be used to reduce the 768*3=2304-dimensional multimodal semantic vector to target feature information of 56 to 128 dimensions, and add the target feature information of 56 to 128 dimensions to the target feature information set.

[0070] Optionally, CLIP can be used to extract the semantic vectors of images (e.g., video cover, keyframes, etc.) and text (e.g., video summary, title, etc.) for each video separately, and then fuse the two to obtain the multimodal semantic information of each video. Then, MLP (Multilayer Perceptron) is used to reduce the dimensionality of the multimodal semantic information of each video to obtain the target feature information of each video.

[0071] In some embodiments, the training process for the multilayer perceptron may include: training for a preset number of rounds based on a first sample video in a first sample dataset and the score corresponding to the first sample video to obtain a multilayer perceptron, wherein the multilayer perceptron is used to reduce the dimensionality of the multimodal semantic vector of the target video to obtain the target feature information of the target video.

[0072] According to the above embodiments, the multimodal semantic vectors of the video to be processed are extracted by a multimodal model, which improves the efficiency and consistency of multimodal semantic vector extraction. Furthermore, the dimensionality reduction of the multimodal semantic vectors is performed by a multilayer perceptron, which improves the efficiency and consistency of dimensionality reduction and further reduces the computation and storage overhead during online inference.

[0073] In some embodiments, a multimodal model is used to obtain the multimodal semantic vector of the video to be processed in the video library, including:

[0074] A multimodal model is used to process one or more pieces of information from the video clips, titles, and summaries of the video to be processed, resulting in a multimodal semantic vector of the video.

[0075] For example, the video segments of the video to be processed may include key segments from the video. For instance, the video segments may include the first three frames, the last three frames, and the video cover. Alternatively, the video content of the video to be processed can be uniformly sampled every three seconds to obtain the video segments.

[0076] Optionally, the title and summary of the video to be processed can be text uploaded by the video author when publishing the video, or text generated by the video recommendation device based on the video content of the video to be processed.

[0077] Optionally, a multimodal model can be used to extract semantic vectors from video segments, titles, and summaries respectively, and then the semantic vectors from video segments, titles, and summaries can be fused to obtain the multimodal semantic vector of the video to be processed.

[0078] According to the above embodiments, multimodal semantic vectors can accurately represent video segments, titles, and summaries of the video to be processed, improving the accuracy of multimodal semantic vectors, thereby enabling the determination of recommended videos that are more closely aligned with the target audience during subsequent online inference.

[0079] In some embodiments, the video recommendation method may further include:

[0080] Based on the target object's historical records, determine the satisfactory videos of the target object from the video library.

[0081] In this embodiment of the disclosure, the target object's historical records may include a record of the target object's feedback behaviors in response to watched videos. Based on the historical records, videos in which the target object exhibits positive feedback behaviors can be identified as videos that satisfy the target object.

[0082] Optionally, the target's historical records may include the completeness (or duration) of the video watched by the target, likes, and favorites. Videos with high completeness (or long duration) of viewing, videos liked, and videos favorited by the target can be identified as videos that the target is satisfied with.

[0083] According to the above embodiments, the target object's satisfactory videos can be accurately determined through the target object's historical records, thereby determining recommended videos that are more closely aligned with the target object.

[0084] In some embodiments, step S120, determining the score of the candidate video based on the target feature information of the satisfactory video and the target feature information of the candidate video, may include:

[0085] A ranking model is used to process the target feature information of satisfactory videos, the target feature information of candidate videos, and the identifier class features of target objects to obtain the scores of candidate videos.

[0086] In this embodiment of the disclosure, the ranking model only needs to query the target feature information of the previously stored satisfactory videos and the target feature information of the candidate videos to calculate the score of the candidate videos, without having to re-perform the repeated calculation from multimodal semantic information to target feature information.

[0087] Alternatively, the ranking model can be any model from related technologies. For example, the ranking model can be DIN (Deep Interest Network).

[0088] Optionally, the target feature information of multiple satisfactory videos, the target feature information of candidate videos, and the identifier class features of the target object can be simultaneously input into the ranking model. The ranking model can then combine the target feature information of multiple satisfactory videos to give a score for the candidate video.

[0089] Optionally, the identifier class feature of the target object may include the ID of the target object, so that the score of the candidate video can be associated with the target object. The score of the same candidate video for different target objects can be different, thereby realizing personalized scoring for different target objects.

[0090] In some embodiments, the training process for the ranking model may include:

[0091] Based on the first sample video in the first sample dataset and the corresponding score of the first sample video, a target model is obtained by training for a preset number of rounds. The target model includes a multilayer perceptron and a ranking model. The multilayer perceptron is used to reduce the dimensionality of the multimodal semantic vector of the target video to obtain the target feature information of the target video. The ranking model is used to determine the score of the target video based on the target feature information of the target video.

[0092] The multilayer perceptron is used to determine the target feature information of the second sample video in the second sample dataset;

[0093] The ranking model is trained online based on the target feature information of the second sample video and the score corresponding to the second sample video.

[0094] According to the above embodiments, by using a ranking model to process the target feature information of satisfactory videos, the target feature information of candidate videos, and the identification class features of target objects, the efficiency and accuracy of score prediction can be improved, thereby further enhancing the video viewing experience of target objects in immersive scenarios.

[0095] In some embodiments, the identification class features include the category identifier and / or video identifier of the processed video of the target object.

[0096] In this embodiment of the disclosure, the identifier class feature can characterize the target user's satisfaction with the processed video, and the category identifier and / or video identifier of the processed video of the target object can guide the ranking model to score the candidate video.

[0097] Optionally, the video category identifier may include identifiers for categories such as sports and music, and the video identifier may include video ID, video publisher ID, etc.

[0098] According to the above embodiments, the category identifier and / or video identifier of the processed video of the target object can guide the ranking model to score the candidate video, further improving the accuracy of score prediction.

[0099] Figure 2This disclosure illustrates a model training method for video recommendation according to an embodiment. This method can be applied to a model training apparatus for video recommendation, which can be deployed in an electronic device. The electronic device can be a single-machine or multi-machine terminal, server, or other processing device. The terminal can be a mobile device, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, or other user equipment (UE); the server can be a single-machine server or a server cluster. In some possible implementations, the method can also be implemented by a processor calling computer-readable instructions stored in memory. Figure 2 As shown, the method may include the following steps S210 to S230.

[0100] Step S210: Based on the first sample video in the first sample dataset and the score corresponding to the first sample video, perform training for a preset number of rounds to obtain the target model; wherein, the target model includes a multilayer perceptron and a ranking model. The multilayer perceptron is used to reduce the dimensionality of the multimodal semantic vector of the target video to obtain the target feature information of the target video, and the ranking model is used to determine the score of the target video based on the target feature information of the target video.

[0101] The model training method for video recommendation provided in this disclosure can train a target model for video recommendation methods, and the target model can be applied to any of the above-described video recommendation methods.

[0102] In this embodiment, firstly, based on the first sample video and its corresponding score in the first sample dataset, a multilayer perceptron and a ranking model are trained together. After a preset number of training rounds, the parameters of the multilayer perceptron and the ranking model converge to a preset standard, resulting in a target model. This target model includes the trained multilayer perceptron and ranking model. Based on this, the multilayer perceptron can be used to reduce the dimensionality of the multimodal semantic vectors of the target video, obtaining the target feature information of the target video. The ranking model is then used to determine the score of the target video based on the target feature information.

[0103] Optionally, the target video can be any video, such as a candidate video when a video recommendation request for the target is received.

[0104] Optionally, in step S210 above, training can be performed for a preset number of rounds based on the first sample video, the satisfactory video of a specific object (e.g., a specific user, the aforementioned target object), and the rating of the first sample video when making recommendations for that specific object. For example, the first sample video and the satisfactory video are respectively input into the multimodal model. After processing by the multimodal model and the multilayer perceptron, the target feature information of the first sample video and the target feature information of the satisfactory video are obtained. Then, the target feature information of the first sample video and the target feature information of the satisfactory video are input into the ranking model to obtain the model output rating. The loss is calculated based on the model output rating and the pre-obtained rating corresponding to the first sample video. The parameters of the multimodal model, the multilayer perceptron, and the ranking model are optimized based on this loss.

[0105] Step S220: Use a multilayer perceptron to determine the target feature information of the second sample video in the second sample dataset.

[0106] In this embodiment, the video content of the second sample video may be the same as or different from the video content of the first sample video. According to step S220 above, the parameters of the multilayer perceptron remain fixed, and the multilayer perceptron can be used to reduce the dimensionality of the second sample video in the second sample dataset to obtain the target feature information of the second sample video.

[0107] Optionally, there can be multiple second sample videos, and a multilayer perceptron can be used to determine the target feature information of each second sample video.

[0108] Step S230: Based on the target feature information of the second sample video and the score corresponding to the second sample video, train the ranking model online.

[0109] In this embodiment of the disclosure, the target feature information of the second sample video can be input into a preset model to obtain the score output by the ranking model. Based on the difference between the score corresponding to the second sample video and the score output by the ranking model, the ranking model is trained online.

[0110] Optionally, the second sample video can be a video determined based on historical recommendation actions after the target model goes live on the video recommendation platform. For example, the second sample video can be a candidate video processed during the process of determining the recommended video when a recommendation request is received from the target object. For example, the second sample video can include recommended videos.

[0111] Optionally, after the target model is launched, the target objects' ratings of the recommended videos can be collected, and the recommended videos can be used as second sample videos. The target objects' ratings of the recommended videos can be determined as the scores of the second sample videos. Based on the target feature information of the second sample videos and the scores corresponding to the second sample videos, the ranking model can be trained online.

[0112] The method provided in this embodiment of the present disclosure can keep the parameters of the multilayer perceptron obtained from training fixed. The multilayer perceptron can be directly used to reduce the dimensionality of the multimodal semantic vector of the second sample video to obtain low-dimensional target feature information. When the target model needs to be trained online, only the low-dimensional target feature information of the second sample video needs to be input into the ranking model to train the ranking model online. This can reduce the computational load of online training and the storage overhead of accumulating samples for online training, which is beneficial to improving the efficiency of model updates and the performance of the updated model.

[0113] To better understand the technical solutions of the embodiments of this disclosure, a specific application example is provided below, such as... Figure 3 As shown, CLIP is used to extract the cover image vector, title vector, and content summary vector of videos in the video library in advance, and then the multimodal semantic information of the videos is fused to obtain the multimodal semantic information of the videos. MLP is then used to reduce the dimensionality of the multimodal semantic information of the videos to obtain the target feature information of the videos in the video library. When a video recommendation request for a target object is received, the target feature information of two satisfactory videos of the target object and the target feature information of candidate videos can be extracted. The above target feature information, the vector features of other user ID features, and the vector features of context ID features are input into DIN. DIN can output the rating of the candidate videos.

[0114] The method provided in this embodiment can read pre-stored target feature information after receiving a video recommendation request for a target object. It can directly process the target feature information of satisfactory videos and candidate videos through DIN, without re-performing the repetitive calculation from multimodal semantic information to target feature information. This can determine the recommended video that fits the target object, reduce the computation and storage overhead during online inference, improve the efficiency of video recommendation, and improve the recommendation effect of new videos, thereby enhancing the video viewing experience of the target object in immersive scenarios.

[0115] According to embodiments of this disclosure, this disclosure also provides a video recommendation device. Figure 4 A schematic block diagram of a video recommendation device provided in an embodiment of this disclosure is shown, such as... Figure 4 As shown, the device includes:

[0116] The feature information extraction module 410 is used to extract the target feature information of the satisfactory video and the target feature information of the candidate video from a pre-configured target feature information set in response to receiving a video recommendation request for the target object; wherein, the target feature information set includes the target feature information of multiple videos in the video library, and the target feature information is obtained by dimensionality reduction based on the multimodal semantic information of its corresponding video;

[0117] The video scoring module 420 is used to determine the score of the candidate video based on the target feature information of the satisfactory video and the target feature information of the candidate video.

[0118] The recommended video determination module 430 is used to determine recommended videos for the target audience based on the scores of candidate videos.

[0119] In some embodiments, such as Figure 5 As shown, the video recommendation device may also include:

[0120] The vector acquisition module 510 is used to acquire the multimodal semantic vector of the video to be processed in the video library using a multimodal model;

[0121] The vector dimensionality reduction module 520 is used to reduce the dimensionality of multimodal semantic vectors using a multilayer perceptron to obtain the target feature information of the video to be processed;

[0122] Add a collection module 530 to add the target feature information of the video to be processed to the target feature information collection.

[0123] In some embodiments, the vector acquisition module 510 is further configured to:

[0124] A multimodal model is used to process one or more pieces of information from the video clips, titles, and summaries of the video to be processed, resulting in a multimodal semantic vector of the video.

[0125] In some embodiments, such as Figure 6 As shown, the video recommendation device may also include:

[0126] The Satisfactory Video Determination Module 610 is used to determine the satisfactory video of the target object from the video library based on the historical records of the target object.

[0127] In some embodiments, the video scoring module 420 is further configured to:

[0128] A ranking model is used to process the target feature information of satisfactory videos, the target feature information of candidate videos, and the identifier class features of target objects to obtain the scores of candidate videos.

[0129] In some embodiments, the identification class features include the category identifier and / or video identifier of the processed video of the target object.

[0130] According to embodiments of this disclosure, this disclosure also provides a model training apparatus for video recommendation. Figure 7 A schematic block diagram of a model training apparatus for video recommendation provided in an embodiment of this disclosure is shown, such as... Figure 7 As shown, the device includes:

[0131] The first model training module 710 is used to train the target model for a preset number of rounds based on the first sample video and the corresponding score in the first sample dataset. The target model includes a multilayer perceptron and a ranking model. The multilayer perceptron is used to reduce the dimensionality of the multimodal semantic vector of the target video to obtain the target feature information of the target video. The ranking model is used to determine the score of the target video based on the target feature information of the target video.

[0132] The feature information determination module 720 is used to determine the target feature information of the second sample video in the second sample dataset using a multilayer perceptron.

[0133] The second model training module 730 is used to train the ranking model online based on the target feature information of the second sample video and the score corresponding to the second sample video.

[0134] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0135] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0136] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0137] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0138] like Figure 8As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 807 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0139] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0140] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as video recommendation methods and / or model training methods for video recommendation. For example, in some embodiments, the video recommendation methods and / or model training methods for video recommendation can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by computing unit 801, one or more steps of the video recommendation method and / or model training method for video recommendation described above can be performed. Alternatively, in other embodiments, computing unit 801 can be configured to perform the video recommendation method and / or model training method for video recommendation by any other suitable means (e.g., by means of firmware).

[0141] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0142] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0143] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM), flash memory, optical fiber, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0144] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0145] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0146] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0147] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0148] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A video recommendation method, comprising: In response to receiving a video recommendation request for a target object, the system extracts target feature information of satisfactory videos and target feature information of candidate videos of the target object from a pre-configured target feature information set; wherein, the target feature information set includes target feature information of multiple videos in the video library, and the target feature information is obtained by dimensionality reduction based on the multimodal semantic information of its corresponding video; Based on the target feature information of the satisfactory video and the target feature information of the candidate video, the score of the candidate video is determined; Based on the ratings of the candidate videos, recommended videos are determined for the target audience.

2. The method according to claim 1, further comprising: A multimodal model is used to obtain the multimodal semantic vectors of the videos to be processed in the video library; The multi-layer perceptron is used to reduce the dimensionality of the multimodal semantic vector to obtain the target feature information of the video to be processed; The target feature information of the video to be processed is added to the target feature information set.

3. The method according to claim 2, wherein, The step of obtaining the multimodal semantic vector of the video to be processed in the video library using a multimodal model includes: A multimodal model is used to process one or more pieces of information from the video segments, titles, and summaries of the video to be processed, resulting in a multimodal semantic vector of the video to be processed.

4. The method according to any one of claims 1-3, further comprising: Based on the historical records of the target object, a satisfactory video of the target object is determined in the video library.

5. The method according to any one of claims 1-4, wherein, The step of determining the score of the candidate video based on the target feature information of the satisfactory video and the target feature information of the candidate video includes: A ranking model is used to process the target feature information of the satisfactory video, the target feature information of the candidate video, and the identifier class features of the target object to obtain the score of the candidate video.

6. The method according to claim 5, wherein, The identifier features include the category identifier and / or video identifier of the processed video of the target object.

7. A model training method for video recommendation, comprising: Based on the first sample video in the first sample dataset and the corresponding score of the first sample video, a target model is obtained by training for a preset number of rounds. The target model includes a multilayer perceptron and a ranking model. The multilayer perceptron is used to reduce the dimensionality of the multimodal semantic vector of the target video to obtain the target feature information of the target video. The ranking model is used to determine the score of the target video based on the target feature information of the target video. The multilayer perceptron is used to determine the target feature information of the second sample video in the second sample dataset; The ranking model is trained online based on the target feature information of the second sample video and the score corresponding to the second sample video.

8. A video recommendation device, comprising: The feature information extraction module is used to respond to a received video recommendation request for a target object by extracting target feature information of satisfactory videos and target feature information of candidate videos of the target object from a pre-configured target feature information set; wherein, the target feature information set includes target feature information of multiple videos in the video library, and the target feature information is obtained by dimensionality reduction based on the multimodal semantic information of its corresponding video; The video scoring module is used to determine the score of the candidate video based on the target feature information of the satisfactory video and the target feature information of the candidate video; The recommended video determination module is used to determine recommended videos for the target object based on the ratings of the candidate videos.

9. A model training device for video recommendation, comprising: The first model training module is used to train a target model for a preset number of rounds based on the first sample video in the first sample dataset and the score corresponding to the first sample video. The target model includes a multilayer perceptron and a ranking model. The multilayer perceptron is used to reduce the dimensionality of the multimodal semantic vector of the target video to obtain the target feature information of the target video. The ranking model is used to determine the score of the target video based on the target feature information of the target video. The feature information determination module is used to determine the target feature information of the second sample video in the second sample dataset using the multilayer perceptron; The second model training module is used to train the ranking model online based on the target feature information of the second sample video and the score corresponding to the second sample video.

10. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

11. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

12. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images