A short video personalized recommendation method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of recommendation and the technical field of artificial intelligence, and discloses a short video personalized recommendation method and device, an electronic device and a storage medium, the method comprising the following steps: determining semantic features of each historical short video; clustering the semantic features of each historical short video to generate a plurality of content clusters; generating a boredom increment of each viewing event of each content cluster through a boredom increment generation model; generating a final boredom value of each content cluster based on the boredom increment of each viewing event of each content cluster and a predefined boredom value generation model; generating a final recommendation score of each candidate short video based on the final boredom value of each content cluster and a predefined score model; sorting the final recommendation scores of each candidate short video; and grouping candidate short videos in the top N positions in the sorting to form a short video recommendation list of a user. The application can reduce the phenomenon of intensive pushing of similar content and enrich the diversity of the short video recommendation list.

Description

Technical Field

[0001] This application relates to the fields of recommendation technology and artificial intelligence technology, and in particular to a method, apparatus, electronic device and storage medium for personalized recommendation of short videos. Background Technology

[0002] With the popularization of mobile internet and the rapid development of multimedia technology, short video platforms have gradually become the core carrier of information dissemination and content consumption. Faced with massive and rapidly growing user-generated video content, personalized recommendation systems have become a key technical means to alleviate information overload and improve user experience.

[0003] Existing short video recommendation methods tend to continuously push similar content to videos already watched based on a user's historical click history. As users spend extended periods within the same content cluster, their viewing time and engagement gradually decrease. However, existing methods fail to detect users' level of boredom with the content cluster, making it difficult to match recommendations to users' current needs and resulting in insufficient accuracy. Therefore, how to generate a short video recommendation list based on users' level of boredom with content clusters has become a pressing technical problem to be solved in this field. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and storage medium for personalized short video recommendations, in order to solve the aforementioned technical problem of how to generate a user's short video recommendation list based on the user's level of boredom with content clusters.

[0005] In a first aspect, embodiments of this application provide a personalized recommendation method for short videos, applied to electronic devices, the personalized recommendation method for short videos comprising: A training sample is formed by combining the multimodal data of the preset short video and the semantic features of the preset short video. A multimodal large model is trained based on the training sample. The first model is used to generate the repulsion loss of the multimodal large model, and the second model is used to generate the contrastive loss of the multimodal large model. The multimodal large model generates a comprehensive loss value by using the repulsion loss, contrastive loss, and predefined comprehensive loss model of the multimodal large model. When the comprehensive loss value is less than the preset loss value, the training of the multimodal large model is stopped, and the trained multimodal large model is saved. Based on the user's historical viewing sequence and the trained multimodal large model, the semantic features of each historically viewed short video are determined; The semantic features of each historical short video are clustered to generate multiple content clusters. The boredom increment generation model is used to generate the boredom increment of each content cluster for each viewing event. Based on the boredom increment of each viewing event for each content cluster and a predefined boredom value generation model, the final boredom value for each content cluster is generated. Based on the final boredom value for each content cluster and a predefined scoring model, the final recommendation score for each candidate short video is generated. The final recommendation scores for each candidate short video are sorted, and the top N candidate short videos are combined into the user's short video recommendation list. The first model is used to calculate the repulsion loss of a large multimodal model; the second model is used to calculate the contrastive loss of a large multimodal model. The rejection loss is the loss that constrains the ability of a multimodal large model to distinguish features between different samples within the same topic content. The smaller the rejection loss, the stronger the ability of the multimodal large model to distinguish features between different samples within the same topic content; the larger the rejection loss, the weaker the ability of the multimodal large model to distinguish features between different samples within the same topic content. Contrast loss is the loss that constrains the ability of a multimodal large model to distinguish between positive and negative sample pairs; the smaller the contrast loss, the better the contrast learning effect of the multimodal large model; the larger the contrast loss, the worse the contrast learning effect of the multimodal large model. The comprehensive loss model refers to a loss function that sums the contrast loss and the rejection loss after being weighted by a preset adjustment coefficient. The boredom value generation model is a model that generates the final boredom value of each content cluster based on the original boredom value of each content cluster. The original boredom value is the unadjusted level of boredom; the final boredom value is the adjusted level of boredom. The boredom increment for each viewing event of a content cluster refers to the increase in boredom for the current viewing event relative to the previous viewing event for each content cluster.

[0006] In one possible implementation of the first aspect, a training sample is formed by combining the multimodal data of a preset short video and the semantic features of the preset short video. A multimodal large model is trained based on the training sample. A first model is used to generate the repulsion loss of the multimodal large model, and a second model is used to generate the contrastive loss of the multimodal large model, including: Acquire multimodal data of a preset short video, which includes the keyframe sequence of the preset short video, the Mel spectrogram of the preset short video, and the text string of the preset short video; A training sample is formed by combining the multimodal data and semantic features of a preset short video. Different training samples are combined into a training set. A multimodal large model is trained based on the training set. The repulsion loss of the multimodal large model is generated through the first model, and the contrastive loss of the multimodal large model is generated through the second model.

[0007] In one possible implementation of the first aspect, semantic features of each historically viewed short video are determined based on the user's historical viewing sequence and a trained multimodal large model, including: Multiple historical short videos are obtained from the user's historical viewing sequence. Multimodal data of the historical short videos is obtained, including keyframe sequences, Mel spectrograms, and text strings of the historical short videos. Feature extraction is performed on the keyframe sequence of the historically viewed short videos to obtain the features of the keyframe sequence of the short videos. Feature extraction is performed on the Mel spectrogram of the historically viewed short videos to obtain the features of the Mel spectrogram of the historically viewed short videos. Feature extraction is performed on the text string of the historically viewed short videos to obtain the features of the text string of the historically viewed short videos. The features of keyframe sequences, Mel spectrograms, and text strings from historically viewed short videos are fused to obtain the current features. These current features are then processed by the feature extractor in the trained multimodal large model to generate semantic features of the historically viewed short videos.

[0008] In one possible implementation of the first aspect, based on the boredom increment of each viewing event for each content cluster and a predefined boredom value generation model, a final boredom value for each content cluster is generated. Based on the final boredom value of each content cluster and a predefined scoring model, a final recommendation score for each candidate short video is generated. The final recommendation scores of each candidate short video are sorted, and the top N candidate short videos are combined into a user's short video recommendation list, including: The event sequence for each content cluster is generated by concatenating the boredom increment, completion rate, quick swipe indicator, negative feedback indicator, and repeated browsing indicator for each viewing event of each content cluster. The event sequence of each content cluster is input into the Transformer encoder in the trained multimodal large model. The Transformer encoder generates the encoding sequence of each content cluster. The encoding sequence of each content cluster is input into the fully connected layer in the trained multimodal large model. The fully connected layer generates the original boredom value of each content cluster. Based on the original boredom value and the predefined boredom value of each content cluster, the model is generated to generate the final boredom value of each content cluster. The final boredom value of each content cluster is mapped to the cluster identifier of each content cluster. The system retrieves a candidate list of users from the short video platform, obtains the cluster identifier corresponding to each candidate short video from the candidate list, retrieves the final boredom value of the content cluster to which each candidate short video belongs from the mapping table based on the cluster identifier of each candidate short video, generates the final recommendation score for each candidate short video by combining the final boredom value of the content cluster to which each candidate short video belongs with a predefined scoring model, sorts the final recommendation scores of each candidate short video, and forms the user's short video recommendation list by selecting the top N candidate short videos, where N is a positive integer.

[0009] In one possible implementation of the first aspect, the first model is defined as follows: ; This represents the repulsive loss of a large multimodal model. This represents the set of training samples used in a single training iteration. i represents any training sample in the training sample set; This represents the set of nearest neighbor samples that belong to the same topic as training sample i; the larger the nearest neighbor set, the more nearest neighbors there are to training sample i; the smaller the nearest neighbor set, the fewer nearest neighbors there are to training sample i. j represents any nearest neighbor sample in the nearest neighbor sample set; This represents the feature vector of training sample i after projection and normalization; This represents the feature vector of the nearest neighbor sample j after projection and normalization; This represents the dot product of the feature vectors of training sample i and its nearest neighbor sample j; Indicates the rejection threshold; The larger the value, the more lenient the conditions for triggering the rejection penalty; The smaller the value, the stricter the conditions for triggering the rejection penalty; The hinge truncation term is used to penalize the training sample i when the dot product of the feature vectors of the training sample i and the nearest neighbor sample j exceeds the repulsion threshold; the hinge truncation term does not penalize the training sample i when the dot product of the feature vectors of the training sample i and the nearest neighbor sample j does not exceed the repulsion threshold.

[0010] In one possible implementation of the first aspect, the second model is defined as follows: ; This represents the contrastive loss of a large multimodal model. This represents the set of training samples used in a single training iteration. i represents any training sample in the training sample set; This represents the set of positive samples corresponding to training sample i; the larger the set of positive samples, the more samples that form a positive sample pair with training sample i; the smaller the set of positive samples, the fewer samples that form a positive sample pair with training sample i. p represents any positive sample in the set of positive samples; m represents the remaining samples in the training sample set excluding training sample i; Indicates the temperature coefficient; This represents the feature vector of training sample i after projection and normalization; This represents the feature vector of a positive sample p after projection and normalization; exp is an exponential function with base e; The comprehensive loss model is defined as follows: ; This represents the overall loss of a large multimodal model. This represents the contrastive loss of a large multimodal model. This represents the repulsive loss of a large multimodal model. This represents the adjustment coefficient for rejection loss, used to adjust the degree of influence of rejection loss on overall loss.

[0011] In one possible implementation of the first aspect, the incremental generation model is weary of the following definition: ; This represents the boredom increment of the k-th viewing event for the c-th content cluster; The higher the value, the greater the user's level of boredom with watching specific short videos; The lower the value, the lower the user's level of boredom with the specified short video; watching a short video refers to the historical short video watched corresponding to the k-th viewing event of the c-th content cluster. This represents the completion rate of the k-th view event for the c-th content cluster; The quick swipe indicator represents the k-th view event of the c-th content cluster; ; When the value is 1, it means that the dwell time of the k-th viewing event of the c-th content cluster is less than 2 seconds; When the value is 0, it means that the dwell time of the k-th viewing event of the c-th content cluster is not less than 2 seconds; The negative feedback indicator represents the k-th view event of the c-th content cluster; ; When the value is 1, it means that in the k-th view event of the c-th content cluster, the user clicked on a control that was not of interest. A value of 0 indicates that in the k-th view event of the c-th content cluster, the user did not click on any controls they were not interested in. This is a repeat browsing indicator for the k-th view event of the c-th content cluster; ; When the value is 1, it indicates that the short video being watched is a repeatedly viewed content; repeatedly viewed content refers to short videos that the same user has watched within 24 hours prior to this viewing time. A value of 0 indicates that the short video being watched is not considered content that has been viewed repeatedly. , , , These are the preset first weight coefficient, second weight coefficient, third weight coefficient, and fourth weight coefficient; The boredom value generation model is defined as follows: ; This represents the final boredom value of the Cth content cluster; The higher the value, the greater the user's ultimate level of boredom with the Cth content cluster; The lower the value, the lower the user's final level of boredom with the Cth content cluster; This represents the original boredom value of the Cth content cluster; The higher the value, the greater the user's initial aversion to the Cth content cluster; The lower the value, the lower the user's initial level of boredom with the Cth content cluster; Describes the minimum value function; This indicates the preset maximum aversion value; The scoring model is defined as follows: ; This represents the original recommendation score of the Vth candidate short video; This represents the final recommendation score of the Vth candidate short video; This represents the final boredom value of the content cluster to which the Vth candidate short video belongs; This represents the exponential decay factor.

[0012] Secondly, embodiments of this application provide a short video personalized recommendation device, applied to electronic devices, including: The first generation module is used to combine the multimodal data of the preset short video and the semantic features of the preset short video into a training sample, train a multimodal large model based on the training sample, generate the repulsion loss of the multimodal large model through the first model, and generate the contrastive loss of the multimodal large model through the second model. The second generation module is used to generate the comprehensive loss value of the multimodal large model through the repulsion loss of the multimodal large model, the contrastive loss of the multimodal large model and the predefined comprehensive loss model. When the comprehensive loss value is less than the preset loss value, the training of the multimodal large model is stopped and the trained multimodal large model is saved. The third generation module is used to determine the semantic features of each historically viewed short video based on the user's historical viewing sequence and the trained multimodal large model; The clustering module is used to cluster the semantic features of each historically viewed short video to generate multiple content clusters. The boredom increment generation model generates the boredom increment of each content cluster for each viewing event. The recommendation module generates a final boredom value for each content cluster based on the boredom increment of each viewing event and a predefined boredom value generation model. Based on the final boredom value and a predefined scoring model, it generates a final recommendation score for each candidate short video. The final recommendation scores of each candidate short video are then sorted, and the top N short videos are combined into a user's short video recommendation list. The first model is used to calculate the rejection loss of the multimodal large model; the second model is used to calculate the contrast loss of the multimodal large model. The rejection loss is the loss that constrains the ability of a multimodal large model to distinguish features between different samples within the same topic content. The smaller the rejection loss, the stronger the ability of the multimodal large model to distinguish features between different samples within the same topic content; the larger the rejection loss, the weaker the ability of the multimodal large model to distinguish features between different samples within the same topic content. Contrast loss is the loss that constrains the ability of a multimodal large model to distinguish between positive and negative sample pairs; the smaller the contrast loss, the better the contrast learning effect of the multimodal large model; the larger the contrast loss, the worse the contrast learning effect of the multimodal large model. The comprehensive loss model refers to a loss function that sums the contrast loss and the rejection loss after being weighted by a preset adjustment coefficient. The boredom value generation model is a model that generates the final boredom value of each content cluster based on the original boredom value of each content cluster. The original boredom value is the unadjusted level of boredom; the final boredom value is the adjusted level of boredom. The boredom increment of each viewing event for a content cluster refers to the increase in boredom level for the current viewing event relative to the previous viewing event for each content cluster. Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the short video personalized recommendation method described in the first aspect above.

[0013] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the short video personalized recommendation method described in the first aspect above.

[0014] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to execute the short video personalized recommendation method described in the first aspect above.

[0015] The beneficial effects of the embodiments of this application are as follows: Firstly, based on the boredom increment of each viewing event for each content cluster and a predefined boredom value generation model, a final boredom value for each content cluster is generated. Based on the final boredom value of each content cluster and a predefined scoring model, a final recommendation score for each candidate short video is generated. The final recommendation scores of each candidate short video are sorted, and the top N candidate short videos are combined into a short video recommendation list for the user. In this way, the final boredom value of each content cluster can quantify the user's degree of boredom with each content cluster. Based on the final boredom value of each content cluster and the predefined scoring model, the final recommendation score of each candidate short video can be dynamically adjusted, reducing the phenomenon of dense push of similar content, enriching the diversity of the short video recommendation list, and meeting the user's current real needs, which is conducive to improving the recommendation accuracy of candidate short videos. Secondly, since the user's short video recommendation list is automatically generated and is not affected by human intervention, it helps to improve the reliability and stability of the user's short video recommendation list. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the personalized short video recommendation method provided in the embodiments of this application; Figure 2 A schematic block diagram of a short video personalized recommendation device provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0019] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0020] It should be understood that in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance. The terms "comprising," "including," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0021] Furthermore, the technical solutions of the various embodiments can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0022] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the described order. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0023] The personalized short video recommendation method provided in this application can be applied to electronic devices, including but not limited to servers, mobile phones, tablets, wearable devices, in-vehicle devices, and laptops. This application does not impose any restrictions on the specific type of electronic device.

[0024] Please see Figure 1 , Figure 1 This is a flowchart illustrating the personalized recommendation method for short videos provided in this application embodiment, which can be applied to electronic devices.

[0025] like Figure 1 As shown in the embodiments of this application, the personalized recommendation method for short videos includes the following steps, which are detailed below: S101, combine the multimodal data of the preset short video and the semantic features of the preset short video into a training sample, train a multimodal large model based on the training sample, generate the repulsion loss of the multimodal large model through the first model, and generate the contrastive loss of the multimodal large model through the second model. The training sample is composed of multimodal data and semantic features of preset short videos. A multimodal large model is trained based on the training sample. The first model generates the repulsion loss of the multimodal large model, and the second model generates the contrastive loss of the multimodal large model, including: Acquire multimodal data of a preset short video, which includes the keyframe sequence of the preset short video, the Mel spectrogram of the preset short video, and the text string of the preset short video; A training sample is formed by combining the multimodal data and semantic features of a preset short video. Different training samples are combined into a training set. A multimodal large model is trained based on the training set. The repulsion loss of the multimodal large model is generated through the first model, and the contrastive loss of the multimodal large model is generated through the second model.

[0026] The first model is defined as follows: ; This represents the repulsive loss of a large multimodal model. This represents the set of training samples used in a single training iteration. i represents any training sample in the training sample set; This represents the set of nearest neighbor samples that belong to the same topic as training sample i; the larger the nearest neighbor set, the more nearest neighbors there are to training sample i; the smaller the nearest neighbor set, the fewer nearest neighbors there are to training sample i. j represents any nearest neighbor sample in the nearest neighbor sample set; This represents the feature vector of training sample i after projection and normalization; This represents the feature vector of the nearest neighbor sample j after projection and normalization; This represents the dot product of the feature vectors of training sample i and its nearest neighbor sample j; Indicates the rejection threshold; The larger the value, the more lenient the conditions for triggering the rejection penalty; The smaller the value, the stricter the conditions for triggering the rejection penalty; The hinge truncation term is used to penalize the training sample i when the dot product of the feature vectors of the training sample i and the nearest neighbor sample j exceeds the repulsion threshold; the hinge truncation term does not penalize the training sample i when the dot product of the feature vectors of the training sample i and the nearest neighbor sample j does not exceed the repulsion threshold.

[0027] The second model is defined as follows: ; This represents the contrastive loss of a large multimodal model. This represents the set of training samples used in a single training iteration. i represents any training sample in the training sample set; This represents the set of positive samples corresponding to training sample i; the larger the set of positive samples, the more samples that form a positive sample pair with training sample i; the smaller the set of positive samples, the fewer samples that form a positive sample pair with training sample i. p represents any positive sample in the set of positive samples; m represents the remaining samples in the training sample set excluding training sample i; Indicates the temperature coefficient; This represents the feature vector of training sample i after projection and normalization; This represents the feature vector of a positive sample p after projection and normalization; exp is an exponential function with base e.

[0028] e is the natural constant.

[0029] The training samples consist of modal data and semantic features of preset short videos. These preset short videos are existing short videos in the platform's short video library, which can be randomly selected or sampled according to content category. For each preset short video, the corresponding modal data and semantic features are extracted and paired to form a set of training samples. Multiple sets of samples together constitute the training sample set.

[0030] Multimodal large models refer to neural network models capable of processing two or more modalities of data and pre-trained on large-scale data. Multimodal large models include, but are not limited to, the Doubao large model and the CLIP model. The CLIP model is a contrastive language-image pre-trained model, containing a text encoder and an image encoder. Both the Doubao large model and the CLIP model contain a Transformer encoder.

[0031] S102: Generate the comprehensive loss value of the multimodal large model by using the repulsion loss, the contrastive loss, and the predefined comprehensive loss model of the multimodal large model. When the comprehensive loss value is less than the preset loss value, stop training the multimodal large model and save the trained multimodal large model. The comprehensive loss model is defined as follows: ; This represents the overall loss of a large multimodal model. This represents the contrastive loss of a large multimodal model. This represents the repulsive loss of a large multimodal model. This represents the adjustment coefficient for rejection loss, used to adjust the degree of influence of rejection loss on overall loss.

[0032] S103, based on the user's historical viewing sequence and the trained multimodal large model, determine the semantic features of each historically viewed short video; Specifically, based on the user's historical viewing sequence and the trained multimodal large model, the semantic features of each historically viewed short video are determined, including: Multiple historical short videos are obtained from the user's historical viewing sequence. Multimodal data of the historical short videos is obtained, including keyframe sequences, Mel spectrograms, and text strings of the historical short videos. Feature extraction is performed on the keyframe sequence of the historically viewed short videos to obtain the features of the keyframe sequence of the short videos. Feature extraction is performed on the Mel spectrogram of the historically viewed short videos to obtain the features of the Mel spectrogram of the historically viewed short videos. Feature extraction is performed on the text string of the historically viewed short videos to obtain the features of the text string of the historically viewed short videos. The features of keyframe sequences, Mel spectrograms, and text strings from historically viewed short videos are fused to obtain the current features. These current features are then processed by the feature extractor in the trained multimodal large model to generate semantic features of the historically viewed short videos.

[0033] S104: Cluster the semantic features of each historical short video to generate multiple content clusters. Then, generate the boredom increment of each content cluster for each viewing event through the boredom increment generation model. Clustering algorithms are used to cluster the semantic features of various historically viewed short videos, generating multiple content clusters. Clustering algorithms include density-based clustering and hierarchical clustering. Density-based and hierarchical clustering are common algorithms and will not be elaborated upon here.

[0034] S105: Based on the boredom increment of each viewing event for each content cluster and a predefined boredom value generation model, generate the final boredom value for each content cluster. Based on the final boredom value of each content cluster and a predefined scoring model, generate the final recommendation score for each candidate short video. Sort the final recommendation scores of each candidate short video and form the user's short video recommendation list with the top N ranked candidate short videos. The first model is a model used to calculate the rejection loss of the multimodal large model. The second model is a model used to calculate the contrast loss of the multimodal large model. The rejection loss is the loss that constrains the ability of a multimodal large model to distinguish features between different samples within the same topic content. The smaller the rejection loss, the stronger the ability of the multimodal large model to distinguish features between different samples within the same topic content; the larger the rejection loss, the weaker the ability of the multimodal large model to distinguish features between different samples within the same topic content. Contrast loss is the loss that constrains the ability of a multimodal large model to distinguish between positive and negative sample pairs; the smaller the contrast loss, the better the contrast learning effect of the multimodal large model; the larger the contrast loss, the worse the contrast learning effect of the multimodal large model. The comprehensive loss model refers to a loss function that sums the contrast loss and the rejection loss after being weighted by a preset adjustment coefficient. The boredom value generation model is a model that generates the final boredom value of each content cluster based on the original boredom value of each content cluster. The original boredom value is the unadjusted level of boredom; the final boredom value is the adjusted level of boredom. The boredom increment for each viewing event of a content cluster refers to the increase in boredom for the current viewing event relative to the previous viewing event for each content cluster.

[0035] For ease of explanation, let's take a specific content cluster as an example. Suppose a user watches the c-th content cluster multiple times. At the end of the (k-1)-th watch, the corresponding level of boredom for this content cluster is 0.3; at the end of the k-th watch, the corresponding level of boredom for this content cluster is 0.5. Then, the increase in the level of boredom for the k-th watch event compared to the (k-1)-th watch event is: that is, the increase in boredom for the c-th content cluster at the k-th watch event is 0.2.

[0036] Among them, the incremental generation model is defined as follows: ; This represents the boredom increment of the k-th viewing event for the c-th content cluster; The higher the value, the greater the user's level of boredom with watching specific short videos; The lower the value, the lower the user's level of boredom with the specified short video; watching a short video refers to the historical short video watched corresponding to the k-th viewing event of the c-th content cluster. This represents the completion rate of the k-th view event for the c-th content cluster; The quick swipe indicator represents the k-th view event of the c-th content cluster; ; When the value is 1, it means that the dwell time of the k-th viewing event of the c-th content cluster is less than 2 seconds; When the value is 0, it means that the dwell time of the k-th viewing event of the c-th content cluster is not less than 2 seconds; The negative feedback indicator represents the k-th view event of the c-th content cluster; ; When the value is 1, it means that in the k-th view event of the c-th content cluster, the user clicked on a control that was not of interest. A value of 0 indicates that in the k-th view event of the c-th content cluster, the user did not click on any controls they were not interested in. This is a repeat browsing indicator for the k-th view event of the c-th content cluster; ; When the value is 1, it indicates that the short video being watched is a repeatedly viewed content; repeatedly viewed content refers to short videos that the same user has watched within 24 hours prior to this viewing time. A value of 0 indicates that the short video being watched is not considered content that has been viewed repeatedly. , , , These are the preset first weight coefficient, second weight coefficient, third weight coefficient, and fourth weight coefficient; The boredom value generation model is defined as follows: ; This represents the final boredom value of the Cth content cluster; The higher the value, the greater the user's ultimate level of boredom with the Cth content cluster; The lower the value, the lower the user's final level of boredom with the Cth content cluster; This represents the original boredom value of the Cth content cluster; The higher the value, the greater the user's initial aversion to the Cth content cluster; The lower the value, the lower the user's initial level of boredom with the Cth content cluster; Describes the minimum value function; This indicates the preset maximum aversion value; The scoring model is defined as follows: ; This represents the original recommendation score of the Vth candidate short video; This represents the final recommendation score of the Vth candidate short video; This represents the final boredom value of the content cluster to which the Vth candidate short video belongs; This represents the exponential decay factor.

[0037] Specifically, based on the boredom increment of each viewing event for each content cluster and a predefined boredom value generation model, a final boredom value is generated for each content cluster. Based on the final boredom value of each content cluster and a predefined scoring model, a final recommendation score is generated for each candidate short video. The final recommendation scores of each candidate short video are sorted, and the top N candidate short videos are combined into a user's short video recommendation list, including: The event sequence for each content cluster is generated by concatenating the boredom increment, completion rate, quick swipe indicator, negative feedback indicator, and repeated browsing indicator for each viewing event of each content cluster. The event sequence of each content cluster is input into the Transformer encoder in the trained multimodal large model. The Transformer encoder generates the encoding sequence of each content cluster. The encoding sequence of each content cluster is input into the fully connected layer in the trained multimodal large model. The fully connected layer generates the original boredom value of each content cluster. Based on the original boredom value and the predefined boredom value of each content cluster, the model is generated to generate the final boredom value of each content cluster. The final boredom value of each content cluster is mapped to the cluster identifier of each content cluster. The system retrieves a candidate list of users from the short video platform, obtains the cluster identifier corresponding to each candidate short video from the candidate list, retrieves the final boredom value of the content cluster to which each candidate short video belongs from the mapping table based on the cluster identifier of each candidate short video, generates the final recommendation score for each candidate short video by combining the final boredom value of the content cluster to which each candidate short video belongs with a predefined scoring model, sorts the final recommendation scores of each candidate short video, and forms the user's short video recommendation list by selecting the top N candidate short videos, where N is a positive integer.

[0038] Specifically, the encoded sequence of each content cluster is input into the fully connected layer of the trained multimodal large model. The fully connected layer generates the original boredom value for each content cluster, including: The encoded sequence of each content cluster is input into the fully connected layer of the trained multimodal large model. The fully connected layer reads the encoded sequence of each content cluster and performs matrix multiplication with the weight matrix of the fully connected layer to obtain the weighted feature vector of each content cluster. The weighted feature vector of each content cluster is then added element-wise with the preset bias vector of the fully connected layer to obtain the linear feature vector corresponding to each content cluster. The ReLU function performs a non-linear transformation on the linear feature vector corresponding to each content cluster to generate the activation feature vector corresponding to each content cluster. The activation feature vectors corresponding to each content cluster are then averaged and aggregated to generate the original fatigue value of each content cluster. The ReLU function stands for Linear Rectified Flow Function.

[0039] Based on the cluster identifier corresponding to each candidate short video, the final fatigue value of the content cluster to which each candidate short video belongs is obtained from the mapping table. For ease of explanation, an example is as follows: For example, if the cluster identifier of the candidate short video is "pet", the final boredom value of the pet category is obtained from the mapping table, and the final boredom value of the pet category is selected as the final boredom value of the content cluster to which the candidate short video belongs.

[0040] For example, if the candidate short video is clustered as food, the final boredom value of the pet category is obtained from the mapping table, and the final boredom value of the food category is selected as the final boredom value of the content cluster to which the candidate short video belongs.

[0041] Preset short videos refer to the short videos used in the training phase of multimodal large models. The larger the number of preset short videos, the richer the content types covered by the training data and the more comprehensive the semantic representations learned by the multimodal large models. The smaller the number of preset short videos, the more limited the content types covered by the training data and the more singular the semantic representations learned by the multimodal large models.

[0042] Historical short video viewing refers to short videos that a user has watched before the current recommendation time. The larger the number of historical short video viewings, the more comprehensive the behavioral data that can be used to characterize user interests and preferences; the smaller the number of historical short video viewings, the more limited the behavioral data that can be used to characterize user interests and preferences.

[0043] Candidate short videos refer to short videos in the candidate list that have not yet been recommended to users; the larger the number of candidate short videos, the wider the range of content available for filtering and recommendation at the current moment; the smaller the number of candidate short videos, the narrower the range of content available for filtering and recommendation at the current moment.

[0044] The final recommendation scores of each candidate short video are sorted, and the top N candidate short videos are combined into a user's short video recommendation list, including: Sort the candidate short videos by their final recommendation scores, perform a greedy rearrangement on the top N candidate short videos, and use the rearranged top N candidate short videos as the final recommendation list.

[0045] Greedy reordering refers to a successive selection strategy used in constructing the final recommendation list. In each round, the best video is selected from the candidate set based on a comprehensive scoring function and added to the final recommendation list. After each selected candidate video, the comprehensive score of the remaining candidates is updated in real time, and this process is iterated until the final recommendation list reaches a preset length. The candidate set is the collection of the top N ranked candidate videos. Preferably, N is 20.

[0046] The comprehensive scoring function is defined as follows: ; This represents the candidate short video selected in step t; The candidate set is a collection of the top N short videos in the ranking. This is the candidate set from the previous step; This represents the final recommendation score of the Vth candidate short video; The content cluster to which the Vth candidate short video belongs; Selected short videos The content cluster to which it belongs; For indicator functions; The difference between the current location and the location index of the nearest similar video; 3 is the distance decay factor, and 3.0 is the decay constant, meaning that the penalty decays to approximately every 3 positions. ; 0.7 and 0.3 are balancing coefficients, which control the weights of the scoring items and the diversity penalty items, respectively.

[0047] Among them, a greedy rearrangement is performed on the top N candidate short videos. This greedy selection process introduces diversity constraints, optimizes the distribution ratio of various types of content, and balances recommendation relevance and content diversity.

[0048] The interaction metrics for each viewing event within each content cluster include completion rate, quick swipe indicator, negative feedback indicator, and repeat browsing indicator. The acquisition process for each of these metrics is explained below: The completion rate is calculated from the playback logs reported by the client player. When a user plays a video using the client, the player reports the current playback progress to the server at fixed time intervals, which can be set to every second. For the k-th viewing event within the c-th content cluster, the electronic device extracts two key fields from the playback logs: the cumulative playback duration of that viewing and the total duration of the video content. The completion rate is calculated by dividing the cumulative playback duration of that viewing by the total duration of the video content; the quotient is the completion rate. The completion rate ranges from 0 to 1. When a user watches the entire video, the completion rate is 1; when a user only watches the beginning of the video, the completion rate approaches 0.

[0049] In the specific implementation, the cumulative playback duration is taken from the playback progress value last reported by the user in the current viewing event, and the total video duration is obtained from the video content metadata.

[0050] The quick swipe indicator is calculated from the playback start and playback termination logs reported by the client player.

[0051] When a user clicks to play a video on the client side, the player records the start time of playback and reports it to the server. When the user leaves the current playback page or switches to the next video, the player records the end time of playback and reports it to the server. The electronic device calculates the dwell time based on the start and end times of the same video.

[0052] The specific method for determining the quick swipe indicator is as follows: when the dwell time is less than a preset time threshold, the viewing session is marked as a quick swipe, and the quick swipe indicator value is 1. When the dwell time is greater than or equal to the preset time threshold, the quick swipe indicator value is 0. In a preferred embodiment, the preset time threshold is 2 seconds.

[0053] In practice, if a user swipes away from the playback page in less than 2 seconds, it indicates that the user lacks even the most basic interest in watching the video content. Therefore, this behavior is considered an important signal of boredom.

[0054] The negative feedback indicator is obtained through user feedback logs reported by the client's interactive interface.

[0055] The client's playback interface provides a feedback entry point, through which users can perform negative feedback actions, including but not limited to clicking the "not interested" button and clicking the "block creator" button. When a user performs any of the above negative feedback actions, the client generates a corresponding negative feedback log and reports it to the server.

[0056] The specific determination method for the negative feedback indicator is as follows: when the electronic device receives a negative feedback log from the user regarding the currently viewed video, the negative feedback indicator for that viewing event is set to 1. When the electronic device does not receive a negative feedback log for the currently viewed video within a preset response time, the negative feedback indicator is set to 0. In the specific implementation, this preset response time can be a fixed time window after the end of the viewing event, such as 30 seconds, or it can be the entire time until the user performs the next viewing action.

[0057] Negative feedback represents the user's most explicit expression of rejection towards video content, and its indicator value directly reflects whether the user has a negative attitude towards the content.

[0058] The repeat viewing indicator is calculated by comparing the current viewing event with the video identifiers in the historical viewing record.

[0059] The electronic device records each user's entire viewing history, with each viewing record containing at least a video identifier and a viewing time. When the k-th viewing event occurs, the electronic device obtains the video identifier for that viewing event and checks the user's viewing history to see if that video identifier has been viewed by the user within a preset time window prior to this viewing time.

[0060] The specific method for determining the repeat viewing indicator is as follows: when there is a record in the historical viewing records that has the same video identifier as the currently viewed video and whose viewing time is within a preset time window before the current viewing time, the repeat viewing indicator is set to 1; otherwise, it is set to 0. In a preferred embodiment, the preset time window is 24 hours.

[0061] The beneficial effects of the embodiments of this application are as follows: Firstly, based on the boredom increment of each viewing event for each content cluster and a predefined boredom value generation model, a final boredom value for each content cluster is generated. Based on the final boredom value of each content cluster and a predefined scoring model, a final recommendation score for each candidate short video is generated. The final recommendation scores of each candidate short video are sorted, and the top N candidate short videos are combined into a short video recommendation list for the user. In this way, the final boredom value of each content cluster can quantify the user's degree of boredom with each content cluster. Based on the final boredom value of each content cluster and the predefined scoring model, the final recommendation score of each candidate short video can be dynamically adjusted, reducing the phenomenon of dense push of similar content, enriching the diversity of the short video recommendation list, and meeting the user's current real needs, which is conducive to improving the recommendation accuracy of candidate short videos. Secondly, since the user's short video recommendation list is automatically generated and is not affected by human intervention, it helps to improve the reliability and stability of the user's short video recommendation list.

[0062] For the short video personalized recommendation method described in the above embodiments, please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic block diagram of a short video personalized recommendation device provided in an embodiment of this application. Figure 2 The short video personalized recommendation device 200 shown can be applied to, for example... Figure 1 The application scenario diagram shows electronic devices. The following section uses electronic devices as an example to illustrate this. Figure 2 The short video personalized recommendation device 200 shown will be described in detail. The short video personalized recommendation device 200 may include a first generation module 201, a second generation module 202, a third generation module 203, a clustering module 204, and a recommendation module 205.

[0063] The first generation module 201 is used to combine the multimodal data of the preset short video and the semantic features of the preset short video into a training sample, train a multimodal large model based on the training sample, generate the repulsion loss of the multimodal large model through the first model, and generate the contrastive loss of the multimodal large model through the second model. The second generation module 202 is used to generate the comprehensive loss value of the multimodal large model through the repulsion loss of the multimodal large model, the contrastive loss of the multimodal large model and the predefined comprehensive loss model. When the comprehensive loss value is less than the preset loss value, the training of the multimodal large model is stopped and the trained multimodal large model is saved. The third generation module 203 is used to determine the semantic features of each historically viewed short video based on the user's historical viewing sequence and the trained multimodal large model; Clustering module 204 is used to cluster the semantic features of each historically viewed short video to generate multiple content clusters. Through the boredom increment generation model, the boredom increment of each content cluster for each viewing event is generated. The recommendation module 205 is used to generate a final boredom value for each content cluster based on the boredom increment of each viewing event and a predefined boredom value generation model. Based on the final boredom value of each content cluster and a predefined scoring model, it generates a final recommendation score for each candidate short video. The final recommendation scores of each candidate short video are sorted, and the top N candidate short videos are combined into a short video recommendation list for the user. The first model is a model used to calculate the rejection loss of the multimodal large model; the second model is a model used to calculate the contrast loss of the multimodal large model. The rejection loss is the loss that constrains the ability of a multimodal large model to distinguish features between different samples within the same topic content. The smaller the rejection loss, the stronger the ability of the multimodal large model to distinguish features between different samples within the same topic content; the larger the rejection loss, the weaker the ability of the multimodal large model to distinguish features between different samples within the same topic content. Contrast loss is the loss that constrains the ability of a multimodal large model to distinguish between positive and negative sample pairs; the smaller the contrast loss, the better the contrast learning effect of the multimodal large model; the larger the contrast loss, the worse the contrast learning effect of the multimodal large model. The comprehensive loss model refers to a loss function that sums the contrast loss and the rejection loss after being weighted by a preset adjustment coefficient. The boredom value generation model is a model that generates the final boredom value of each content cluster based on the original boredom value of each content cluster. The original boredom value is the unadjusted level of boredom; the final boredom value is the adjusted level of boredom. The boredom increment for each viewing event of a content cluster refers to the increase in boredom for the current viewing event relative to the previous viewing event for each content cluster.

[0064] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0065] The beneficial effects of the embodiments of this application are as follows: Firstly, based on the boredom increment of each viewing event for each content cluster and a predefined boredom value generation model, a final boredom value for each content cluster is generated. Based on the final boredom value of each content cluster and a predefined scoring model, a final recommendation score for each candidate short video is generated. The final recommendation scores of each candidate short video are sorted, and the top N candidate short videos are combined into a short video recommendation list for the user. In this way, the final boredom value of each content cluster can quantify the user's degree of boredom with each content cluster. Based on the final boredom value of each content cluster and the predefined scoring model, the final recommendation score of each candidate short video can be dynamically adjusted, reducing the phenomenon of dense push of similar content, enriching the diversity of the short video recommendation list, and meeting the user's current real needs, which is conducive to improving the recommendation accuracy of candidate short videos. Secondly, since the user's short video recommendation list is automatically generated and is not affected by human intervention, it helps to improve the reliability and stability of the user's short video recommendation list.

[0066] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0067] like Figure 3 As shown, Figure 3 The electronic device includes: at least one processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the at least one processor 30, wherein the processor 30 executes the computer program 32 to implement the steps in any of the above method embodiments.

[0068] The electronic device may include, but is not limited to, processor 30 and memory 31. Those skilled in the art will understand that... Figure 3 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than shown in the figure, or combinations of certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0069] The processor 30 is used to run a computer program 32 stored in the memory 31, and performs the following steps when executing the computer program 32: A training sample is formed by combining the multimodal data of the preset short video and the semantic features of the preset short video. A multimodal large model is trained based on the training sample. The first model is used to generate the repulsion loss of the multimodal large model, and the second model is used to generate the contrastive loss of the multimodal large model. The multimodal large model generates a comprehensive loss value by using the repulsion loss, contrastive loss, and predefined comprehensive loss model of the multimodal large model. When the comprehensive loss value is less than the preset loss value, the training of the multimodal large model is stopped, and the trained multimodal large model is saved. Based on the user's historical viewing sequence and the trained multimodal large model, the semantic features of each historically viewed short video are determined; The semantic features of each historical short video are clustered to generate multiple content clusters. The boredom increment generation model is used to generate the boredom increment of each content cluster for each viewing event. Based on the boredom increment of each viewing event for each content cluster and a predefined boredom value generation model, the final boredom value for each content cluster is generated. Based on the final boredom value for each content cluster and a predefined scoring model, the final recommendation score for each candidate short video is generated. The final recommendation scores for each candidate short video are sorted, and the top N candidate short videos are combined into the user's short video recommendation list. The first model is used to calculate the repulsion loss of a large multimodal model; the second model is used to calculate the contrastive loss of a large multimodal model. The rejection loss is the loss that constrains the ability of a multimodal large model to distinguish features between different samples within the same topic content. The smaller the rejection loss, the stronger the ability of the multimodal large model to distinguish features between different samples within the same topic content; the larger the rejection loss, the weaker the ability of the multimodal large model to distinguish features between different samples within the same topic content. Contrast loss is the loss that constrains the ability of a multimodal large model to distinguish between positive and negative sample pairs; the smaller the contrast loss, the better the contrast learning effect of the multimodal large model; the larger the contrast loss, the worse the contrast learning effect of the multimodal large model. The comprehensive loss model refers to a loss function that sums the contrast loss and the rejection loss after being weighted by a preset adjustment coefficient. The boredom value generation model is a model that generates the final boredom value of each content cluster based on the original boredom value of each content cluster. The original boredom value is the unadjusted level of boredom; the final boredom value is the adjusted level of boredom. The boredom increment for each viewing event of a content cluster refers to the increase in boredom for the current viewing event relative to the previous viewing event for each content cluster.

[0070] The processor 30 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors, field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0071] In some embodiments, the memory 31 may be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device. In other embodiments, the memory 31 may also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the electronic device.

[0072] Furthermore, the memory 31 may include both internal storage units and external storage devices of the electronic device. The memory 31 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0073] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0074] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0075] The computer-readable storage medium may also be an external storage device of the short video personalized recommendation device or electronic device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, or non-transitory computer-readable storage medium equipped on the short video personalized recommendation device or electronic device.

[0076] Since the computer program stored in the computer-readable storage medium can execute any of the short video personalized recommendation methods provided in the embodiments of this application, the computer-readable storage medium can achieve the beneficial effects that any of the short video personalized recommendation methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0077] Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium includes: an entity or device for carrying computer program code to an electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium.

[0078] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0079] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A personalized recommendation method for short videos, characterized in that, The short video personalized recommendation method, applied to electronic devices, includes: A training sample is formed by combining the multimodal data of the preset short video and the semantic features of the preset short video. A multimodal large model is trained based on the training sample. The first model is used to generate the repulsion loss of the multimodal large model, and the second model is used to generate the contrastive loss of the multimodal large model. The multimodal large model generates a comprehensive loss value by using the repulsion loss, contrastive loss, and predefined comprehensive loss model of the multimodal large model. When the comprehensive loss value is less than the preset loss value, the training of the multimodal large model is stopped, and the trained multimodal large model is saved. Based on the user's historical viewing sequence and the trained multimodal large model, the semantic features of each historically viewed short video are determined; The semantic features of each historical short video are clustered to generate multiple content clusters. The boredom increment generation model is used to generate the boredom increment of each content cluster for each viewing event. Based on the boredom increment of each viewing event for each content cluster and a predefined boredom value generation model, the final boredom value for each content cluster is generated. Based on the final boredom value for each content cluster and a predefined scoring model, the final recommendation score for each candidate short video is generated. The final recommendation scores for each candidate short video are sorted, and the top N candidate short videos are combined into the user's short video recommendation list. The first model is used to calculate the repulsion loss of a large multimodal model; the second model is used to calculate the contrastive loss of a large multimodal model. The rejection loss is the loss that constrains the ability of a multimodal large model to distinguish features between different samples within the same topic content. The smaller the rejection loss, the stronger the ability of the multimodal large model to distinguish features between different samples within the same topic content; the larger the rejection loss, the weaker the ability of the multimodal large model to distinguish features between different samples within the same topic content. Contrast loss is the loss that constrains the ability of a multimodal large model to distinguish between positive and negative sample pairs; the smaller the contrast loss, the better the contrastive learning effect of the multimodal large model; the larger the contrast loss, the worse the contrastive learning effect of the multimodal large model. The comprehensive loss model refers to a loss function that sums the contrast loss and the rejection loss after being weighted by a preset adjustment coefficient. The boredom value generation model is a model that generates the final boredom value of each content cluster based on the original boredom value of each content cluster. The original boredom value is the unadjusted level of boredom; the final boredom value is the adjusted level of boredom. The boredom increment for each viewing event of a content cluster refers to the increase in boredom for the current viewing event relative to the previous viewing event for each content cluster.

2. The short video personalized recommendation method according to claim 1, characterized in that, A training sample is composed of multimodal data and semantic features of a preset short video. A multimodal large-scale model is trained based on this training sample. The first model generates the repulsion loss of the multimodal large-scale model, and the second model generates the contrastive loss, including: Acquire multimodal data of a preset short video, which includes the keyframe sequence of the preset short video, the Mel spectrogram of the preset short video, and the text string of the preset short video; A training sample is formed by combining the multimodal data and semantic features of a preset short video. Different training samples are combined into a training set. A multimodal large model is trained based on the training set. The repulsion loss of the multimodal large model is generated through the first model, and the contrastive loss of the multimodal large model is generated through the second model.

3. The short video personalized recommendation method according to claim 1, characterized in that, Based on the user's historical viewing sequence and the trained multimodal large model, the semantic features of each historically viewed short video are determined, including: Multiple historical short videos are obtained from the user's historical viewing sequence. Multimodal data of the historical short videos is obtained, including keyframe sequences, Mel spectrograms, and text strings of the historical short videos. Feature extraction is performed on the keyframe sequence of the historically viewed short videos to obtain the features of the keyframe sequence of the short videos. Feature extraction is performed on the Mel spectrogram of the historically viewed short videos to obtain the features of the Mel spectrogram of the historically viewed short videos. Feature extraction is performed on the text string of the historically viewed short videos to obtain the features of the text string of the historically viewed short videos. The features of keyframe sequences, Mel spectrograms, and text strings from historically viewed short videos are fused to obtain the current features. These current features are then processed by the feature extractor in the trained multimodal large model to generate semantic features of the historically viewed short videos.

4. The short video personalized recommendation method according to claim 1, characterized in that, Based on the boredom increment of each viewing event for each content cluster and a predefined boredom value generation model, a final boredom value is generated for each content cluster. Based on the final boredom value of each content cluster and a predefined scoring model, a final recommendation score is generated for each candidate short video. The final recommendation scores of each candidate short video are sorted, and the top N candidate short videos are combined into a short video recommendation list for the user, including: The event sequence for each content cluster is generated by concatenating the boredom increment, completion rate, quick swipe indicator, negative feedback indicator, and repeated browsing indicator for each viewing event of each content cluster. The event sequence of each content cluster is input into the Transformer encoder in the trained multimodal large model. The Transformer encoder generates the encoding sequence of each content cluster. The encoding sequence of each content cluster is input into the fully connected layer in the trained multimodal large model. The fully connected layer generates the original boredom value of each content cluster. Based on the original boredom value and the predefined boredom value of each content cluster, the model is generated to generate the final boredom value of each content cluster. The final boredom value of each content cluster is mapped to the cluster identifier of each content cluster. The system retrieves a candidate list of users from the short video platform, obtains the cluster identifier corresponding to each candidate short video from the candidate list, retrieves the final boredom value of the content cluster to which each candidate short video belongs from the mapping table based on the cluster identifier of each candidate short video, generates the final recommendation score for each candidate short video by combining the final boredom value of the content cluster to which each candidate short video belongs with a predefined scoring model, sorts the final recommendation scores of each candidate short video, and forms the user's short video recommendation list by selecting the top N candidate short videos, where N is a positive integer.

5. The short video personalized recommendation method according to claim 2, characterized in that, The first model is defined as follows: ； This represents the repulsive loss of a large multimodal model. This represents the set of training samples used in a single training iteration. i represents any training sample in the training sample set; This represents the set of nearest neighbor samples that belong to the same topic as training sample i; The larger the nearest neighbor set, the more neighbors there are with the same topic as the training sample i; the smaller the nearest neighbor set, the fewer neighbors there are with the same topic as the training sample i. j represents any nearest neighbor sample in the nearest neighbor sample set; This represents the feature vector of training sample i after projection and normalization; This represents the feature vector of the nearest neighbor sample j after projection and normalization; This represents the dot product of the feature vectors of training sample i and its nearest neighbor sample j; Indicates the rejection threshold; The larger the value, the more lenient the conditions for triggering the rejection penalty; The smaller the value, the stricter the conditions for triggering the rejection penalty; The hinge truncation term is used to penalize the training sample i when the dot product of the feature vectors of the training sample i and the nearest neighbor sample j exceeds the repulsion threshold; the hinge truncation term does not penalize the training sample i when the dot product of the feature vectors of the training sample i and the nearest neighbor sample j does not exceed the repulsion threshold.

6. The short video personalized recommendation method according to claim 2, characterized in that, The second model is defined as follows: ； This represents the contrastive loss of a large multimodal model. This represents the set of training samples used in a single training iteration. i represents any training sample in the training sample set; This represents the set of positive samples corresponding to training sample i; the larger the set of positive samples, the more samples that form a positive sample pair with training sample i; the smaller the set of positive samples, the fewer samples that form a positive sample pair with training sample i. p represents any positive sample in the set of positive samples; m represents the remaining samples in the training sample set excluding training sample i; Indicates the temperature coefficient; This represents the feature vector of training sample i after projection and normalization; This represents the feature vector of a positive sample p after projection and normalization; exp is an exponential function with base e; The comprehensive loss model is defined as follows: ； This represents the overall loss of a large multimodal model. This represents the contrastive loss of a large multimodal model. This represents the repulsive loss of a large multimodal model. This represents the adjustment coefficient for rejection loss, used to adjust the degree of influence of rejection loss on overall loss.

7. The personalized recommendation method for short videos according to claim 1, characterized in that, Tired of incremental generation models, we define the following: ； This represents the boredom increment of the k-th viewing event for the c-th content cluster; The higher the value, the greater the user's level of boredom with watching specific short videos; The lower the value, the lower the user's level of boredom with the specified short video; watching a short video refers to the historical short video watched corresponding to the k-th viewing event of the c-th content cluster. This represents the completion rate of the k-th view event for the c-th content cluster; The quick swipe indicator represents the k-th view event of the c-th content cluster; ; When the value is 1, it means that the dwell time of the k-th viewing event of the c-th content cluster is less than 2 seconds; When the value is 0, it means that the dwell time of the k-th viewing event of the c-th content cluster is not less than 2 seconds; The negative feedback indicator represents the k-th view event of the c-th content cluster; ; When the value is 1, it means that in the k-th view event of the c-th content cluster, the user clicked on a control that was not of interest. A value of 0 indicates that in the k-th view event of the c-th content cluster, the user did not click on any controls they were not interested in. This is a repeat browsing indicator for the k-th view event of the c-th content cluster; ; When the value is 1, it indicates that the short video being watched is a repeatedly viewed content; repeatedly viewed content refers to short videos that the same user has watched within 24 hours prior to this viewing time. A value of 0 indicates that the short video being watched is not considered content that has been viewed repeatedly. , , These are the preset first weight coefficient, second weight coefficient, third weight coefficient, and fourth weight coefficient; The boredom value generation model is defined as follows: ； This represents the final boredom value of the Cth content cluster; The higher the value, the greater the user's ultimate level of boredom with the Cth content cluster; The lower the value, the lower the user's final level of boredom with the Cth content cluster; This represents the original boredom value of the Cth content cluster; The higher the value, the greater the user's initial aversion to the Cth content cluster; The lower the value, the lower the user's initial level of boredom with the Cth content cluster; Describes the minimum value function; This indicates the preset maximum aversion value; The scoring model is defined as follows: ； This represents the original recommendation score of the Vth candidate short video; This represents the final recommendation score of the Vth candidate short video; This represents the final boredom value of the content cluster to which the Vth candidate short video belongs; This represents the exponential decay factor.

8. A personalized recommendation device for short videos, characterized in that, Applied to electronic devices, including: The first generation module is used to combine the multimodal data of the preset short video and the semantic features of the preset short video into a training sample, train a multimodal large model based on the training sample, generate the repulsion loss of the multimodal large model through the first model, and generate the contrastive loss of the multimodal large model through the second model. The second generation module is used to generate the comprehensive loss value of the multimodal large model through the repulsion loss of the multimodal large model, the contrastive loss of the multimodal large model and the predefined comprehensive loss model. When the comprehensive loss value is less than the preset loss value, the training of the multimodal large model is stopped and the trained multimodal large model is saved. The third generation module is used to determine the semantic features of each historically viewed short video based on the user's historical viewing sequence and the trained multimodal large model; The clustering module is used to cluster the semantic features of each historically viewed short video to generate multiple content clusters. The boredom increment generation model generates the boredom increment of each content cluster for each viewing event. The recommendation module is used to generate the final boredom value for each content cluster based on the boredom increment of each viewing event and a predefined boredom value generation model. Based on the final boredom value of each content cluster and a predefined scoring model, it generates the final recommendation score for each candidate short video. The final recommendation scores of each candidate short video are sorted, and the top N candidate short videos are combined into the user's short video recommendation list. The first model is used to calculate the repulsion loss of a large multimodal model; the second model is used to calculate the contrastive loss of a large multimodal model. The rejection loss is the loss that constrains the ability of a multimodal large model to distinguish features between different samples within the same topic content. The smaller the rejection loss, the stronger the ability of the multimodal large model to distinguish features between different samples within the same topic content; the larger the rejection loss, the weaker the ability of the multimodal large model to distinguish features between different samples within the same topic content. Contrast loss is the loss that constrains the ability of a multimodal large model to distinguish between positive and negative sample pairs; the smaller the contrast loss, the better the contrastive learning effect of the multimodal large model; the larger the contrast loss, the worse the contrastive learning effect of the multimodal large model. The comprehensive loss model refers to a loss function that sums the contrast loss and the rejection loss after being weighted by a preset adjustment coefficient. The boredom value generation model is a model that generates the final boredom value of each content cluster based on the original boredom value of each content cluster. The original boredom value is the unadjusted level of boredom; the final boredom value is the adjusted level of boredom. The boredom increment for each viewing event of a content cluster refers to the increase in boredom for the current viewing event relative to the previous viewing event for each content cluster.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the short video personalized recommendation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the short video personalized recommendation method as described in any one of claims 1 to 7.

Images