Video recommendation method and system of collaborative knowledge graph based on semantic enhancement

Abstract

The invention relates to the field of video recommendation, and discloses a video recommendation method and system for a collaborative knowledge graph based on semantic enhancement, and the method comprises the following steps: carrying out the conjoint analysis of a video cover and text description through a multi-modal large language model, generating an article portrait, generating a user portrait through the user interaction history and the article portrait, and carrying out the recommendation of the user. A collaborative knowledge graph with enhanced semantic attributes is constructed in combination with article portraits and user portraits, complex relations among users, videos and deep attributes are represented in a structured mode, and finally a hierarchical fusion architecture with a cross-attention mechanism and adaptive gating is introduced. Multi-source heterogeneous representations such as texts, images and knowledge maps are efficiently aligned and integrated, and the problem of modal imbalance is effectively relieved. As a plug-and-play enhancement framework, the method can be seamlessly integrated in an existing recommendation model, the accuracy and the calibration degree of click rate prediction are remarkably improved, and more accurate and more robust personalized video recommendation is realized.

Description

Technical Field

[0001] This invention relates to the field of video recommendation, and in particular to a video recommendation method and system based on semantically enhanced collaborative knowledge graphs. Background Technology

[0002] With the booming development of streaming media and short video platforms, multimodal video recommendation systems have become a core engine for enhancing user engagement and platform commercial value. These systems need to effectively process and integrate heterogeneous data sources such as video covers, titles, tags, and user interaction records to accurately predict user click behavior and achieve personalized content distribution.

[0003] However, existing recommendation methods face two major challenges. First, their semantic understanding is superficial: traditional models, whether based on feature interactions or knowledge graphs, mostly focus on modeling explicit features, failing to deeply understand the rich and subtle semantic information contained in video content, such as visual aesthetics, emotional tone, or cross-modal irony, and lacking a precise grasp of the deeper intentions behind the content. Second, multimodal fusion fails: how to efficiently fuse heterogeneous data such as text, images, and user behavior remains a huge challenge. Traditional fusion strategies often lead to "modality imbalance," where certain modalities (such as text) become overly dominant during learning, resulting in the underutilization of other important information, such as visual information, ultimately limiting the performance ceiling of the recommendation system.

[0004] In recent years, the introduction of Large Language Models (LLMs) has provided a new opportunity to enhance the semantic understanding capabilities of recommendation systems. However, directly applying LLMs to multimodal recommendation scenarios presents significant bottlenecks. Most methods treat the text enhancements generated by LLMs merely as isolated additional features, failing to effectively perform cross-modal deep alignment and inference with visual information such as video covers. Furthermore, this simplistic enhancement approach ignores the complex structured relationships between users, items, and their rich semantic attributes, resulting in the deep semantic knowledge generated by large models not being fully and systematically integrated and utilized, thus limiting further improvements in recommendation performance. Therefore, a new technical framework is urgently needed to organically combine the deep semantic understanding capabilities of large models with structured knowledge representations, and to design advanced fusion mechanisms to solve the core challenges in current multimodal recommendation. Summary of the Invention

[0005] This invention overcomes the shortcomings of existing technologies and provides a video recommendation method and system based on semantically enhanced collaborative knowledge graphs.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: The first aspect of this invention provides a video recommendation method based on semantically enhanced collaborative knowledge graphs, comprising the following steps: By using a multimodal large model, deep semantic enhancement and reasoning processing are performed on historical video content to construct structured user profiles; By combining structured user profiles, a unified collaborative knowledge graph can be constructed. The collaborative knowledge graph is embedded using a graph embedding algorithm, and user structured representation vectors and video structured representation vectors are output after the embedding learning. Within the updated collaborative knowledge graph, a video recommendation model is constructed through hierarchical cross-modal fusion and adaptive fusion.

[0007] Furthermore, in a preferred embodiment of the present invention, the step of constructing a structured user profile by performing deep semantic enhancement and inference processing on historical video content through a multimodal large model specifically includes: A video recommendation device is acquired, and all historical videos that need to be recommended are acquired from the video recommendation device and marked as videos to be recommended. The original multimodal information is extracted from all videos to be recommended, wherein the original multimodal information is the cover image of the video to be recommended and the associated text description information. Image standardization preprocessing and text description information cleaning and standardization processing are performed on the original multimodal information to obtain preprocessed original multimodal information; Obtain a multimodal large language model and a multimodal large language model structure prompt template, and design and extract core themes, analyze sentiment tendencies, define visual styles, mark audience groups, and generate corresponding specific content themes in the multimodal large language model structure prompt template to obtain a target structure prompt template; In the multimodal large language model, target structure prompt templates and preprocessed raw multimodal information are imported together, and a visual-language attention mechanism is introduced to extract visual feature information of images in the preprocessed raw multimodal information in real time. At the same time, cross-modal association analysis is performed by combining the associated text description information. During cross-modal association analysis, the multimodal large language model generates an item profile based on the target structure prompt template. The item profile is generated by defining the sentiment tendency, defining the visual style, marking the target audience, and generating the theme of the corresponding specific content based on the visual feature information of the image and the associated text description information. Different semantic spaces are predefined, including topic classification semantic space, sentiment semantic space and visual style semantic space. Item profiles are mapped into different semantic spaces, and enhanced semantic vectors of videos to be recommended are output. Historical user interaction records of videos to be recommended are collected to construct user interaction sequences and integrate corresponding enhanced semantic vectors to obtain structured user profiles. The structured user profiles record the preference tags of different historical users of the videos to be recommended. During the integration of user interaction sequences and corresponding enhanced semantic vectors, the structured user profiles are jointly constructed through multi-model cross-cooperative verification. Each user corresponds to one structured user profile.

[0008] Furthermore, in a preferred embodiment of the present invention, the step of combining structured user profiles to construct a unified collaborative knowledge graph specifically includes: Within the video recommendation device, based on structured user profiles and item profiles, entity nodes are constructed for different users and all videos to be recommended, resulting in different user entity nodes, video entity nodes, and semantic attribute entity nodes. The semantic attribute entity nodes are constructed using the enhanced semantic vectors of the videos to be recommended. Construct edges between different entity nodes, including user-video interaction edges, video-semantic attribute interaction edges, and user-semantic attribute interaction edges; The edges between different entity nodes are defined with attributes. Among them, the user-video interaction edge defines the behavior records of different users on different videos to be recommended and the corresponding number of times. The video-semantic attribute interaction edge defines the semantic attributes of different videos to be recommended. The user-semantic attribute interaction edge defines the preference strength of different users for different semantic attributes. By combining the edges between different entity nodes after the attribute definition, a collaborative knowledge graph is constructed, and the edge weights are dynamically adjusted within the collaborative knowledge graph. Specifically, the edge weights are dynamically adjusted in real time according to the interaction frequency of different entity nodes, and node normalization and redundant edge merging are performed simultaneously. Among them, node normalization processing is to normalize different entity nodes with a number of connections greater than the standard value, and redundant edge merging processing is to merge the user-video interaction edges corresponding to the same user and video to obtain a single user-video interaction edge.

[0009] Furthermore, in a preferred embodiment of the present invention, the step of performing embedding learning on the collaborative knowledge graph using a graph embedding algorithm, and outputting user structured representation vectors and video structured representation vectors after embedding learning, specifically involves: A graph embedding model is introduced into the video recommendation device, wherein the graph embedding model is a model built with RotatE as the basic embedding core algorithm; In the collaborative knowledge graph, positive samples and negative samples are prepared. The positive samples are all known relation triples in the collaborative knowledge graph, which are composed of head entity node-relation edge-tail entity node. The negative samples are those in which the head entity node or tail entity node is replaced in the relation triple, and the replaced relation triple does not exist in the collaborative knowledge graph. Using a graph embedding model, the distance between the head entity node and the tail entity node in each relation triple is calculated in real time, and a standard distance threshold is preset. During real-time computation using the graph embedding model, the relation edges of each relation triplet undergo a rotation operation within the graph embedding model. After the rotation operation, relation triplets whose distance between the head entity node and the tail entity node meets the standard distance threshold are retained and labeled as qualified relation triplets. The system pre-sets standard training rounds, performs real-time computation of the convergence graph embedding model after the standard training rounds, and inputs all qualified relation triples into the collaborative knowledge graph to obtain an updated collaborative knowledge graph. Among them, when qualified relation triples are input into the collaborative knowledge graph, vector normalization and vector alignment are performed; In the updated collaborative knowledge graph, the structured representation vectors corresponding to user entity nodes and video entity nodes are extracted and labeled as user structured representation vectors and video structured representation vectors.

[0010] Furthermore, in a preferred embodiment of the present invention, the construction of a video recommendation model within the updated collaborative knowledge graph through hierarchical cross-modal fusion and adaptive ensemble specifically involves: Within the updated collaborative knowledge graph, obtain the user's structured representation vector and the video's structured representation vector, and simultaneously obtain the enhanced semantic vector of the video to be recommended; In video recommendation devices, user structured representation vectors, video structured representation vectors, and enhanced semantic vectors of the videos to be recommended are mapped onto a unified fusion space dimension to obtain a linear mapping layer, and feature vector standardization is performed on the linear mapping layer. A bidirectional cross-attention mechanism is introduced within the linear mapping layer. Through this mechanism, a query-key-value vector is constructed in the linear mapping layer. The query-key-value vector includes a vector with text as the query, image as the key and value, used to calculate text-to-image attention, and a vector with image as the query, text as the key and value, used to calculate image-to-text attention. Local feature alignment and global semantic alignment are performed on different query-key-value vectors to achieve multi-level vector interaction fusion and obtain a unified content fusion vector; An adaptive gating unit is introduced to perform feature concatenation interaction between the unified content fusion vector and other vectors in the updated collaborative knowledge graph, and output unified content representation data that integrates the visual semantics and textual semantics of the video to be recommended. A video recommendation model is obtained by modeling a recommendation model based on unified content representation data that integrates the visual and textual semantics of the videos to be recommended.

[0011] Furthermore, in a preferred embodiment of the present invention, the step of modeling a recommendation model based on unified content representation data that integrates the visual semantics and textual semantics of the video to be recommended, to obtain a target video recommendation model, specifically involves: Based on unified content representation data that integrates visual and textual semantics of the videos to be recommended, video entity representation information and user entity representation information are obtained. The video entity representation information and user entity representation information are interactively modeled. The interactive modeling involves integrating the video entity representation information and user entity representation information into a preset video recommendation blank model plugin in the video recommendation device, and capturing the interactive relationship between the video entity representation information and user entity representation information in the video recommendation blank model plugin to obtain interactive feature representation information. Continue to train by integrating and splicing interactive feature representation information with video entity representation information, so that the video recommendation blank model plugin outputs a single highly condensed multimodal fusion vector, and introduces it into the downstream recommendation model in the big data network; Install the blank video recommendation model plugin into the downstream recommendation model to obtain the target video recommendation model.

[0012] This invention addresses the technical deficiencies in the background technology and offers the following beneficial effects: It utilizes a multimodal large language model to jointly analyze video cover images and text descriptions, generating item profiles. It then uses user interaction history and item profiles to generate user profiles. Finally, it combines item and user profiles to construct a collaborative knowledge graph with enhanced semantic attributes, representing the complex relationships between users, videos, and their deep attributes in a structured manner. Finally, it introduces a hierarchical fusion architecture with cross-attention mechanisms and adaptive gating to efficiently align and integrate multi-source heterogeneous representations such as text, images, and knowledge graphs, effectively alleviating the modality imbalance problem. As a plug-and-play enhancement framework, this invention can be seamlessly integrated into existing recommendation models, significantly improving the accuracy and calibration of click-through rate prediction, and achieving more precise and robust personalized video recommendations. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained from these drawings without creative effort.

[0014] Figure 1 A flowchart of a video recommendation method based on semantically enhanced collaborative knowledge graphs is shown; Figure 2 The flowchart illustrating the method for constructing a video recommendation model is shown. Figure 3 A program view of a video recommendation system based on semantically enhanced collaborative knowledge graphs is shown. Detailed Implementation

[0015] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0016] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0017] Figure 1 A flowchart of a video recommendation method based on semantically enhanced collaborative knowledge graphs is shown, including the following steps: S102: Through multimodal large models, perform deep semantic enhancement and reasoning processing on historical video content to build structured user profiles; S104: Combine structured user profiles to build a unified collaborative knowledge graph; S106: The collaborative knowledge graph is embedded using a graph embedding algorithm, and user structured representation vectors and video structured representation vectors are output after the embedding learning. S108: Within the updated collaborative knowledge graph, a video recommendation model is constructed through hierarchical cross-modal fusion and adaptive fusion.

[0018] Furthermore, in a preferred embodiment of the present invention, the step of constructing a structured user profile by performing deep semantic enhancement and inference processing on historical video content through a multimodal large model specifically includes: A video recommendation device is acquired, and all historical videos that need to be recommended are acquired from the video recommendation device and marked as videos to be recommended. The original multimodal information is extracted from all videos to be recommended, wherein the original multimodal information is the cover image of the video to be recommended and the associated text description information. Image standardization preprocessing and text description information cleaning and standardization processing are performed on the original multimodal information to obtain preprocessed original multimodal information; Obtain a multimodal large language model and a multimodal large language model structure prompt template, and design and extract core themes, analyze sentiment tendencies, define visual styles, mark audience groups, and generate corresponding specific content themes in the multimodal large language model structure prompt template to obtain a target structure prompt template; In the multimodal large language model, target structure prompt templates and preprocessed raw multimodal information are imported together, and a visual-language attention mechanism is introduced to extract visual feature information of images in the preprocessed raw multimodal information in real time. At the same time, cross-modal association analysis is performed by combining the associated text description information. During cross-modal association analysis, the multimodal large language model generates an item profile based on the target structure prompt template. The item profile is generated by defining the sentiment tendency, defining the visual style, marking the target audience, and generating the theme of the corresponding specific content based on the visual feature information of the image and the associated text description information. Different semantic spaces are predefined, including topic classification semantic space, sentiment semantic space and visual style semantic space. Item profiles are mapped into different semantic spaces, and enhanced semantic vectors of videos to be recommended are output. Historical user interaction records of videos to be recommended are collected to construct user interaction sequences and integrate corresponding enhanced semantic vectors to obtain structured user profiles. The structured user profiles record the preference tags of different historical users of the videos to be recommended. During the integration of user interaction sequences and corresponding enhanced semantic vectors, the structured user profiles are jointly constructed through multi-model cross-cooperative verification. Each user corresponds to one structured user profile.

[0019] It should be noted that this approach utilizes deep semantic enhancement and reasoning based on a multimodal large model to generate structured descriptions rich in deep semantics for each independent video to be recommended. First, cross-modal enhancement of the video content is performed. This involves aggregating the cover images and text information of any video to be recommended, such as titles, descriptions, and tags, and designing a dedicated structured prompt template for content enhancement. The structural information conforming to the template content is extracted from the preprocessed original multimodal information, thus generating the item profile. The data is represented in vector form, mapping the item profile into different semantic spaces, outputting the enhanced semantic vector of the video to be recommended. Finally, a structured user profile is obtained based on the vector and user interaction sequences. The user profile accurately describes the user's deep preferences in the form of structured fields (such as "interest tag set," "preference style," "user profile," and "recommendation strategy suggestions"), providing a core basis for personalized recommendations. This solution achieves a deep semantic understanding of video content and user preferences. Compared with traditional shallow feature extraction methods, this method can capture richer and more subtle semantic information, providing a high-quality semantic foundation for subsequent knowledge graph construction and multimodal fusion.

[0020] Furthermore, in a preferred embodiment of the present invention, the step of combining structured user profiles to construct a unified collaborative knowledge graph specifically includes: Within the video recommendation device, based on structured user profiles and item profiles, entity nodes are constructed for different users and all videos to be recommended, resulting in different user entity nodes, video entity nodes, and semantic attribute entity nodes. The semantic attribute entity nodes are constructed using the enhanced semantic vectors of the videos to be recommended. Construct edges between different entity nodes, including user-video interaction edges, video-semantic attribute interaction edges, and user-semantic attribute interaction edges; The edges between different entity nodes are defined with attributes. Among them, the user-video interaction edge defines the behavior records of different users on different videos to be recommended and the corresponding number of times. The video-semantic attribute interaction edge defines the semantic attributes of different videos to be recommended. The user-semantic attribute interaction edge defines the preference strength of different users for different semantic attributes. By combining the edges between different entity nodes after the attribute definition, a collaborative knowledge graph is constructed, and the edge weights are dynamically adjusted within the collaborative knowledge graph. Specifically, the edge weights are dynamically adjusted in real time according to the interaction frequency of different entity nodes, and node normalization and redundant edge merging are performed simultaneously. Among them, node normalization processing is to normalize different entity nodes with a number of connections greater than the standard value, and redundant edge merging processing is to merge the user-video interaction edges corresponding to the same user and video to obtain a single user-video interaction edge.

[0021] It should be noted that this method aims to systematically integrate the generated structured user profiles and other discrete, unstructured deep semantic information with user interaction behavior to construct a unified knowledge carrier, namely, a collaborative knowledge graph. The knowledge graph includes different nodes and corresponding edges between nodes. Nodes contain three core entities: user entity nodes, video entity nodes, and semantic attribute entity nodes. User entity nodes represent all independent users in the system, video entity nodes represent all videos to be recommended, and semantic attribute entity nodes represent generated structured attributes, including preference tags from different historical users of the videos to be recommended. Edges are constructed between different entity nodes, representing the relationships between different entities. Edges combined with entity nodes constitute the knowledge graph. The edge weights are dynamically adjusted in real time based on the interaction frequency of different entity nodes, assigning higher weights to recent user behavior and high-frequency user behavior. Degree normalization is performed on highly connected nodes (such as popular videos and general attributes) to avoid imbalances in node influence. Redundant edge merging is performed by merging user-video interaction edges corresponding to the same user and video, reducing the computational complexity of the knowledge graph.

[0022] Furthermore, in a preferred embodiment of the present invention, the step of performing embedding learning on the collaborative knowledge graph using a graph embedding algorithm, and outputting user structured representation vectors and video structured representation vectors after embedding learning, specifically involves: A graph embedding model is introduced into the video recommendation device, wherein the graph embedding model is a model built with RotatE as the basic embedding core algorithm; In the collaborative knowledge graph, positive samples and negative samples are prepared. The positive samples are all known relation triples in the collaborative knowledge graph, which are composed of head entity node-relation edge-tail entity node. The negative samples are those in which the head entity node or tail entity node is replaced in the relation triple, and the replaced relation triple does not exist in the collaborative knowledge graph. Using a graph embedding model, the distance between the head entity node and the tail entity node in each relation triple is calculated in real time, and a standard distance threshold is preset. During real-time computation using the graph embedding model, the relation edges of each relation triplet undergo a rotation operation within the graph embedding model. After the rotation operation, relation triplets whose distance between the head entity node and the tail entity node meets the standard distance threshold are retained and labeled as qualified relation triplets. The system pre-sets standard training rounds, performs real-time computation of the convergence graph embedding model after the standard training rounds, and inputs all qualified relation triples into the collaborative knowledge graph to obtain an updated collaborative knowledge graph. Among them, when qualified relation triples are input into the collaborative knowledge graph, vector normalization and vector alignment are performed; In the updated collaborative knowledge graph, the structured representation vectors corresponding to user entity nodes and video entity nodes are extracted and labeled as user structured representation vectors and video structured representation vectors.

[0023] It should be noted that a graph embedding model is employed, mapping each discrete node and relation in the graph to a low-dimensional, continuous vector space. RotatE is chosen as the basic embedding algorithm because it effectively models symmetric / antisymmetric, inversion, and combinatorial relations. This model represents each entity as a point in a complex vector space and treats each relation as a rotation operation in this space to ensure that after training in the vector space, the head entity vector, after the rotation corresponding to the relation, is close to the tail entity vector. In the collaborative knowledge graph, positive and negative samples are prepared, and training with both positive and negative samples together can improve training accuracy. A relation triple consists of a head entity node, a relation edge, and a tail entity node. By calculating the distance between the head and tail entity nodes in each relation triple, the probability of the relation triple being valid is determined, achieving gradient descent-optimized embedding vectors. After the model training converges, each node in the graph will obtain a complex vector containing its neighborhood structure and semantic information, namely, a user structured representation vector and a video structured representation vector. This invention integrates users, videos, and their semantic attributes into a unified knowledge graph and transforms it into a structured vector representation. Compared with traditional methods based on shallow features, this approach can capture complex multi-hop relationships between users, videos, and attributes, providing rich structured knowledge input for downstream multimodal fusion and enhancing the semantic understanding and reasoning capabilities of recommendation systems.

[0024] Figure 2 The flowchart illustrating the method for building a video recommendation model includes the following steps: S202: Within the updated collaborative knowledge graph, a video recommendation model is constructed through hierarchical cross-modal fusion and adaptive ensemble. S204: The recommendation model is modeled based on the unified content representation data that integrates the visual and textual semantics of the video to be recommended, resulting in a video recommendation model.

[0025] Furthermore, in a preferred embodiment of the present invention, the construction of a video recommendation model within the updated collaborative knowledge graph through hierarchical cross-modal fusion and adaptive ensemble specifically involves: Within the updated collaborative knowledge graph, obtain the user's structured representation vector and the video's structured representation vector, and simultaneously obtain the enhanced semantic vector of the video to be recommended; In video recommendation devices, user structured representation vectors, video structured representation vectors, and enhanced semantic vectors of the videos to be recommended are mapped onto a unified fusion space dimension to obtain a linear mapping layer, and feature vector standardization is performed on the linear mapping layer. A bidirectional cross-attention mechanism is introduced within the linear mapping layer. Through this mechanism, a query-key-value vector is constructed in the linear mapping layer. The query-key-value vector includes a vector with text as the query, image as the key and value, used to calculate text-to-image attention, and a vector with image as the query, text as the key and value, used to calculate image-to-text attention. Local feature alignment and global semantic alignment are performed on different query-key-value vectors to achieve multi-level vector interaction fusion and obtain a unified content fusion vector; An adaptive gating unit is introduced to perform feature concatenation interaction between the unified content fusion vector and other vectors in the updated collaborative knowledge graph, and output unified content representation data that integrates the visual semantics and textual semantics of the video to be recommended. A video recommendation model is obtained by modeling a recommendation model based on unified content representation data that integrates the visual and textual semantics of the videos to be recommended.

[0026] It's important to note that this method aims to systematically align, fuse, and integrate various generated heterogeneous representation vectors to create a unified multimodal feature vector that comprehensively captures user-video relationships. First, it receives known user structured representation vectors, video structured representation vectors, and enhanced semantic vectors of the video to be recommended. These are then uniformly mapped into a shared and fixed-dimensional fusion space, resulting in a linear mapping layer, which lays the foundation for subsequent effective interaction and fusion. Standardization is then performed to eliminate dimensional differences between features. A bidirectional cross-attention mechanism is introduced within the linear mapping layer to fuse the two core content modalities of the video (text and image) to generate a unified content representation. Text features serve as queries, focusing on key regions in image features; conversely, image features also serve as queries, focusing on key semantics in text features. This allows the model to capture fine-grained cross-modal associations such as "a specific object in an image corresponds to a word in a title." Therefore, query-key-value vectors are constructed, and different query-key-value vectors undergo local feature alignment and global semantic alignment to achieve multi-level vector interaction and fusion. This involves fine-grained matching at the word / image patch level and semantic association at the sentence / complete image level, resulting in a unified content fusion vector. This unified content fusion vector is then concatenated with the original feature information through an adaptive gating unit. In other words, the unified content fusion vector interacts with other vectors within the updated collaborative knowledge graph through feature concatenation, effectively alleviating the modality imbalance problem caused by excessive information in a single modality and generating a unified content representation that fuses visual and textual semantics.

[0027] Furthermore, in a preferred embodiment of the present invention, the step of modeling a recommendation model based on unified content representation data that integrates the visual semantics and textual semantics of the video to be recommended, to obtain a video recommendation model, specifically involves: Based on unified content representation data that integrates visual and textual semantics of the videos to be recommended, video entity representation information and user entity representation information are obtained. The video entity representation information and user entity representation information are interactively modeled. The interactive modeling involves integrating the video entity representation information and user entity representation information into a preset video recommendation blank model plugin in the video recommendation device, and capturing the interactive relationship between the video entity representation information and user entity representation information in the video recommendation blank model plugin to obtain interactive feature representation information. Continue to train by integrating and splicing interactive feature representation information with video entity representation information, so that the video recommendation blank model plugin outputs a single highly condensed multimodal fusion vector, and introduces it into the downstream recommendation model in the big data network; Install the blank video recommendation model plugin into the downstream recommendation model to obtain the target video recommendation model.

[0028] It should be noted that, based on unified content representation data that integrates the visual and textual semantics of the videos to be recommended, video entity representation information and user entity representation information are generated. These two information are then fused through a plugin to simulate and capture the complex interaction between them, resulting in interaction feature representation information. Finally, the video entity representation and the interaction feature representation (representing the matching relationship between users and items) are concatenated, and a final linear transformation layer is applied to output the final product of the entire framework—a single, highly condensed multimodal fusion vector. The downstream recommendation model is a known model used for video recommendation. A blank video recommendation model plugin is installed within the downstream recommendation model as a powerful enhancement feature, directly input into any mainstream click-through rate (CTR) prediction baseline model (such as DeepFM, Wide&Deep, etc.). The baseline model, based on its original features, combines this newly added feature, rich in deep semantics and structured knowledge, to jointly complete the final CTR prediction task, thus achieving seamless and efficient enhancement of the original model. The upstream model corresponding to the downstream model is the blank video recommendation model plugin.

[0029] like Figure 3 As shown, the second aspect of the present invention also provides a video recommendation system based on semantically enhanced collaborative knowledge graphs. This video recommendation system integrates different modules, and when these modules are executed by the system processor, the following steps are implemented: Deep Semantic Enhancement and Knowledge Generation Module: Responsible for using multimodal large language models to deeply analyze video content and user behavior, and structuring the analyzed semantic information into a collaborative knowledge graph; Multimodal data representation module: responsible for uniformly encoding various types of raw data, enhanced content, and structured knowledge into computable vector representations; The hierarchical multimodal fusion module is responsible for aligning, hierarchically interacting with, and adaptively integrating multi-source heterogeneous representations, generating the final fused feature vector, and connecting it with downstream models. Basic Recommendation Model Module: Existing mainstream recommendation models for click-through rate prediction.

[0030] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A video recommendation method based on semantically enhanced collaborative knowledge graphs, characterized in that, Includes the following steps: By using a multimodal large model, deep semantic enhancement and reasoning processing are performed on historical video content to construct structured user profiles; By combining structured user profiles, a unified collaborative knowledge graph can be constructed. The collaborative knowledge graph is embedded using a graph embedding algorithm, and user structured representation vectors and video structured representation vectors are output after the embedding learning. Within the updated collaborative knowledge graph, a video recommendation model is constructed through hierarchical cross-modal fusion and adaptive fusion.

2. The video recommendation method based on semantically enhanced collaborative knowledge graph as described in claim 1, characterized in that, The process involves using a multimodal large model to perform deep semantic enhancement and reasoning processing on historical video content to construct a structured user profile, specifically: A video recommendation device is acquired, and all historical videos that need to be recommended are acquired from the video recommendation device and marked as videos to be recommended. The original multimodal information is extracted from all videos to be recommended, wherein the original multimodal information is the cover image of the video to be recommended and the associated text description information. Image standardization preprocessing and text description information cleaning and standardization processing are performed on the original multimodal information to obtain preprocessed original multimodal information; Obtain a multimodal large language model and a multimodal large language model structure prompt template, and design and extract core themes, analyze sentiment tendencies, define visual styles, mark audience groups, and generate corresponding specific content themes in the multimodal large language model structure prompt template to obtain a target structure prompt template; In the multimodal large language model, target structure prompt templates and preprocessed raw multimodal information are imported together, and a visual-language attention mechanism is introduced to extract visual feature information of images in the preprocessed raw multimodal information in real time. At the same time, cross-modal association analysis is performed by combining the associated text description information. During cross-modal association analysis, the multimodal large language model generates an item profile based on the target structure prompt template. The item profile is generated by defining the sentiment tendency, defining the visual style, marking the target audience, and generating the theme of the corresponding specific content based on the visual feature information of the image and the associated text description information. Different semantic spaces are predefined, including topic classification semantic space, sentiment semantic space and visual style semantic space. Item profiles are mapped into different semantic spaces, and enhanced semantic vectors of videos to be recommended are output. Historical user interaction records of videos to be recommended are collected to construct user interaction sequences and integrate corresponding enhanced semantic vectors to obtain structured user profiles. The structured user profiles record the preference tags of different historical users of the videos to be recommended. During the integration of user interaction sequences and corresponding enhanced semantic vectors, the structured user profiles are jointly constructed through multi-model cross-cooperative verification. Each user corresponds to one structured user profile.

3. The video recommendation method based on semantically enhanced collaborative knowledge graph as described in claim 1, characterized in that, The construction of a unified collaborative knowledge graph by combining structured user profiles specifically involves: Within the video recommendation device, based on structured user profiles and item profiles, entity nodes are constructed for different users and all videos to be recommended, resulting in different user entity nodes, video entity nodes, and semantic attribute entity nodes. The semantic attribute entity nodes are constructed using the enhanced semantic vectors of the videos to be recommended. Construct edges between different entity nodes, including user-video interaction edges, video-semantic attribute interaction edges, and user-semantic attribute interaction edges; The edges between different entity nodes are defined with attributes. Among them, the user-video interaction edge defines the behavior records of different users on different videos to be recommended and the corresponding number of times. The video-semantic attribute interaction edge defines the semantic attributes of different videos to be recommended. The user-semantic attribute interaction edge defines the preference strength of different users for different semantic attributes. By combining the edges between different entity nodes after the attribute definition, a collaborative knowledge graph is constructed, and the edge weights are dynamically adjusted within the collaborative knowledge graph. Specifically, the edge weights are dynamically adjusted in real time according to the interaction frequency of different entity nodes, and node normalization and redundant edge merging are performed simultaneously. Among them, node normalization processing is to normalize different entity nodes with a number of connections greater than the standard value, and redundant edge merging processing is to merge the user-video interaction edges corresponding to the same user and video to obtain a single user-video interaction edge.

4. The video recommendation method based on semantically enhanced collaborative knowledge graph as described in claim 1, characterized in that, The method involves using a graph embedding algorithm to perform embedding learning on the collaborative knowledge graph, and outputting user structured representation vectors and video structured representation vectors after embedding learning. Specifically: A graph embedding model is introduced into the video recommendation device, wherein the graph embedding model is a model built with RotatE as the basic embedding core algorithm; In the collaborative knowledge graph, positive samples and negative samples are prepared. The positive samples are all known relation triples in the collaborative knowledge graph, which are composed of head entity node-relation edge-tail entity node. The negative samples are those in which the head entity node or tail entity node is replaced in the relation triple, and the replaced relation triple does not exist in the collaborative knowledge graph. Using a graph embedding model, the distance between the head entity node and the tail entity node in each relation triple is calculated in real time, and a standard distance threshold is preset. During real-time computation using the graph embedding model, the relation edges of each relation triplet undergo a rotation operation within the graph embedding model. After the rotation operation, relation triplets whose distance between the head entity node and the tail entity node meets the standard distance threshold are retained and labeled as qualified relation triplets. The system pre-sets standard training rounds, performs real-time computation of the convergence graph embedding model after the standard training rounds, and inputs all qualified relation triples into the collaborative knowledge graph to obtain an updated collaborative knowledge graph. Among them, when qualified relation triples are input into the collaborative knowledge graph, vector normalization and vector alignment are performed; In the updated collaborative knowledge graph, the structured representation vectors corresponding to user entity nodes and video entity nodes are extracted and labeled as user structured representation vectors and video structured representation vectors.

5. The video recommendation method based on semantically enhanced collaborative knowledge graph as described in claim 1, characterized in that, Within the updated collaborative knowledge graph, a video recommendation model is constructed through hierarchical cross-modal fusion and adaptive ensemble, specifically as follows: Within the updated collaborative knowledge graph, obtain the user's structured representation vector and the video's structured representation vector, and simultaneously obtain the enhanced semantic vector of the video to be recommended; In video recommendation devices, user structured representation vectors, video structured representation vectors, and enhanced semantic vectors of the videos to be recommended are mapped onto a unified fusion space dimension to obtain a linear mapping layer, and feature vector standardization is performed on the linear mapping layer. A bidirectional cross-attention mechanism is introduced within the linear mapping layer. Through this mechanism, a query-key-value vector is constructed in the linear mapping layer. The query-key-value vector includes a vector with text as the query, image as the key and value, used to calculate text-to-image attention, and a vector with image as the query, text as the key and value, used to calculate image-to-text attention. Local feature alignment and global semantic alignment are performed on different query-key-value vectors to achieve multi-level vector interaction fusion and obtain a unified content fusion vector; An adaptive gating unit is introduced to perform feature concatenation interaction between the unified content fusion vector and other vectors in the updated collaborative knowledge graph, and output unified content representation data that integrates the visual semantics and textual semantics of the video to be recommended. A video recommendation model is obtained by modeling a recommendation model based on unified content representation data that integrates the visual and textual semantics of the videos to be recommended.

6. The video recommendation method based on semantically enhanced collaborative knowledge graph as described in claim 5, characterized in that, The recommendation model is obtained by modeling a recommendation model based on unified content representation data that integrates the visual and textual semantics of the video to be recommended, specifically as follows: Based on unified content representation data that integrates visual and textual semantics of the videos to be recommended, video entity representation information and user entity representation information are obtained. The video entity representation information and user entity representation information are interactively modeled. The interactive modeling involves integrating the video entity representation information and user entity representation information into a preset video recommendation blank model plugin in the video recommendation device, and capturing the interactive relationship between the video entity representation information and user entity representation information in the video recommendation blank model plugin to obtain interactive feature representation information. Continue to train by integrating and splicing interactive feature representation information with video entity representation information, so that the video recommendation blank model plugin outputs a single highly condensed multimodal fusion vector, and introduces it into the downstream recommendation model in the big data network; Install the blank video recommendation model plugin into the downstream recommendation model to obtain the target video recommendation model.

7. A video recommendation system based on semantically enhanced collaborative knowledge graphs, characterized in that, The video recommendation system integrates different modules, and when the different modules are executed by the system processor, the video recommendation steps as described in any one of claims 1-6 are implemented.

Images