A multi-modal news recommendation method and system combined with clock interest

Abstract

This invention discloses a multimodal news recommendation method and system that incorporates clock-based interests, belonging to the field of data mining and recommendation methods. The method includes extracting user behavior sequences, candidate news sets, and related timestamp sets; obtaining multimodal encoded representations of historical interactive news through time-aware multimodal feature encoding; performing clock-based interest modeling based on these representations, and fusing long-term and short-term interests through Gaussian weighted aggregation and long-term interest enhancement operations to obtain user interest vectors; calculating the matching degree between candidate news and the current user interest vector, and applying time-sensitive suppression through time-series gating; using a linear network to predict click probabilities for dynamic recommendation decisions. This invention leverages time-context-aware feature fusion technology to capture hourly-level interest continuity, enabling more accurate matching of recommended content with user needs at different times.

Description

Technical Field

[0001] This invention relates to the field of data mining and recommendation methods, and more specifically, to a multimodal news recommendation method and system that incorporates clock-based interests. Background Technology

[0002] In today's digital age, personalized news recommendation systems have become one of the core technologies for alleviating information overload. Traditional recommendation systems construct static interest profiles by mining users' historical behavior, achieving significant results in e-commerce, video, and other fields. For news recommendation, multimodal fusion technology has become a research hotspot in recent years. Existing news recommendation frameworks based on multimodal knowledge graphs construct cross-modal knowledge graphs by crawling news text and image data, and utilize VilBERT pre-trained models to achieve semantic alignment between text and images, validating the effectiveness of cross-modal representations. However, news reading scenarios have unique time sensitivity; users tend to browse international politics during morning commutes, prefer sports news during lunch breaks, and may focus on entertainment late at night. This time-fluctuating interest pattern poses new requirements for dynamic adaptation in recommendation systems. While existing research has made some progress in cross-modal recommendation and temporal modeling, few studies have systematically explored the impact of the diurnal rhythm characteristics of user interests on news consumption.

[0003] Current mainstream methods suffer from three main shortcomings: First, static recommendation frameworks based on collaborative filtering cannot capture hourly interest shifts, leading to users receiving irrelevant content during specific time periods. Second, existing time-series recommendation models focus primarily on long-term interest evolution, neglecting intraday periodic patterns (such as differences between weekdays and weekends). Finally, multimodal recommendation systems often simply concatenate text and image features, failing to establish a dynamic relationship between temporal context and multimodal content. Recent research shows that user click behavior fluctuates significantly within a 24-hour period, but existing systems still employ fixed-time-window update strategies, resulting in a misalignment between recommendation results and real-time needs. To address these challenges, particularly by systematically exploring the impact of the diurnal rhythm characteristics of user interests on news, there is an urgent need to develop a multimodal news recommendation method that incorporates clock-based interests. Summary of the Invention

[0004] To address the aforementioned technical issues, this invention proposes a multimodal news recommendation method and system that incorporates clock-based interests. By leveraging time-context-aware feature fusion technology, it captures hourly-level interest continuity to improve recommendation accuracy.

[0005] The first aspect of this invention provides a multimodal news recommendation method that incorporates clock-based interests, comprising the following steps:

[0006] Construct a time-series interaction sequence from user historical behavior logs, and extract user behavior sequences, candidate news sets, and related timestamp sets based on the time-series interaction sequence;

[0007] Extract historical interactive news that meets preset time requirements from the user behavior sequence and related timestamp set, perform time-aware multimodal feature encoding on the historical interactive news, and obtain the multimodal encoded representation of the historical interactive news;

[0008] Clock interest modeling is performed based on the multimodal encoding representation of the historical interactive news. Through Gaussian weighted aggregation and long-term interest enhancement operations, vector representations of users' short-term and long-term interests are generated, and user interest vectors are obtained by fusing short-term and long-term interests.

[0009] The matching degree between candidate news and the current user interest vector is calculated, and time-sensitive suppression is applied through time-series gating. The output of time-series gating is used to predict the click probability through a linear network. Dynamic recommendation decisions are made based on the prediction results, and the user interest database is updated through online learning.

[0010] In this solution, a time-series interaction sequence is constructed from user historical behavior logs. Based on the time-series interaction sequence, user behavior sequences, candidate news sets, and related timestamp sets are extracted, specifically as follows:

[0011] Extract user ID, news ID, interaction behavior and timestamp as raw data from user history behavior logs, and perform data cleaning on the raw data to remove duplicate records and invalid data;

[0012] The cleaned raw data is grouped according to user ID to generate an interaction behavior dataset for each user. The interaction behavior dataset is then sorted in ascending order according to timestamp to form a time-series interaction sequence.

[0013] Extract user interaction behaviors and timestamps corresponding to each interaction behavior from the time-series interaction sequence, construct user behavior sequences and related timestamp sequences, and obtain all news in the current system to construct a candidate news set.

[0014] In this scheme, historical interactive news that meets preset time requirements is extracted from the user behavior sequence and related timestamp set. Time-aware multimodal feature encoding is then performed on the historical interactive news to obtain its multimodal encoded representation. Specifically:

[0015] Based on the recommendation requirements, a preset time window is used to traverse the user behavior sequence and related timestamp set, extract the interaction records that meet the preset time conditions, and filter the news that the user has interacted with within the preset time window.

[0016] The selected news IDs and associated multimodal data are combined to form a historical interactive news set, where each news item includes text content and image content. A pre-trained BERT model is used to semantically encode the news text content to obtain an initial text feature vector.

[0017] Extract short-segment information of news interaction time, encode time features using a sine function, map the time features to the semantic space through linear transformation, add them to BERT text features, and enhance the initial text feature vector to obtain a time-enhanced text representation;

[0018] Mask R-CNN is used to detect targets in news images and extract salient regions. The salient regions and news text are then input into the ViLBERT model, which generates visual-semantic aligned feature representations through a cross-modal attention mechanism.

[0019] The temporally enhanced text representation is concatenated with the visual-semantic aligned feature representation to generate the final multimodal encoded representation.

[0020] In this scheme, clock interest modeling is performed based on the multimodal encoding representation of the historical interactive news. Through Gaussian weighted aggregation and long-term interest enhancement operations, context vectors of the user's short-term and long-term interests are generated, specifically:

[0021] Build a user interest database based on 24-hour timeframe ,in Storage users in The multimodal encoding representation of historical interactive news for a given time period is updated in the interest library whenever a user interacts in a new time period.

[0022] Short-term interest modeling is performed using Gaussian weighted aggregation, with the Gaussian kernel function applied at the current time step. Calculate adjacent time periods centered on the target. The weights of the interest vectors for 24 time periods. By performing weighted summation, a smoothed short-term interest representation is obtained. , represented as:

[0023] ,

[0024] in, Indicates adjacent time periods The weight, Indicates smoothness intensity;

[0025] Retrieve relevant news from users' historical data across all time periods to enhance long-term interest representation. Filter historical news with semantic similarity to current candidate news from the historical interaction news set, and calculate the weight of each historical news item based on semantic relevance and time decay. ;

[0026] Multimodal coding representation of weighted aggregated historical news Generate long-term interest representations :

[0027] ,

[0028] in, This represents a collection of historical interactive news articles.

[0029] In this solution, user interest vectors are obtained by fusing long-term and short-term interests, specifically as follows:

[0030] Characterizing short-term interests through Gaussian aggregation and long-term interests The vectors are concatenated into a high-dimensional vector, and then dimensionality is reduced using a learnable matrix to generate the final user interest vector. , represented as:

[0031] ,

[0032] in, This represents a learnable fusion matrix. The dimension is The real number space.

[0033] In this scheme, the matching degree between candidate news items and the current user's interest vector is calculated, and time-sensitive suppression is applied through time-series gating. The output of the time-series gating is then used to predict click probabilities using a linear network. Dynamic recommendation decisions are made based on the prediction results. Specifically:

[0034] For each news item in the candidate news set, a corresponding multimodal coding representation is generated using the same time-aware multimodal coding as historical news.

[0035] Modality-aware matching is performed between the user interest vector and the multimodal encoded representation of the candidate news. The matching scores of the text and visual modalities are combined and normalized by the Sigmoid function to obtain the initial matching score.

[0036] A predefined taboo time period rule is established, and a time period-category inhibition matrix is ​​defined. Based on the time period-category inhibition matrix, time period-sensitive inhibition is applied to the initial matching score through temporal gating. Reinforcement learning is introduced to dynamically adjust the inhibition strength. The matching score after applying time period-sensitive inhibition is... Represented as:

[0037] ,

[0038] in Indicates the initial matching score. Indicate category During forbidden periods The intensity of inhibition, Indicates the total number of categories. Indicates an indicator function, Indicates the current moment;

[0039] The matching scores after applying time-sensitive suppression are input into the fully connected layer, and the output click probability prediction value is used. Softmax is used to normalize the click probability prediction values ​​of all candidate news items to obtain the final recommendation probability distribution.

[0040] A preset number of candidate news items are selected for recommendation based on the final recommendation probability distribution, and category diversification constraints are introduced to avoid homogeneous recommendations.

[0041] In this solution, the user interest database is updated through online learning, specifically as follows:

[0042] Real-time capture of user interaction behavior on the front-end interface, recording the corresponding news ID, timestamp and behavior type, and obtaining the multimodal encoding representation of the news of the new interaction. Based on the behavior timestamp, the time period to which it belongs is determined and time period positioning is performed.

[0043] The interest vector is updated in real time using an exponential moving average over the specified time period. Represented as:

[0044] ,

[0045] in Indicates the historical memory screening coefficient. Multimodal encoding representation of new interactive news. This indicates a new interactive news collection. This represents the historical interest vector for that time period;

[0046] The KL divergence between the new and old interest vectors of a time period is used to represent the degree of interest drift. The KL divergence is compared with a preset threshold to make a judgment, and the model is fine-tuned based on the judgment result.

[0047] A second aspect of the present invention provides a multimodal news recommendation system that incorporates clock interests, the system comprising a multimodal feature encoding module, a clock interest modeling module, a dynamic recommendation decision module, and an online learning module;

[0048] The multimodal feature encoding module extracts historical interactive news that meets preset time requirements from user behavior sequences and related timestamp sets, performs time-aware multimodal feature encoding on the historical interactive news, and obtains the multimodal encoded representation of the historical interactive news;

[0049] The clock interest modeling module performs clock interest modeling based on the multimodal encoding representation of historical interactive news. Through Gaussian weighted aggregation and long-term interest enhancement operations, it generates vector representations of users' short-term and long-term interests, and merges short-term and long-term interests to obtain user interest vectors.

[0050] The dynamic recommendation decision module calculates the matching degree between candidate news and the current user's interest vector through a cross-modal attention mechanism, applies time-sensitive inhibition through time-series gating, uses a linear network to predict click probability for the output of time-series gating, and makes dynamic recommendation decisions based on the prediction results.

[0051] The online learning module employs a streaming interest distillation mechanism to achieve real-time updates while avoiding sudden noise interference, ensuring that the recommendation system can promptly reflect the dynamic changes in user interests.

[0052] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0053] This invention proposes a multimodal news recommendation framework that integrates neurobiological clock theory, overcoming the modeling limitations of static profiles and improving the accuracy of temporal correlation modeling of multimodal features. Through a cross-modal temporal gating component, dynamic alignment of text and image content with temporal context is achieved, automatically adjusting the feature fusion strategy based on time period characteristics. A lightweight streaming update pipeline is constructed, completing interest drift detection and model adjustment under low-latency constraints, effectively solving the problem of interest drift accumulation in traditional streaming recommendation. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments or examples of the present invention, the drawings used in the embodiments or examples will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained according to these drawings without creative effort.

[0055] Figure 1 A flowchart of a multimodal news recommendation method incorporating clock-based interests is shown;

[0056] Figure 2 The flowchart for generating short-term and long-term user interest representations is shown;

[0057] Figure 3 The flowchart for making dynamic recommendation decisions is shown;

[0058] Figure 4 A flowchart of a multimodal news recommendation method incorporating clock interests is shown.

[0059] Figure 5 A block diagram of a multimodal news recommendation system incorporating clock-based interests is shown. Detailed Implementation

[0060] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0061] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0062] like Figure 1 and Figure 4 As shown, this embodiment provides a multimodal news recommendation method that combines clock interests, including:

[0063] S102, construct a time-series interaction sequence from the user's historical behavior log, and extract a user behavior sequence, a candidate news set, and a relevant timestamp set based on the time-series interaction sequence;

[0064] S104, extract historical interactive news that meets the preset time requirements from the user behavior sequence and related timestamp set, perform time-aware multimodal feature encoding on the historical interactive news, and obtain the multimodal encoded representation of the historical interactive news;

[0065] S106, Based on the multimodal encoding representation of the historical interactive news, clock interest modeling is performed. Through Gaussian weighted aggregation and long-term interest enhancement operations, vector representations of the user's short-term and long-term interests are generated, and the user's interest vector is obtained by fusing short-term and long-term interests.

[0066] S108 calculates the matching degree between candidate news and the current user interest vector, applies time-sensitive suppression through time-series gating, uses a linear network to predict click probability for the output of time-series gating, makes dynamic recommendation decisions based on the prediction results, and updates the user interest database through online learning.

[0067] It should be noted that user ID, news ID, interaction behavior, and timestamp are extracted from user historical behavior logs as raw data. The raw data is then cleaned to remove duplicate records and invalid data to ensure the integrity and accuracy of the data.

[0068] The cleaned raw data is grouped by user ID, and all interactive behaviors of each user are categorized into the same group, generating an interactive behavior dataset for each user. This dataset is then sorted in ascending order by timestamp to ensure that the user's behavior sequence is arranged chronologically, forming a time-series interactive sequence. User interactive behaviors are extracted from this time-series interactive sequence to construct a user behavior sequence. , Indicates the user's timestamp Interactive behavior, , The total number of timestamps; extract the timestamp corresponding to each interaction to construct a relevant timestamp sequence. , Indicates user interaction behavior The time of occurrence, And obtain all news in the current system to construct a candidate news set. , Indicates candidate news, , This represents the total number of news items. The constructed time-series interaction sequences, user behavior sequences, candidate news sets, and related timestamp sets are stored in the database for easy model retrieval. A periodic or real-time update mechanism is set in the database to ensure the data reflects the latest user behavior dynamics.

[0069] It should be noted that, based on the recommendation requirements, a preset time window is used, such as the past 24 hours, the past week, or a specific time period like "7-9 am". This preset time window is then used to traverse user behavior sequences and related timestamp sets, extracting interaction records that meet the preset time conditions, and filtering news items that users have interacted with within the preset time window. The filtered news IDs and associated multimodal data are then combined to form a historical interaction news set. , Each of these news items Including both text and image content, a pre-trained BERT model was used to semantically encode the news text content, resulting in an initial text feature vector. Extract short-segment information of news interaction time and use a sine function to encode time features. Through linear transformation Mapping temporal features to the semantic space, and comparing them with BERT text features. By injecting time-aware capabilities, the initial text feature vector is enhanced to obtain a time-enhanced text representation. , represented as:

[0070] ,

[0071] in Encode the current time using a sine function. , Temporal features are projected into the semantic space. Temporally enhanced text feature encoding ensures that text features not only contain semantic information but also incorporate temporal context.

[0072] Mask R-CNN was used to detect targets in news images and extract salient regions. The salient regions and news text are input into the ViLBERT model, and a visual-semantic aligned feature representation is generated through a cross-modal attention mechanism. , represented as:

[0073] ,

[0074] in Indicates the total number of significant regions. Indicates the first A significant region, This indicates the content of the news text. This represents the output of the ViLBERT model. The cross-modal attention mechanism in ViLBERT ensures that visual concepts are aligned with the textual context.

[0075] The temporally enhanced text representation is concatenated with the visual-semantic aligned feature representation to generate the final multimodal encoded representation. , Multimodal coding representation It also contains semantic, visual, and temporal information, providing input for subsequent clock interest modeling.

[0076] Figure 2 The flowchart for generating user short-term and long-term interest representations is shown.

[0077] According to an embodiment of the present invention, clock interest modeling is performed based on the multimodal encoding representation of the historical interactive news. Through Gaussian weighted aggregation and long-term interest enhancement operations, a context vector that comprehensively reflects the user's short-term and long-term interests is generated, providing important interest basis for dynamic recommendation decisions. Specifically:

[0078] S202, Building a User Interest Database Based on 24-Hour Dimensions ,in Storage users in The multimodal encoding representation of historical interactive news for a given time period is updated in the interest library whenever a user interacts in a new time period.

[0079] Whenever a user interacts during a new time period, the multimodal encoding of the corresponding news item is updated in the interest database for that time period. For example, if a user clicks on a news item at 10:00 AM, the database is updated accordingly. .

[0080] S204 uses Gaussian weighted aggregation for short-term interest modeling, employing a Gaussian kernel function with the current time step as the basis. Calculate adjacent time periods centered on the target. The weights of the interest vectors for 24 time periods. By performing weighted summation, a smoothed short-term interest representation is obtained. , represented as:

[0081] ,

[0082] in, Indicates adjacent time periods The weight, To control the smoothing intensity, testing revealed that 1 hour was insufficient for acquiring information, while 2 hours resulted in excessive noise. Therefore, the duration was set to 1.5 hours. This represents a radial basis function used to smooth out abrupt changes in interest at time boundaries, such as when interest suddenly switches from sports to work-related news after "lunch break ends".

[0083] S206: Retrieve relevant news from users' historical data across all time periods to enhance long-term interest representation. Filter historical news with semantic similarity to the current candidate news from the historical interaction news set, and calculate the weight of each historical news item based on semantic relevance and time decay. Multimodal encoding representation of weighted aggregated historical news Generate long-term interest representations :

[0084] ,

[0085] in, This represents a collection of historical interactive news articles.

[0086] To avoid over-reliance on recent interests, relevant news is retrieved from users' historical data across all time periods. Long-term interest enhancement is performed by calculating weights for each historical news item based on semantic relevance and time decay. , represented as:

[0087] ,

[0088] in Represents the time decay function. Control the decay rate, This represents the transpose of the multimodal encoded representation of the current candidate news. , This represents the learnable parameters.

[0089] S208 represents the short-term interest characterization of Gaussian aggregation. and long-term interests The vectors are concatenated into a high-dimensional vector, and then dimensionality is reduced using a learnable matrix to generate the final user interest vector. , represented as:

[0090] ,

[0091] in, This represents a learnable fusion matrix. The dimension is The real space. The final user interest vector. It also includes the user's preferences for the current time period and stable interests across time periods, providing a basis for dynamic recommendation decisions.

[0092] Figure 3 A flowchart for making dynamic recommendation decisions is shown.

[0093] According to an embodiment of the present invention, the matching degree between candidate news and the current user's interest vector is calculated by combining cross-modal attention, and time-sensitive suppression is applied by time-series gating. The output of time-series gating is used to predict click probability using a linear network, and dynamic recommendation decisions are made based on the prediction results, thereby providing users with recommended news that better matches their current interests and time-series preferences. Specifically:

[0094] S302, For each news item in the candidate news set, use the same time-aware multimodal coding as historical news to generate a corresponding multimodal coding representation;

[0095] S304, combining user interest vectors with candidate news Multimodal encoding is used for modality-aware matching. The matching scores of text and visual modalities are combined and normalized using the Sigmoid function to obtain the initial matching score. , represented as:

[0096] ,

[0097] in Representing different modes The corresponding user interest components include the text components of user interests. and the visual weight of user interests , Indicating different modalities of news The corresponding multimodal coding representation includes news text features and news image features. An initial matching score, obtained by combining text matching degree and visual matching degree, measures the degree of semantic and visual fit between candidate news and the user's current interests.

[0098] S306, a forbidden time period rule is preset, a time period-category inhibition matrix is ​​defined, and time period-sensitive inhibition is applied to the initial matching score based on the time period-category inhibition matrix through temporal gating, and reinforcement learning is introduced to dynamically adjust the inhibition strength;

[0099] Preset rules regarding restricted time periods, such as not recommending entertainment news late at night, and corresponding restricted time periods. Inhibition strength If the current time falls within a certain category of forbidden time periods, suppression is triggered, and the matching score after applying time-sensitive suppression is adjusted. Represented as:

[0100] ,

[0101] in Indicates the initial matching score. Indicate category During forbidden periods The intensity of inhibition, This indicates the total number of categories. If the current time is 1 AM and the news category is entertainment, the match score is reduced by 80%. Indicates an indicator function, Indicates the current moment, when The value is 1 if the condition is met, and 0 otherwise. The suppression strength is dynamically adjusted through reinforcement learning. For example, if a user frequently clicks on a suppressed news item, the suppression strength is reduced.

[0102] S308 inputs the matching score after applying time-sensitive suppression into the fully connected layer, outputs the click probability prediction value, and uses Softmax to normalize the click probability prediction values ​​of all candidate news to obtain the final recommendation probability distribution;

[0103] Click probability prediction using a linear network , represented as , express Activation function This represents a multilayer perceptron, which, as a nonlinear transformation function, is responsible for learning the complex mapping relationship between matching scores and the final click probability.

[0104] S310: Select a preset number of candidate news items for recommendation based on the final recommendation probability distribution, and introduce category diversification constraints to avoid homogenization of recommendations.

[0105] The generated news recommendation list is returned to the user's front-end interface, displayed according to priority, and user feedback on the recommendation results is recorded for subsequent model optimization of the online learning module.

[0106] It should be noted that the online learning module is responsible for updating the user's interest library in real time to adapt to dynamic changes in user interests. It captures user interactions on the front-end interface in real time, such as clicks, browsing duration, and bookmarks, recording the corresponding news ID, timestamp, and behavior type. It also obtains the multimodal encoding representation of the news with the new interaction, determines the corresponding time period based on the behavior timestamp, and performs time-segment positioning.

[0107] The interest vector is updated in real time using an exponential moving average over the specified time period. Represented as:

[0108] ,

[0109] in This represents the historical memory screening coefficient, which determines the degree to which historical information is retained. Multimodal encoding representation of new interactive news. This indicates a new interactive news collection. This represents the historical interest vector for that time period;

[0110] Calculate the KL divergence between the old and new interest vectors over the time period. The degree of interest drift is characterized by comparing the KL divergence with a preset threshold. Based on the judgment result, the model is fine-tuned, including local and global adjustments. Local adjustments only update the gradients of parameters for relevant time periods in the clock interest modeling module. If drift is detected in multiple consecutive time periods, a global adjustment is performed, initiating full model fine-tuning. The interest library is fully scanned weekly, merging interests from similar time periods, deleting expired interests, and supplementing the interest library for new or inactive users using collaborative filtering.

[0111] It should be noted that during model training, end-to-end optimization is achieved through multiple rounds of iteration. Each round sequentially executes time-aware multimodal encoding (integrating Gaussian temporal convolution and temporal gating mechanisms), cross-modal attention fusion, and clock interest modeling. In the loss calculation stage, the cross-entropy loss function L is used, and the parameters are jointly updated using the Adam optimizer. Simultaneously, online adaptive adjustment of model parameters is achieved based on a streaming interest distillation mechanism. The click-through rate prediction probability is output through linear projection transformation and the Sigmoid activation function.

[0112] Figure 5 A block diagram of a multimodal news recommendation system incorporating clock-based interests is shown.

[0113] The second embodiment of the present invention provides a multimodal news recommendation system 5 that combines clock interests, including: a multimodal feature encoding module 501, a clock interest modeling module 502, a dynamic recommendation decision module 503, and an online learning module 504;

[0114] The multimodal feature encoding module extracts historical interactive news that meets preset time requirements from user behavior sequences and related timestamp sets, performs time-aware multimodal feature encoding on the historical interactive news, and obtains the multimodal encoded representation of the historical interactive news;

[0115] It should be noted that text encoding uses a pre-trained BERT model to extract semantic features and injects practical awareness; image encoding uses Mask R-CNN to detect salient regions and inputs them into ViBERT to generate visual-language aligned features. The cross-modal attention mechanism in ViBERT ensures alignment between visual concepts and textual context. The multimodal encoding module enhances the temporal context of BERT text encoding and ViLBERT visual encoding, providing richer temporal dimension information for subsequent interest analysis.

[0116] The clock interest modeling module performs clock interest modeling based on the multimodal encoding representation of historical interactive news. Through Gaussian weighted aggregation and long-term interest enhancement operations, it generates vector representations of users' short-term and long-term interests, and merges short-term and long-term interests to obtain user interest vectors.

[0117] It should be noted that, to capture users' day-night interest patterns, an hourly interest database is maintained for each user. Furthermore, Gaussian weighted aggregation of interests from adjacent time periods is used for interest activation at the current moment. In addition, to avoid over-reliance on recent interests, relevant news is retrieved from users' historical data across all time periods for long-term interest enhancement. The clock interest modeling module, based on the multimodal feature representation of historical news, aggregates interest patterns from adjacent time periods through Gaussian temporal convolution, effectively integrating changes in user interests across different time periods.

[0118] The dynamic recommendation decision module calculates the matching degree between candidate news and the current user's interest vector through a cross-modal attention mechanism, applies time-sensitive inhibition through time-series gating, uses a linear network to predict click probability for the output of time-series gating, and makes dynamic recommendation decisions based on the prediction results.

[0119] It should be noted that the click probability of candidate news is calculated through cross-modal attention, and the matching degree between the candidate news and the user's current interests is calculated through modality-aware matching. Furthermore, to avoid inappropriate behavior such as recommending entertainment news late at night, time-sensitive suppression is performed through time-series gating, thereby providing users with news recommendations that match their current interests and time-series preferences. The dynamic recommendation decision module uses interest vectors generated by clock-based interest modeling, combined with cross-modal attention and time-series gating mechanisms, to dynamically adjust the weights of image and text features according to the characteristics of different time periods, and integrates the time-series salience of visual elements to achieve accurate matching between image and text content and temporal context.

[0120] The online learning module employs a streaming interest distillation mechanism to achieve real-time updates while avoiding sudden noise interference, ensuring that the recommendation system can promptly reflect the dynamic changes in user interests.

[0121] It should be noted that the interest database is updated in real time using exponential moving averages. The new state of the interest database is jointly determined by the old state and the new news interaction features. Then, KL divergence detection is used to detect interest drift and fine-tune the model. The online learning module uses a streaming interest distillation mechanism to achieve real-time updates while avoiding sudden noise interference, ensuring that the recommendation system can reflect the dynamic changes in user interests in a timely manner.

[0122] A third aspect of the present invention provides a computer-readable storage medium comprising a multimodal news recommendation method program incorporating clock interests, wherein when executed by a processor, the multimodal news recommendation method program incorporating clock interests implements the steps of a multimodal news recommendation method incorporating clock interests.

[0123] In the several embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be indirect coupling or communication connection through some interfaces, devices, or modules, and can be electrical, mechanical, or other forms. Furthermore, in the various embodiments of the present invention, all functional modules can be integrated into one processing module, or each module can be a separate module, or two or more modules can be integrated into one module; the integrated modules can be implemented in hardware or in the form of hardware plus software functional modules.

[0124] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0125] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A multimodal news recommendation method incorporating clock-based interests, characterized in that, Includes the following steps: Construct a time-series interaction sequence from user historical behavior logs, and extract user behavior sequences, candidate news sets, and related timestamp sets based on the time-series interaction sequence; Extract historical interactive news that meets preset time requirements from the user behavior sequence and related timestamp set, perform time-aware multimodal feature encoding on the historical interactive news, and obtain the multimodal encoded representation of the historical interactive news; Clock interest modeling is performed based on the multimodal encoding representation of the historical interactive news. Through Gaussian weighted aggregation and long-term interest enhancement operations, vector representations of users' short-term and long-term interests are generated, and user interest vectors are obtained by fusing short-term and long-term interests. The matching degree between candidate news and the current user interest vector is calculated, and time-sensitive suppression is applied through time-series gating. The output of time-series gating is used to predict the click probability through a linear network. Dynamic recommendation decisions are made based on the prediction results, and the user interest database is updated through online learning. Based on the multimodal encoding representation of the historical interactive news, clock interest modeling is performed. Through Gaussian weighted aggregation and long-term interest enhancement operations, context vectors of users' short-term and long-term interests are generated, specifically: Build a user interest database based on 24-hour timeframe ,in Storage users in The multimodal encoding representation of historical interactive news for a given time period is updated in the interest library whenever a user interacts in a new time period. Short-term interest modeling is performed using Gaussian weighted aggregation, with the Gaussian kernel function applied at the current time step. Calculate adjacent time periods centered on the target. The weights of the interest vectors for 24 time periods. By performing weighted summation, a smoothed short-term interest representation is obtained. , represented as: ， in, Indicates adjacent time periods The weight, Indicates smoothness intensity; Retrieve relevant news from users' historical data across all time periods to enhance long-term interest representation. Filter historical news with semantic similarity to current candidate news from the historical interaction news set, and calculate the weight of each historical news item based on semantic relevance and time decay. ; Multimodal coding representation of weighted aggregated historical news Generate long-term interest representations : ， in, This represents a collection of historical interactive news articles.

2. The multimodal news recommendation method combining clock interests according to claim 1, characterized in that, A time-series interaction sequence is constructed from user historical behavior logs. Based on the time-series interaction sequence, user behavior sequences, candidate news sets, and related timestamp sets are extracted, specifically as follows: Extract user ID, news ID, interaction behavior and timestamp as raw data from user history behavior logs, and perform data cleaning on the raw data to remove duplicate records and invalid data; The cleaned raw data is grouped according to user ID to generate an interaction behavior dataset for each user. The interaction behavior dataset is then sorted in ascending order according to timestamp to form a time-series interaction sequence. Extract user interaction behaviors and timestamps corresponding to each interaction behavior from the time-series interaction sequence, construct user behavior sequences and related timestamp sequences, and obtain all news in the current system to construct a candidate news set.

3. The multimodal news recommendation method combining clock interests according to claim 1, characterized in that, Historical interaction news that meets preset time requirements is extracted from the user behavior sequence and related timestamp set. Time-aware multimodal feature encoding is performed on the historical interaction news to obtain a multimodal encoded representation of the historical interaction news, specifically: Based on the recommendation requirements, a preset time window is used to traverse the user behavior sequence and related timestamp set, extract the interaction records that meet the preset time conditions, and filter the news that the user has interacted with within the preset time window. The selected news IDs and associated multimodal data are combined to form a historical interactive news set, where each news item includes text content and image content. A pre-trained BERT model is used to semantically encode the news text content to obtain an initial text feature vector. Extract short-segment information of news interaction time, encode time features using a sine function, map the time features to the semantic space through linear transformation, add them to BERT text features, and enhance the initial text feature vector to obtain a time-enhanced text representation; Mask R-CNN is used to detect targets in news images and extract salient regions. The salient regions and news text are then input into the ViLBERT model, which generates visual-semantic aligned feature representations through a cross-modal attention mechanism. The temporally enhanced text representation is concatenated with the visual-semantic aligned feature representation to generate the final multimodal encoded representation.

4. The multimodal news recommendation method combining clock interests according to claim 1, characterized in that, The user interest vector is obtained by fusing long-term and short-term interests, specifically as follows: Characterizing short-term interests through Gaussian aggregation and long-term interests The vectors are concatenated into a high-dimensional vector, and then dimensionality is reduced using a learnable matrix to generate the final user interest vector. , represented as: ， in, This represents a learnable fusion matrix. Indicates size is The real number space.

5. The multimodal news recommendation method combining clock interests according to claim 1, characterized in that, The matching degree between candidate news items and the current user's interest vector is calculated, and time-sensitive suppression is applied through time-series gating. The output of the time-series gating is then used to predict click probabilities using a linear network. Dynamic recommendation decisions are made based on the prediction results. Specifically: For each news item in the candidate news set, a corresponding multimodal coding representation is generated using the same time-aware multimodal coding as historical news. Modality-aware matching is performed between the user interest vector and the multimodal encoded representation of the candidate news. The matching scores of the text and visual modalities are combined and normalized by the Sigmoid function to obtain the initial matching score. A time-class suppression matrix is ​​defined. Based on this matrix, time-sensitive suppression is applied to the initial matching score through temporal gating. Reinforcement learning is then introduced to dynamically adjust the suppression strength. The matching score after applying time-sensitive suppression is... Represented as: ， in Indicates the initial matching score. Indicates category During forbidden periods The intensity of inhibition, Indicates the total number of categories. Indicates an indicator function, Indicates the current time; The matching scores after applying time-sensitive suppression are input into the fully connected layer, and the output click probability prediction value is used. Softmax is used to normalize the click probability prediction values ​​of all candidate news items to obtain the final recommendation probability distribution. A preset number of candidate news items are selected for recommendation based on the final recommendation probability distribution, and category diversification constraints are introduced to avoid homogeneous recommendations.

6. The multimodal news recommendation method combining clock interests according to claim 1, characterized in that, The user interest database is updated through online learning, specifically as follows: Capture user interaction behavior in real time, record the corresponding news ID, timestamp and behavior type, obtain the multimodal encoding representation of the news of the new interaction, determine the time period to which it belongs based on the behavior timestamp, and perform time period positioning; The interest vector is updated in real time using an exponential moving average over the specified time period. Represented as: ， in Indicates the historical memory screening coefficient. Multimodal encoding representation of new interactive news. This indicates a new interactive news collection. This represents the historical interest vector for that time period; The KL divergence between the new and old interest vectors of a time period is used to represent the degree of interest drift. The KL divergence is compared with a preset threshold to make a judgment, and the model is fine-tuned based on the judgment result.

7. A multimodal news recommendation system incorporating clock-based interests, characterized in that, To implement the multimodal news recommendation method combining clock interests as described in any one of claims 1-6, the system includes a multimodal feature encoding module, a clock interest modeling module, a dynamic recommendation decision module, and an online learning module; The multimodal feature encoding module extracts historical interactive news that meets preset time requirements from user behavior sequences and related timestamp sets, performs time-aware multimodal feature encoding on the historical interactive news, and obtains the multimodal encoded representation of the historical interactive news; The clock interest modeling module performs clock interest modeling based on the multimodal encoding representation of historical interactive news. Through Gaussian weighted aggregation and long-term interest enhancement operations, it generates vector representations of users' short-term and long-term interests, and merges short-term and long-term interests to obtain user interest vectors. The dynamic recommendation decision module calculates the matching degree between candidate news and the current user's interest vector through a cross-modal attention mechanism, applies time-sensitive inhibition through time-series gating, uses a linear network to predict click probability for the output of time-series gating, and makes dynamic recommendation decisions based on the prediction results. The online learning module employs a streaming interest distillation mechanism to achieve real-time updates while avoiding sudden noise interference, ensuring that the recommendation system can promptly reflect the dynamic changes in user interests.

Images