Multi-modal news recommendation method and system in combination with clock interests

Abstract

The invention discloses a multi-modal news recommendation method and system in combination with clock interests, and relates to the technical field of data mining and recommendation methods.The method comprises the steps that a user behavior sequence, a candidate news set and a related timestamp set are extracted; obtaining the multi-modal coding representation of the historical interaction news through the time perception multi-modal feature coding; performing clock interest modeling based on multi-modal coding representation of historical interactive news, and fusing long and short-term interests through Gaussian weighted aggregation and long-term interest enhancement operation to obtain a user interest vector; and calculating the matching degree of the candidate news and the current user interest vector, applying time period sensitive suppression through time sequence gating, performing click probability prediction on the output of the time sequence gating by adopting a linear network, and performing a dynamic recommendation decision. According to the method, by means of the feature fusion technology of time context perception, the interest continuity of the hour granularity is captured, and the recommended content is more accurately matched with the requirements of the user in different time periods.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data mining and recommendation method, more particularly, to a multi-modal news recommendation method and system combined with clock interest. BACKGROUND

[0002] In the current digital era, personalized news recommendation systems have become one of the core technologies to alleviate information overload. Traditional recommendation systems build static interest profiles by mining user historical behavior, achieving remarkable results in e-commerce, video, and other fields. For the news recommendation field, multi-modal fusion technology has become a research hotspot in recent years. Existing news recommendation frameworks based on multi-modal knowledge graph build cross-modal knowledge graph by crawling news text and image data, use the VilBERT pre-training model to realize semantic alignment of text and image, and verify the effectiveness of cross-modal representation. However, the news reading scenario has a unique time-sensitive nature, with users tending to browse international political news during morning commute, preferring sports information during lunch break, and possibly focusing on entertainment at night. This fluctuating interest pattern over time presents new requirements for dynamic adaptation of recommendation systems. Although existing research has made some progress in cross-modal recommendation and time series modeling, few works have systematically explored the impact of the circadian rhythm characteristics of user interest on news consumption.

[0003] The current mainstream method has the following three shortcomings: first, the static recommendation framework based on collaborative filtering cannot capture the interest drift at the hour level, resulting in irrelevant content being received by users at a specific time; second, existing time series recommendation models focus more on long-term interest evolution, ignoring the daily periodic pattern (such as the difference between weekdays and weekends); finally, multi-modal recommendation systems often simply concatenate text and image features, failing to establish a dynamic association between temporal context and multi-modal content. Recent research has shown that user click behavior exhibits significant fluctuations within a 24-hour cycle, but existing systems still use a fixed time window update strategy, causing a mismatch between the recommended results and real-time needs. To address the above challenges, especially to systematically explore the impact of the circadian rhythm characteristics of user interest on news, there is an urgent need to develop a multi-modal news recommendation method combined with clock interest. SUMMARY

[0004] To solve the above technical problems, the present application provides a multi-modal news recommendation method and system combined with clock interest, which captures the interest continuity at the hour level through time context-aware feature fusion technology to improve the accuracy of recommendation.

[0005] The first aspect of the present application provides a multi-modal news recommendation method combined with clock interest, comprising the following steps: constructing a time series interaction sequence from user historical behavior logs, extracting a user behavior sequence, a candidate news set, and a related timestamp set based on the time series interaction sequence; extract historical interactive news meeting preset time requirements from the user behavior sequence and the related timestamp set, perform time-aware multi-modal feature coding on the historical interactive news, and obtain multi-modal coding representation of the historical interactive news; Based on the multi-modal coding representation of the historical interactive news, clock interest modeling is performed, and through Gaussian weighted aggregation and long-term interest enhancement operation, vector representation of user short-term and long-term interest is generated, and user interest vector is obtained by fusing long and short-term interest; Calculate the matching degree of the candidate news and the current user interest vector, and apply time period sensitive inhibition through time sequence gating, use linear network to predict the click probability of the output of the time sequence gating, make dynamic recommendation decision according to the prediction result, and update the user interest library through online learning.

[0006] In this scheme, a time sequence interaction sequence is constructed from the user historical behavior log, and the user behavior sequence, the candidate news set and the related timestamp set are extracted based on the time sequence interaction sequence, specifically: Extract user ID, news ID, interaction behavior and timestamp from user historical behavior log as original data, and perform data cleaning on the original data to remove duplicate records and invalid data; Group the original data after data cleaning according to user ID, generate interaction behavior data set of each user, and sort the interaction behavior data set in ascending order according to timestamp to form a time sequence interaction sequence; Extract the interaction behavior of the user and the timestamp corresponding to each interaction behavior from the time sequence interaction sequence, construct the user behavior sequence and the related timestamp sequence, and obtain all the news in the current system to construct the candidate news set.

[0007] In this scheme, historical interactive news meeting preset time requirements are extracted from the user behavior sequence and the related timestamp set, and time-aware multi-modal feature coding is performed on the historical interactive news to obtain multi-modal coding representation of the historical interactive news, specifically: According to the recommendation requirement, preset a time window, use the preset time window to traverse the user behavior sequence and the related timestamp set, extract the interaction records meeting the preset time condition, and filter the news interacted by the user within the preset time window; The filtered news ID and associated multi-modal data form a historical interactive news set, wherein each news includes text content and image content, and a pre-trained BERT model is used to perform semantic coding on the news text content to obtain an initial text feature vector; Extract the hourly period information of news interaction time, encode the time features using a sine function, map the time features to a semantic space through a linear transformation, add them to the BERT text features, and enhance the initial text feature vector to obtain a time-enhanced text representation; Use Mask R-CNN to detect objects in news images and extract salient regions. These salient regions and news text are then fed into the ViLBERT model, where a cross-modal attention mechanism is used to generate visually-semantically aligned feature representations. The temporally enhanced text representation is concatenated with the visual-semantic aligned feature representation to generate the final multimodal encoding representation.

[0008] In this solution, clock interest modeling is performed based on the multimodal encoding representation of the historical interactive news. Through Gaussian weighted aggregation and long-term interest enhancement operations, context vectors of users' short-term and long-term interests are generated. Specifically, Build a 24-hour user interest database ,in Store user Multimodal encoding representation of historical interactive news in a time period. Whenever a user interacts in a new time period, the multimodal encoding representation of the corresponding news is updated to the corresponding time period in the interest library. Use Gaussian weighted aggregation for short-term interest modeling, and use the Gaussian kernel function to calculate the current moment As the center, calculate the adjacent time periods The weight of interest vector for 24 time periods Perform weighted summation to obtain a smoothed short-term interest representation , expressed as: , in, Indicates adjacent time periods The weight of Indicates the smoothing strength; Retrieve relevant news from the user's full-time historical data, enhance long-term interest representation, filter historical news with similar semantics to the current candidate news in the historical interactive news collection, and calculate the weight for each historical news based on semantic relevance and time decay. ; Multimodal encoding representation of weighted aggregated historical news , generating long-term interest representations : , in, Represents a collection of historical interactive news.

[0009] In this solution, the user interest vector is obtained by integrating long-term and short-term interests, specifically: Short-term interest representation of Gaussian aggregation and long-term interest Spliced into a high-dimensional vector, reduced by a learnable matrix, and a final user interest vector is generated , denoted as: , wherein, denotes a learnable fusion matrix, denotes a real number space with a dimension of .

[0010] In the scheme, the matching degree of the candidate news and the current user interest vector is calculated, and the time period sensitive inhibition is applied through the time sequence gate, the output of the time sequence gate is used for click probability prediction by a linear network, and dynamic recommendation decision is made according to the prediction result, specifically: For each news in the candidate news set, the same time-aware multi-modal encoding as the historical news is used to generate the corresponding multi-modal encoding representation; The user interest vector and the multi-modal encoding representation of the candidate news are matched in modal perception, the matching scores of the text and visual modalities are integrated, and the initial matching score is obtained by normalizing through the Sigmoid function; Pre-set taboo period rules, define a period-category inhibition matrix, apply time-sensitive inhibition to the initial matching score based on the period-category inhibition matrix through the time sequence gate, and introduce reinforcement learning to dynamically adjust the inhibition strength, and the matching score after applying the time-sensitive inhibition denoted as: , wherein denotes the initial matching score, denotes the inhibition strength of the category in the taboo period , denotes the total number of categories, denotes an indicator function, denotes the current time; The matching score after applying the time-sensitive inhibition is input into a fully connected layer, and a click probability prediction value is output, and the click probability prediction values of all candidate news are normalized by using Softmax to obtain a final recommendation probability distribution; According to the final recommendation probability distribution, a predetermined number of candidate news are selected for recommendation, and a category diversification constraint is introduced to avoid homogenization of the recommendation.

[0011] In the scheme, the user interest library is updated through online learning, specifically: Real-time capture of user interaction behavior in front-end interface, record corresponding news ID, timestamp and behavior type, and obtain multi-modal encoding representation of new interaction news, determine the hour segment according to the behavior timestamp, and perform time period positioning; The time period is updated in real time by an exponential moving average, and the updated interest vector is expressed as: , wherein represents a historical memory screening coefficient, represents a multi-modal encoding representation of the new interaction news, represents a set of new interaction news, represents a historical interest vector of the time period. The KL divergence of the new and old interest vectors of the time period represents the interest drift degree, the KL divergence is compared with a preset threshold, and the model is fine-tuned according to the judgment result.

[0012] The second aspect of the application provides a multi-modal news recommendation system combined with clock interest, which comprises a multi-modal feature encoding module, a clock interest modeling module, a dynamic recommendation decision module and an online learning module. The multi-modal feature encoding module extracts historical interaction news meeting a preset time requirement from a user behavior sequence and a related timestamp set, performs time-aware multi-modal feature encoding on the historical interaction news, and obtains a multi-modal encoding representation of the historical interaction news. The clock interest modeling module performs clock interest modeling based on the multi-modal encoding representation of the historical interaction news, generates vector representations of user short-term and long-term interests through Gaussian weighted aggregation and long-term interest enhancement operations, and obtains a user interest vector by fusing long-term and short-term interests. The dynamic recommendation decision module calculates the matching degree of candidate news and the current user interest vector through a cross-modal attention mechanism, applies time period sensitive inhibition through a time sequence gate, uses a linear network to predict the click probability of the output of the time sequence gate, and makes a dynamic recommendation decision according to the prediction result. The online learning module uses a streaming interest distillation mechanism to realize real-time updating while avoiding sudden noise interference, and ensures that the recommendation system can timely reflect the dynamic changes of user interest.

[0013] Compared with the prior art, the application has the following beneficial effects: The application proposes a multi-modal news recommendation framework combined with the theory of neural biological clock, breaks through the modeling limitation of static portrait, and improves the accuracy of time correlation modeling of multi-modal features. Through the cross-modal time sequence gating component, dynamic alignment of image-text content and time context is realized, and the feature fusion strategy is automatically adjusted according to the time period characteristics. A lightweight streaming update pipeline is constructed, and interest drift detection and model adjustment are completed under low delay constraints, effectively solving the problem of interest drift accumulation in traditional streaming recommendation. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments or examples of the present application, the drawings needed to be used in the embodiments or example descriptions will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained according to the drawings shown by those skilled in the art without creative labor.

[0015] Figure 1 A flowchart of a multi-modal news recommendation method combined with clock interest is shown; Figure 2 A flowchart of generating user short-term and long-term interest representation is shown; Figure 3 A flowchart of dynamic recommendation decision-making is shown; Figure 4 A flowchart of a multi-modal news recommendation method combined with clock interest is shown; Figure 5 A block diagram of a multi-modal news recommendation system combined with clock interest is shown. DETAILED DESCRIPTION

[0016] In order to more clearly illustrate the technical solutions in the embodiments or examples of the present application, the drawings needed to be used in the embodiments or example descriptions will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained according to the drawings shown by those skilled in the art without creative labor.

[0017] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, however, the present application can also be implemented in other ways different from those described herein, therefore, the protection scope of the present application is not limited by the specific embodiments disclosed below.

[0018] As shown in Figure 1 and Figure 4 The present embodiment provides a multi-modal news recommendation method combined with clock interest, comprising: S102, constructing a time sequence interaction sequence from a user historical behavior log, extracting a user behavior sequence, a candidate news set and a related timestamp set based on the time sequence interaction sequence; S104, extract historical interactive news meeting preset time requirements from the user behavior sequence and related timestamp set, time-aware multi-modal feature encoding is performed on the historical interactive news, and multi-modal encoding representation of the historical interactive news is obtained; S106, clock interest modeling is performed based on the multi-modal encoding representation of the historical interactive news, vector representation of user short-term and long-term interests is generated through Gaussian weighted aggregation and long-term interest enhancement operation, user interest vector is obtained by fusing long-term and short-term interests; S108, the matching degree of the candidate news and the current user interest vector is calculated, and time period sensitive inhibition is applied through time sequence gating, click probability prediction is performed on the output of the time sequence gating by using a linear network, dynamic recommendation decision is made according to the prediction result, and the user interest library is updated through online learning.

[0019] It should be noted that user ID, news ID, interaction behavior and timestamp are extracted from user historical behavior log as original data, data cleaning is performed on the original data, and repeated records and invalid data are removed to ensure the integrity and accuracy of the data; The original data after data cleaning is grouped according to user ID, all interaction behaviors of each user are classified into the same group, interaction behavior data set of each user is generated, the interaction behavior data set is sorted in ascending order according to timestamp, and the behavior sequence of the user is arranged in chronological order to form a time sequence interaction sequence; the interaction behavior of the user is extracted from the time sequence interaction sequence, and the user behavior sequence is constructed , represents the interaction behavior of the user at timestamp , , is the total number of timestamps; the timestamp corresponding to each interaction behavior is extracted to construct a related timestamp sequence , represents the time when the user interaction behavior occurs, ; and all news in the current system are obtained to construct a candidate news set , represents the candidate news, , represents the total number of news. The constructed time sequence interaction sequence, user behavior sequence, candidate news set and related timestamp set are stored in the database for subsequent model calling, and a regular or real-time updating mechanism is set in the database to ensure that the data can reflect the latest behavior dynamics of the user.

[0020] It should be noted that a preset time window is used according to the recommendation requirements, such as the past 24 hours, a week, or a specific time period such as "7-9 am". The preset time window is used to traverse the user behavior sequence and the related timestamp set, extract the interaction records that meet the preset time conditions, and filter the news that the user has interacted with within the preset time window; the filtered news IDs and the associated multimodal data are combined into a historical interaction news set , , where each news Including text content and image content, use the pre-trained BERT model to semantically encode the news text content to obtain the initial text feature vector ; Extract the hourly information of news interaction time and use the sine function to encode the time feature , through linear transformation Mapping temporal features to semantic space, and BERT text features Adding time perception capability, enhancing the initial text feature vector, and obtaining a time-enhanced text representation , expressed as: , in Encode the current moment through the sine function , Projecting temporal features into semantic space. Time-enhanced text feature encoding ensures that text features not only contain semantic information but also incorporate temporal context.

[0021] Use Mask R-CNN to monitor news images and extract salient areas , the salient regions and news text are input into the ViLBERT model, and the visual-semantic aligned feature representation is generated through the cross-modal attention mechanism , expressed as: , in represents the total number of significant regions, Indicates the A significant area, Represents the news text content, Represents the output of the ViLBERT model. The cross-modal attention mechanism in ViLBERT ensures that visual concepts are aligned with textual context.

[0022] Concatenate the temporally enhanced text representation with the visual-semantic aligned feature representation to generate the final multimodal encoding representation , , multimodal coding representation It also contains semantic, visual and temporal information, providing input for subsequent clock interest modeling.

[0023] Figure 2 A flowchart for generating user short-term and long-term interest representations is shown.

[0024] According to an embodiment of the present application, clock interest modeling is performed based on the multi-modal encoding representation of the historical interaction news, and a context vector that can comprehensively reflect the user's short-term and long-term interests is generated through Gaussian weighted aggregation and long-term interest enhancement operation, providing an important interest basis for dynamic recommendation decision, specifically: S202, constructing a user 24-hour dimension interest library , wherein the multi-modal encoding representation of the historical interaction news of the user in the time period is stored, and each time the user generates an interaction behavior in a new time period, the multi-modal encoding representation of the corresponding news is updated to the corresponding time period in the interest library; each time the user generates an interaction behavior in a new time period, the multi-modal encoding of the corresponding news is updated to the corresponding time period in the interest library, for example, the user clicks on the news at 10 a.m., then .

[0025] S204, short-term interest modeling using Gaussian weighted aggregation, using a Gaussian kernel function with the current time as the center, calculating the weight of the adjacent time period , and performing weighted summation on the interest vectors of the 24 time periods to obtain a smoothed short-term interest representation , denoted as: , wherein, represents the weight of the adjacent time period , and controls the smoothing strength, and it is found through testing that the information amount obtained in 1 hour is insufficient, and the noise is too large in 2 hours, so it is set to 1.5 hours, represents a radial basis function, which is used to smooth the interest mutation at the time period boundary, for example, the interest suddenly switches back to work-related news after "lunch break".

[0026] S206, retrieving related news from the user's full-time period historical data to enhance the long-term interest representation, filtering historical news with similar semantics to the current candidate news from the historical interaction news set, calculating the weight of each historical news according to the semantic relevance and time decay , and performing weighted aggregation on the multi-modal encoding representation of the historical news to generate a long-term interest representation : , wherein, representing historical interaction news set.

[0027] To avoid over-reliance on recent interests, relevant news is retrieved from the user's full-time historical data Long-term interest enhancement is performed, and weights are calculated for each historical news according to semantic relevance and time decay , represented as: , wherein represents a time decay function, controls the decay rate, represents the transpose of the multi-modal encoding representation of the current candidate news, , represents a learnable parameter.

[0028] S208, the short-term interest representation aggregated by Gaussian and the long-term interest are spliced into a high-dimensional vector, and the final user interest vector is generated by reducing the dimension through a learnable matrix, represented as: , wherein, represents a learnable fusion matrix, represents a real number space with a dimension of . The final user interest vector contains the user's current period preference and stable interest across periods at the same time, providing a basis for dynamic recommendation decision-making.

[0029] Figure 3 A flowchart for making a dynamic recommendation decision is shown.

[0030] According to an embodiment of the present application, the matching degree between the candidate news and the current user interest vector is calculated in combination with cross-modal attention, and the output of the time sequence gate is subjected to click probability prediction by a linear network, and a dynamic recommendation decision is made according to the prediction result, so as to provide the user with recommended news that is more in line with his current interest and period preference, specifically: S302, for each news in the candidate news set, the same time-aware multi-modal encoding as the historical news is used to generate a corresponding multi-modal encoding representation; S304, the user interest vector is matched with the multi-modal encoding representation of the candidate news , the matching scores of text and visual modalities are integrated, and the initial matching score is obtained by normalizing through a Sigmoid function, represented as: , wherein ​Different modalities Corresponding user interest components, including text components of user interest and visual components of user interest , Different modalities of news Corresponding multi-modal encoding representation, including news text features and news image features. The initial matching score obtained by integrating the text matching degree and the visual matching degree measures the semantic and visual fit degree of the candidate news and the current interest of the user.

[0031] S306, preset taboo time period rules define a time period-class inhibition matrix, and based on the time period-class inhibition matrix, time-sensitive inhibition is applied to the initial matching score through time gating, and reinforcement learning is introduced to dynamically adjust the inhibition strength; Preset taboo time period rules, such as not recommending entertainment news at night, corresponding to taboo time periods Inhibition strength ; If the current time belongs to the taboo time period of a certain category, the inhibition is triggered, and the matching score after applying time-sensitive inhibition is expressed as: , Wherein represents the initial matching score, represents the category inhibition strength in the taboo time period , represents the total number of categories, if it is 1 o'clock in the morning and the news category is entertainment, the matching score is attenuated by 80%, represents the indicator function, represents the current time, when 1, otherwise 0; The inhibition strength is dynamically adjusted through reinforcement learning, such as when the user frequently clicks on the inhibited news, the inhibition strength is reduced.

[0032] S308, input the matching score after applying time-sensitive inhibition into the full connection layer, output the click probability prediction value, use Softmax to normalize the click probability prediction value of all candidate news, and get the final recommendation probability distribution; Adopt linear network to predict click probability , expressed as , represents activation function, represents multi-layer perception, as a nonlinear transformation function, responsible for learning the complex mapping relationship between the matching score and the final click probability.

[0033] S310, selecting a preset number of candidate news according to the final recommended probability distribution for recommendation, and introducing a category diversity constraint to avoid homogenization of recommendation.

[0034] The generated news recommendation list is returned to the user front-end interface, displayed according to priority, and user feedback on the recommendation result is recorded for subsequent model optimization of the online learning module.

[0035] It should be noted that the online learning module is responsible for updating the user's interest library in real time to adapt to the dynamic changes of the user's interest. Real-time capture of user interaction behavior in the front-end interface, such as clicking, browsing duration, favorites, etc., records the corresponding news ID, timestamp and behavior type, and obtains the multi-modal encoding representation of the new interaction news, determines the hour segment according to the behavior timestamp, and performs hour positioning; The said hour segment is updated in real time by exponential moving average, and the updated interest vector is expressed as: , wherein represents the historical memory screening coefficient, which determines the retention degree of historical information, represents the multi-modal encoding representation of the new interaction news, represents the set of new interaction news, represents the historical interest vector of the hour segment; The KL divergence of the new and old interest vectors of the hour segment characterizes the interest drift degree, and is compared with a preset threshold according to the KL divergence, and the model is fine-tuned according to the judgment result, and local adjustment and global adjustment are set in the model fine-tuning. Local adjustment only updates the gradient of the parameters of the related hour segment in the clock interest modeling module, and if drift is detected in continuous multiple hour segments, global adjustment is performed to start full model fine-tuning. In the interest library, the interest library is scanned in full every week, similar hour segment interests are merged, expired interests are deleted, and for new users or low-active users, the interest library is supplemented by collaborative filtering.

[0036] It should be noted that in the model training, end-to-end optimization is realized through multiple rounds of iteration, and time-aware multi-modal encoding (combining Gaussian time domain convolution and time gating mechanism), cross-modal attention fusion and clock interest modeling are performed in each round. In the loss calculation stage, the cross-entropy loss function L is used, and the Adam optimizer is used to update the parameters jointly, and the online adaptive adjustment of the model parameters is realized based on the streaming interest distillation mechanism. The click rate prediction probability is output through linear projection transformation and Sigmoid activation function.

[0037] Figure 5 A block diagram of a multi-modal news recommendation system combined with clock interest is shown.

[0038] The second embodiment of the application provides a multi-modal news recommendation system 5 combined with clock interest, comprising a multi-modal feature encoding module 501, a clock interest modeling module 502, a dynamic recommendation decision module 503 and an online learning module 504. The multi-modal feature encoding module extracts historical interactive news meeting preset time requirements from a user behavior sequence and a related timestamp set, performs time-aware multi-modal feature encoding on the historical interactive news, and obtains multi-modal encoding representation of the historical interactive news. It should be noted that the text encoding adopts a pre-trained BERT model to extract semantic features and injects practical perception ability; the image encoding uses Mask R-CNN to detect salient regions and inputs ViBERT to generate visual-linguistic alignment features, and the cross-modal attention mechanism in ViBERT ensures that visual concepts are aligned with text context. The multi-modal encoding module performs time context enhancement on BERT text encoding and ViLBERT visual encoding to provide more rich time dimension information for subsequent interest analysis.

[0039] The clock interest modeling module models clock interest based on the multi-modal encoding representation of the historical interactive news, generates vector representations of short-term and long-term interests of the user through Gaussian weighted aggregation and long-term interest enhancement operation, and fuses long-term and short-term interests to obtain an interest vector of the user. It should be noted that, in order to capture the diurnal interest pattern of the user, an hourly interest library is maintained for each user, and a Gaussian weighted aggregation of adjacent period interests is performed for interest activation at the current time. In addition, in order to avoid excessive dependence on recent interests, relevant news is retrieved from the full-period historical data of the user for long-term interest enhancement. The clock interest modeling module effectively integrates the interest changes of the user at different periods by performing Gaussian time domain convolution to aggregate the interest patterns of adjacent periods based on the multi-modal feature representation of the historical news.

[0040] The dynamic recommendation decision module calculates the matching degree of the candidate news and the current user interest vector through the cross-modal attention mechanism, applies period-sensitive inhibition through the time sequence gate, performs click probability prediction on the output of the time sequence gate through a linear network, and makes a dynamic recommendation decision according to the prediction result. It should be noted that the candidate news is calculated by cross-modal attention to calculate the click probability, and the matching degree of the candidate news to the current interest of the user is calculated by modal perception matching. Further, to avoid the improper behavior of recommending entertainment news at midnight, a time period sensitive inhibition processing is performed through a time sequence gate, so as to provide the user with recommended news that meets the current interest and time period preference. The dynamic recommendation decision module combines the cross-modal attention mechanism and the time sequence gate mechanism, dynamically adjusts the weight of the image-text features according to the characteristics of different time periods, fuses the time period significance of the visual elements, and realizes the accurate matching of the image-text content and the time context.

[0041] The online learning module adopts a streaming interest distillation mechanism to realize real-time updating while avoiding sudden noise interference, and ensures that the recommendation system can timely reflect the dynamic changes of the user interest.

[0042] It should be noted that the interest library is updated in real time by the exponential moving average, the new interest library state is determined by the old interest library state and the new news interaction features, and then the KL divergence detection graph is used to fine-tune the model. The online learning module adopts a streaming interest distillation mechanism to realize real-time updating while avoiding sudden noise interference, and ensures that the recommendation system can timely reflect the dynamic changes of the user interest.

[0043] The third aspect of the present application provides a computer readable storage medium, the computer readable storage medium comprises a kind of multi-modal news recommendation method program combined with clock interest, when the kind of multi-modal news recommendation method program combined with clock interest is executed by processor, the steps of the kind of multi-modal news recommendation method combined with clock interest are realized.

[0044] In several embodiments provided in the present application, it should be understood that the disclosed method and system can be implemented in other ways. The system embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and actual implementation can have another division mode, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling between the displayed or discussed components, or direct coupling or communication connection can be indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms. In addition, the function modules in each embodiment of the present application can be integrated into a processing module, or each module can be a separate module, or two or more modules can be integrated into one module; the integrated module can be realized in the form of hardware or hardware plus software function module.

[0045] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware, the foregoing program can be stored in a computer readable storage medium, and the program executes the steps including the above-mentioned method embodiments when executed; and the foregoing storage medium includes a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disc or an optical disc and various storage medium capable of storing program codes.

[0046] The above merely provides a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A multimodal news recommendation method combining clock interests, characterized in that: The following steps are involved: Constructing a time-series interaction sequence from the user's historical behavior log, and extracting a user behavior sequence, a candidate news set, and a related timestamp set based on the time-series interaction sequence; Extracting historical interactive news that meets preset time requirements from the user behavior sequence and the related timestamp set, performing time-aware multimodal feature encoding on the historical interactive news, and obtaining a multimodal encoded representation of the historical interactive news; Based on the multimodal encoding representation of the historical interactive news, clock interest modeling is performed, and through Gaussian weighted aggregation and long-term interest enhancement operations, vector representations of the user's short-term and long-term interests are generated, and the user interest vector is obtained by fusing the long-term and short-term interests; The matching degree between candidate news and the current user interest vector is calculated, and time-period-sensitive inhibition is applied through temporal gating. The output of temporal gating is used to predict click probability using a linear network. Dynamic recommendation decisions are made based on the prediction results, and the user interest library is updated through online learning.

2. The multimodal news recommendation method combining clock interests according to claim 1 is characterized in that: Construct a time-series interaction sequence from the user's historical behavior log, and extract the user behavior sequence, candidate news set, and related timestamp set based on the time-series interaction sequence, specifically: Extracting user ID, news ID, interaction behavior, and timestamp from user historical behavior logs as raw data, and performing data cleaning on the raw data to remove duplicate records and invalid data; The cleaned raw data is grouped by user ID to generate an interaction behavior dataset for each user, and the interaction behavior dataset is sorted in ascending order by timestamp to form a time-series interaction sequence; The user's interactive behavior and the timestamp corresponding to each interactive behavior are extracted from the temporal interaction sequence, a user behavior sequence and a related timestamp sequence are constructed, and all news in the current system are obtained to construct a candidate news set.

3. The multimodal news recommendation method combining clock interests according to claim 1, characterized in that: Extracting historical interactive news that meets preset time requirements from the user behavior sequence and the related timestamp set, performing time-aware multimodal feature encoding on the historical interactive news, and obtaining a multimodal encoding representation of the historical interactive news, specifically: Preset a time window based on recommendation requirements, use the preset time window to traverse the user behavior sequence and related timestamp set, extract interaction records that meet the preset time conditions, and filter news that the user interacted with within the preset time window; The selected news IDs and associated multimodal data are combined into a historical interactive news collection, where each news item includes text and image content. The pre-trained BERT model is used to semantically encode the news text content to obtain the initial text feature vector. Extract the hourly period information of news interaction time, encode the time features using a sine function, map the time features to a semantic space through a linear transformation, add them to the BERT text features, and enhance the initial text feature vector to obtain a time-enhanced text representation; Use Mask R-CNN to detect objects in news images and extract salient regions. These salient regions and news text are then fed into the ViLBERT model, where a cross-modal attention mechanism is used to generate visually-semantically aligned feature representations. The temporally enhanced text representation is concatenated with the visual-semantic aligned feature representation to generate the final multimodal encoding representation.

4. The multimodal news recommendation method combining clock interests according to claim 1, characterized in that: Based on the multimodal encoding representation of the historical interactive news, clock interest modeling is performed. Through Gaussian weighted aggregation and long-term interest enhancement operations, context vectors of users' short-term and long-term interests are generated. Specifically, Build a 24-hour user interest database ,in Store user Multimodal encoding representation of historical interactive news in a time period. Whenever a user interacts in a new time period, the multimodal encoding representation of the corresponding news is updated to the corresponding time period in the interest library. Use Gaussian weighted aggregation for short-term interest modeling, and use the Gaussian kernel function to calculate the current moment As the center, calculate the adjacent time periods The weight of interest vector for 24 time periods Perform weighted summation to obtain a smoothed short-term interest representation , expressed as: ， in, Indicates adjacent time periods The weight of Indicates the smoothing strength; Retrieve relevant news from the user's full-time historical data, enhance long-term interest representation, filter historical news with similar semantics to the current candidate news in the historical interactive news collection, and calculate the weight for each historical news based on semantic relevance and time decay. ; Multimodal encoding representation of weighted aggregated historical news , generating long-term interest representations : ， in, Represents a collection of historical interactive news.

5. The multimodal news recommendation method combining clock interests according to claim 1, characterized in that: The user interest vector is obtained by integrating long-term and short-term interests, specifically: Short-term interest representation of Gaussian aggregation and long-term interests Splice into high-dimensional vectors, reduce the dimension through the learnable matrix, and generate the final user interest vector , expressed as: ， in, represents the learnable fusion matrix, Indicates size The real number space.

6. The multimodal news recommendation method combining clock interests according to claim 1, characterized in that: Calculate the matching degree between candidate news and the current user's interest vector, and apply time-period-sensitive suppression through temporal gating. Use a linear network to predict the click probability of the output of temporal gating, and make dynamic recommendation decisions based on the prediction results. Specifically: For each news item in the candidate news set, the same time-aware multimodal encoding as that used for historical news is used to generate the corresponding multimodal encoding representation. Perform modality-aware matching between the user interest vector and the multimodal encoding representation of the candidate news, integrate the matching scores of the text and visual modalities, and normalize them using the Sigmoid function to obtain the initial matching score. Define a period-category inhibition matrix, apply period-sensitive inhibition to the initial matching score through temporal gating based on the period-category inhibition matrix, and introduce reinforcement learning to dynamically adjust the inhibition strength. The matching score after applying period-sensitive inhibition is Expressed as: ， in represents the initial matching score, Representation category During the taboo period The inhibitory strength, represents the total number of categories, represents the indicator function, Indicates the current moment; The matching scores after applying time-period sensitive suppression are input into the fully connected layer, and the click probability prediction value is output. The click probability prediction values ​​of all candidate news are normalized using Softmax to obtain the final recommendation probability distribution. A preset number of candidate news items are selected for recommendation based on the final recommendation probability distribution, and category diversification constraints are introduced to avoid recommendation homogeneity.

7. The multimodal news recommendation method combining clock interests according to claim 1, characterized in that: Update the user interest database through online learning, specifically: Capture user interaction behaviors in real time, record the corresponding news ID, timestamp, and behavior type, and obtain a multimodal encoding representation of the newly interacted news. Determine the hourly period to which the behavior belongs based on the timestamp and perform time period positioning. The time period is updated in real time by exponential moving average, and the updated interest vector Expressed as: ， in represents the historical memory screening coefficient, A multimodal encoding representation of new interactive news, Represents a new interactive news collection, Represents the historical interest vector for this period; The KL divergence of the new and old interest vectors in the calculation period represents the degree of interest drift, and a comparison is made between the KL divergence and a preset threshold, and the model is fine-tuned according to the judgment result.

8. A multimodal news recommendation system combining clock interests, characterized by: For implementing the multimodal news recommendation method combined with clock interest as described in any one of claims 1 to 7, the system includes a multimodal feature encoding module, a clock interest modeling module, a dynamic recommendation decision module and an online learning module; The multimodal feature encoding module extracts historical interactive news that meets preset time requirements from the user behavior sequence and the related timestamp set, performs time-aware multimodal feature encoding on the historical interactive news, and obtains a multimodal encoded representation of the historical interactive news; The clock interest modeling module performs clock interest modeling based on the multimodal encoding representation of historical interactive news. Through Gaussian weighted aggregation and long-term interest enhancement operations, it generates vector representations of users' short-term and long-term interests, and integrates long-term and short-term interests to obtain the user interest vector. The dynamic recommendation decision module calculates the matching degree between candidate news and the current user's interest vector through a cross-modal attention mechanism, applies time-period-sensitive suppression through temporal gating, uses a linear network to predict the click probability of the temporal gating output, and makes dynamic recommendation decisions based on the prediction results; The online learning module adopts a streaming interest distillation mechanism to achieve real-time updates while avoiding sudden noise interference, ensuring that the recommendation system can promptly reflect the dynamic changes of user interests.

Images