A user interest prediction method and device based on multi-modal data

Abstract

The present application relates to the field of advertisement recommendation, and particularly relates to a user interest prediction method and device based on multi-modal data, comprising: obtaining multi-modal historical data of a target user; identifying influence variables in the multi-modal historical data; segmenting the influence variables from the multi-modal historical data according to the identified results; constructing a demand analysis model based on the segmented results; obtaining behavior data of the target user in real time, inputting the behavior data into an interest prediction model, outputting a demand data set corresponding to the current behavior data of the target user through the demand analysis model; based on the output results, analyzing the demand data set based on a time stamp, and predicting the current interest label of the target user; and generating advertisement data according to the interest label and sending the advertisement data to the target user. Through multi-angle data integration, the interest prediction is more accurate. The key variables influencing the user interest are identified, and irrelevant information is removed through feature selection, which can significantly improve the precision of the interest prediction model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of advertisement recommendation, in particular to a user interest prediction method and device based on multi-modal data. BACKGROUND

[0002] User interest prediction based on multi-modal data refers to integrating data from different sources and different forms (such as text, image, voice, video, user behavior data, etc.) to more accurately predict and understand user interests, preferences and needs. This process combines multiple data modalities (i.e. multiple information types) for comprehensive analysis to improve the effectiveness of recommendation systems, advertisement delivery, personalized services, etc.

[0003] In the prior art, goods are usually pushed according to the user's short-term purchase behavior. For example, when a user purchases a certain good, the system may push similar goods based on this behavior. However, if this is a one-time purchase by the user due to a discount or promotion, the system is likely to misjudge this behavior as the user's long-term interest. If the user purchases a certain good due to seasonal discounts or promotion (such as air conditioners in summer or down jackets in winter), the system may mistakenly believe that the user will prefer these goods in future needs and continue to push similar goods in the future. This will lead to the user no longer being interested in the goods in actual needs, but the system repeatedly pushes similar goods, resulting in redundant advertisements and reducing user experience. Secondly, the prior art usually focuses on content and personalized recommendation, but pays insufficient attention to the adaptability of the transmission method of advertisements. Different advertisement contents are suitable for different transmission channels and display methods (such as push notifications, emails, pop-ups, social platforms, etc.), and if a uniform transmission method is used, it is easy to cause deterioration of user experience and even miss advertisements that meet user needs.

[0004] Therefore, the prior art has defects and needs to be improved. SUMMARY

[0005] In order to solve one or several problems in the prior art, the main purpose of the present application is to provide a user interest prediction method and device based on multi-modal data.

[0006] In order to achieve the above-mentioned purpose of the application, the present application provides a user interest prediction method based on multi-modal data, which comprises:

[0007] Obtaining multi-modal historical data of a target user;

[0008] Identifying influence variables in the multi-modal historical data;

[0009] According to the identified result, the influence variables are segmented from the multi-modal historical data;

[0010] Based on the segmentation result, a demand analysis model is constructed;

[0011] Real-time behavior data of the target user is acquired, the behavior data is input into the interest prediction model, and a demand data set corresponding to the current behavior data of the target user is output by the demand analysis model;

[0012] Based on the output result, the demand data set is analyzed based on a timestamp, and a current interest label of the target user is predicted;

[0013] An advertisement data is generated according to the interest label and is sent to the target user.

[0014] Embodiments of the present application also provide a user interest prediction device based on multi-modal data, comprising:

[0015] An acquisition module is configured to acquire multi-modal historical data of a target user;

[0016] An identification module is configured to identify an influence variable in the multi-modal historical data;

[0017] A segmentation module is configured to segment the influence variable from the multi-modal historical data according to the identification result;

[0018] A construction module is configured to construct a demand analysis model based on the segmentation result;

[0019] An input module is configured to acquire real-time behavior data of the target user, input the behavior data into the interest prediction model, and output a demand data set corresponding to the current behavior data of the target user by the demand analysis model;

[0020] A prediction module is configured to analyze the demand data set based on a timestamp based on the output result, and predict a current interest label of the target user;

[0021] A sending module is configured to generate an advertisement data according to the interest label and send the advertisement data to the target user.

[0022] The present application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of the preceding embodiments when executing the computer program.

[0023] The present application also provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the method according to any one of the preceding embodiments.

[0024] The user interest prediction method and device based on multi-modal data provided in the embodiments of the present application can comprehensively capture the interests and demands of users from multiple dimensions (such as text, image, behavior, etc.) by obtaining multi-modal historical data. Such multi-angle data integration makes the interest prediction more accurate. Identifying the key variables that affect the user's interest and removing irrelevant information through feature selection can significantly improve the accuracy of the interest prediction model. Through fine-grained analysis of influencing factors, the model is closer to the actual needs of users. By obtaining user behavior data in real time, the system can dynamically adjust the user interest prediction to ensure that the recommended content meets the real-time needs of users, thereby improving the user experience. The demand analysis model can provide more accurate personalized recommendations according to the current interest needs of users through in-depth mining of user behavior and interest, optimize advertising and content pushing, and avoid information overload and irrelevant recommendations. Combined with timestamp information for interest change trend analysis, the prediction can not only accurately reflect the current interest, but also understand the changing needs of users over time, thereby realizing more timely recommendations. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 FIG. 1 is a flowchart of a user interest prediction method based on multi-modal data according to an embodiment of the present application;

[0026] Figure 2 FIG. 1 is a flowchart of a user interest prediction method based on multi-modal data according to an embodiment of the present application;

[0027] Figure 3 FIG. 2 is a structural schematic block diagram of a user interest prediction device based on multi-modal data according to an embodiment of the present application;

[0028] Figure 4 FIG. 3 is a structural schematic block diagram of a computer device according to an embodiment of the present application.

[0029] The implementation, functional features and advantages of the present application will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below in conjunction with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0031] Referring to Figure 1 , a user interest prediction method based on multi-modal data is provided in the embodiments of the present application, and the method comprises:

[0032] S1, obtaining multi-modal historical data of a target user;

[0033] S2, identifying the influencing variables in the multi-modal historical data;

[0034] S3、According to the result of identification, the influence variable is segmented from the multi-modal historical data;

[0035] S4, based on the result of segmentation, a demand analysis model is constructed;

[0036] S5, real-time acquisition of the behavior data of the target user, inputting the behavior data into the interest prediction model, and outputting the demand data set corresponding to the current behavior data of the target user through the demand analysis model;

[0037] S6, based on the output result, analyzing the demand data set based on the time stamp, and predicting the current interest label of the target user;

[0038] S7, generating advertisement data according to the interest label and sending it to the target user.

[0039] As described in steps S1-S3 above, a comprehensive user historical data set is constructed by collecting various behavior data of the user. Multi-modal data includes text (such as search records, comments, social media content), images (such as user uploaded photos or clicked product pictures), audio, video and behavior data (such as clicks, purchases, browsing, etc.). Through these data, multi-angle information of user behavior can be obtained. By integrating data of different modalities, the user's interest, behavior and demand can be comprehensively understood. Combining information from different sources can improve the prediction accuracy of user interest and avoid the bias that may exist in single modal data. Influence variables refer to those data elements that play a key role in predicting user interest, behavior or demand. In this step, by analyzing and processing multi-modal data, features that have a greater impact on interest prediction are identified. For example, user click frequency, comment sentiment, browsing time, visual appeal of product pictures, etc. can all be influence variables. By identifying important variables, irrelevant data interference can be avoided, the input of analysis can be simplified, and computing efficiency can be improved. Identifying key factors that affect interest helps to establish a more targeted interest prediction model, thereby improving the accuracy of prediction. Segmentation of influence variables refers to extracting key influencing factors from multi-modal data and performing feature selection or conversion on these data. This may include feature engineering on user behavior data, selecting the most relevant features for interest prediction, and segmenting important variables. After segmenting the influence variables, irrelevant data interference can be reduced, focusing on information that helps prediction, thereby improving the accuracy and efficiency of the model.

[0040] As described in steps S4-S7 above, the demand analysis model is an analysis framework constructed based on the segmented influence variables, used to capture the potential demand of users in the current context. Generally, the demand analysis model can utilize machine learning or deep learning methods, such as regression analysis, classification model or neural network, to learn the pattern of user demand from historical data. The model can effectively capture the potential demand of users, helping to predict the content or goods that users may be interested in at a certain time point. Based on the demand analysis model, the system can better provide personalized recommendations or services according to the user's interest points and demand, improving the user experience. Real-time acquisition of behavior data refers to continuously collecting user behavior information such as clicks, browsing, searching, purchasing, etc. while the user interacts with the platform. These data are timely input into the interest prediction model so that the model can update the prediction of user interest in real time according to the current behavior. Real-time acquisition of user behavior data can ensure that the update of the interest prediction model is timely, reflecting the current interest changes of users. Through real-time data, the model can dynamically adjust the interest label to timely adapt to the interest changes of users and provide instant personalized recommendations. This process maps the real-time acquired user behavior data to a demand data set through the demand analysis model, and the demand data set represents the current interest demand of the user. The demand data set may contain a set of labels or features reflecting the user's current preferences for content, products or services. The demand data set output by the demand analysis model can accurately map the user's current interest and demand, avoiding inaccurate recommendations due to unclear user demand. According to the accurate demand provided by the demand data set, the conversion rate of advertisements or recommendations can be improved, and the effect of advertisement placement can be optimized. According to the analysis of demand data set and timestamp information, the trend of user interest change can be understood, and the current interest label can be predicted. The timestamp plays a key role here, which helps to identify the regularity of user interest changes over time, for example, users may be interested in news in the morning and entertainment content in the evening. Time series analysis of demand data using timestamps helps to capture the temporal dynamics of user interest, making the prediction more accurate and personalized. Through the time analysis of the demand data set, the changes in user interest can be captured in real time, maintaining the timeliness and relevance of the recommendations. According to the predicted user interest label, the system can generate relevant advertisement data. The advertisement data is based on the user's interest label and demand data, and the advertisement content matching the user's interest is pushed to improve the click-through rate and conversion rate of the advertisement. The advertisement content generated according to the interest label can ensure that the advertisement is highly matched with the user's interest, improving the user's click and interaction willingness. By pushing more relevant advertisements, the user's acceptance of the advertisement is improved, thereby improving the user's satisfaction and the platform's advertising revenue.

[0041] As mentioned above, by acquiring multi-modal historical data, the user's interests and needs can be captured comprehensively from multiple dimensions such as text, image, behavior, etc. This multi-angle data integration makes the interest prediction more accurate. Identifying key variables that affect user interest and removing irrelevant information through feature selection can significantly improve the accuracy of the interest prediction model. By analyzing the influencing factors in detail, the model is more close to the actual needs of the user. By acquiring user behavior data in real time, the system can dynamically adjust the user interest prediction to ensure that the recommended content meets the real-time needs of the user, thereby improving the user experience. The demand analysis model can provide more accurate personalized recommendations according to the user's current interest needs by deeply mining user behavior and interest, optimize advertising and content pushing, and avoid information overload and irrelevant recommendations. Combined with timestamp information for interest change trend analysis, the prediction can not only accurately reflect the current interest, but also understand the user's changing needs over time, so as to realize more time-effective recommendations.

[0042] Referring to Figure 2 In one embodiment, the step of analyzing the demand data set based on the timestamp and predicting the target user's current interest label comprises:

[0043] S61, according to the behavior data, analyzing the scene features of the behavior data;

[0044] S62, based on the analysis result, predicting the interest change feature of the target user under the scene feature;

[0045] S63, timestamp alignment processing of the interest change feature, scene feature and behavior data, and fusion of the processed interest change feature, scene feature and behavior data feature to obtain joint features;

[0046] S64, input the joint features into a preset classification model, and predict the interest label of the target user according to the joint features through the classification model.

[0047] As mentioned in the previous steps, behavior data is multi-dimensional data generated during user interaction with the platform (e.g., clicks, views, purchases, etc.). These behaviors are often related to specific scenarios (e.g., user behavior differs in different scenarios such as shopping, browsing news, watching videos, etc.). By analyzing the behavior data for scenario characteristics, the goal is to extract environmental and contextual factors related to specific behaviors. For example, a user's behavior patterns at a specific time, location, and device may reflect their current interests or needs. Scenario characteristics include time (e.g., morning, noon, evening), geographic location (e.g., workplace, home), device (e.g., mobile phone, computer), etc., which have an impact on user behavior. Analyzing these scenario characteristics helps understand the context of behavior and thus infer changes in user interest in different scenarios. By analyzing scenario characteristics, the system can more accurately capture the underlying needs behind user behavior. For example, when a user is in a shopping scenario, they may exhibit interest in goods, while in a social scenario, they may show interest in entertainment content. By analyzing the relationship between scenario characteristics and user behavior, we can infer trends in user interest changes in that scenario. This change is usually time-dependent, meaning that user interest is not constant but changes over time and with scenario switching. This prediction dynamically captures fluctuations and changes in user interest, not only based on current behavior but also taking into account contextual factors such as time and location, to generate more accurate interest predictions. Using these trends, the system can adjust the recommendation strategy in real time, improving the real-time and accuracy of personalized recommendations. When processing user behavior data, timestamp alignment is an important step because user behavior has a time dependency. Timestamp alignment processing refers to synchronizing all relevant data (interest change characteristics, scenario characteristics, behavior data) based on timestamps to ensure consistency of time information for each feature. This step ensures that the model can accurately understand the relevance of each feature at the same time point when making interest predictions. Behavior data, interest change characteristics, and scenario characteristics may have different time frequencies or sampling intervals, and timestamp alignment can align these features within the same time frame. Timestamp alignment processing ensures that the model can reflect both short-term and long-term interest changes. Through timestamp alignment processing, the system can capture both rapid changes in user interest in the short term and trends over a long time scale. This step helps the model understand the user's interest state at a certain time point, enhancing the accuracy of the prediction results.The behavior data features show the way the user interacts with the platform, which is the direct basis for interest prediction. Classification models (such as logistic regression, support vector machines, deep learning models, etc.) can learn from existing user behavior data and labels to infer the current user's interest labels. The core task of the classification model is to classify according to the feature data and predict the most likely interest label of the user. The pre-set classification model is usually trained based on historical data, and by comparing the joint features of the current user, the model can output the predicted interest label. Through the output of the classification model, accurate personalized recommendations can be provided to the user, ensuring that the recommended content is highly relevant to the user's current interests.

[0048] In an embodiment, the step of sending the generated advertisement data to the target user according to the interest label comprises:

[0049] Obtaining the multi-modal historical data;

[0050] According to the multi-modal historical data, the conversion rate of all transmission paths is determined, and the weight coefficient of each transmission path is calculated according to the conversion rate;

[0051] Obtaining the scene features and advertisement data of the current behavior data;

[0052] According to the scene features and advertisement data, analyze all the conflict paths in the transmission path;

[0053] Based on the analysis result, determine the available transmission path, and calculate the transmission cost of the available transmission path;

[0054] According to the calculation result, select the available transmission path with the lowest transmission cost and the highest weight coefficient as the transmission path of this time;

[0055] Based on the determined transmission path, the advertisement data is sent to the target user.

[0056] As mentioned above, multi-modal historical data includes users' past click behavior, purchase records, browsing history, social media activity, etc. By comprehensively analyzing these historical behaviors, the system can more accurately understand the user's interest changes and their behavior patterns in different environments. These historical data will help the system understand the user's preferences and behavior logic in different scenarios. Multi-modal data can reflect multiple dimensions of user behavior, helping to build a more accurate user interest model. Through the integration of multi-source information, the current needs of users can be better predicted, thereby optimizing the advertising push strategy. The transmission path refers to the propagation path of advertising data from the platform to the user, which can be multiple channels such as recommendation engines, social media, email push, etc. The conversion rate of each channel represents the effect (such as click rate, purchase rate, etc.) that the path can bring in the past delivery. The weight coefficient reflects the relative importance of each path. The conversion rate is based on statistical analysis of historical data, measuring the success rate of different transmission paths (such as different advertising recommendation methods). Paths with high conversion rates may be given higher weights. The weight coefficient is adjusted according to the conversion rate, helping to select the most appropriate advertising propagation method among multiple transmission paths. By calculating the conversion rate and weight coefficient, the most effective transmission path can be selected to improve the conversion effect of advertising. Selecting a path with a higher weight coefficient for advertising transmission can ensure that limited advertising resources are used in channels that are most likely to bring high returns. Current behavior data reflects some of the user's recent behaviors (such as browsing pages, clicking on ads, interacting with the platform, etc.). Scene features describe the specific background of user behavior, such as time, location, device, emotional state, etc. By obtaining current behavior data and related scene features, the advertising system can better understand the user's needs at a certain moment. Scene features help understand the background of user behavior. For example, when a user watches videos on their phone at night, the advertising content may tend to be entertainment and consumer goods; while at work during the day, advertising may be more inclined to office software or news information. Advertising data includes the content of the advertisement itself, the target user group, the advertising budget, etc. Combined with the user's current scene features, it can provide a basis for advertising content selection and customization. By combining the user's current behavior and scene features, advertising delivery can more accurately match the user's immediate needs. Make the advertisement more consistent with the user's interests and scenarios, thereby reducing user interference and improving the acceptance of advertising. In the process of advertising delivery, conflicts or repetitions may occur between multiple transmission paths. For example, if the user has already received similar advertising on other channels, the new advertising path may conflict with the previous advertising path, resulting in repeated delivery of advertising or conflicting content. Conflict path analysis is to avoid repeated delivery of advertising content or to avoid different paths pushing similar or irrelevant advertising content, thereby causing user dissatisfaction or a decline in advertising effectiveness. By identifying and excluding conflicting paths, the system can ensure the diversity and effectiveness of advertising delivery.The repetition of the same advertisement content is reduced, avoiding user annoyance. After excluding conflicting paths, the advertisement can reach the user more effectively, avoiding resource waste. After excluding conflicting paths, the system needs to evaluate the cost of the remaining available transmission paths. Each transmission path can have different costs, for example, some advertising channels may require payment, while others may be free. By evaluating the cost of these paths, the system can choose the path with the highest cost-effectiveness. Transmission costs include the cost of advertising placement, the use of platform resources, possible time delays, and other factors. The system needs to weigh the most appropriate advertising placement approach based on these costs. By calculating the transmission cost, the system can choose an advertising path with lower cost and higher efficiency, avoiding ineffective spending. Ensure that advertising placement can achieve the best effect at the lowest cost, thereby improving the overall return on investment of advertising. After calculating the cost and weight coefficient of each path, the system needs to select the transmission path with the lowest cost and the highest weight coefficient as the final advertising transmission method. This selection is based on the consideration of comprehensive benefits, that is, while ensuring efficient transmission, the cost is minimized as much as possible. Selecting a path with a higher weight coefficient means that the system selects a path that is most likely to bring higher conversion rates for placement, while controlling costs to ensure the economy of placement. This decision ensures the optimal allocation of advertising resources and reduces unnecessary waste. By selecting the most appropriate transmission path, the effect of the advertisement can be maximized while reducing costs. It can ensure that the advertisement is transmitted in the most appropriate channel and at the most economical cost, improving the return on investment of advertising. Finally, based on the above steps, after determining the optimal transmission path, the advertising data is transmitted to the target user. This process involves sending advertising content to users through the selected transmission channel. The selection of the transmission path has been based on the principle of maximizing conversion rates and minimizing costs, and the push of the advertisement will reach the user through the most effective channel. The advertisement can quickly and effectively reach the target user through the most appropriate way, thereby improving the exposure rate, click-through rate, and conversion rate of the advertisement.

[0057] In an embodiment, after the step of generating advertising data according to the interest label and sending it to the target user, the method comprises:

[0058] Receiving behavior feedback of the target user in real time;

[0059] According to the behavior feedback of the target user, it is judged whether the advertising data meets the interest demand of the target user;

[0060] Quantifying the behavior feedback to obtain the interest coefficient of the target user;

[0061] If the interest coefficient is lower than the threshold, it is judged that the advertising data does not meet the interest demand of the target user;

[0062] Based on the fact that the advertising data does not meet the interest demand of the target user, analyzing the advertising data causal characteristics, identifying the driving factors formed by the target user;

[0063] According to the driving factors, determine the reasons for not meeting the interest demand of the target user, and generate the advertising data adjustment strategy according to the reasons for not meeting the interest demand of the target user;

[0064] Based on the adjustment strategy, adjust the advertising data.

[0065] As mentioned above, after the ad is launched, the behavior feedback of the target users is monitored and received in real time, such as click rate, browsing time, interactive behavior, etc. These feedback data reflect the user's reaction and interest level to the ad content. Real-time feedback can help the ad system quickly understand the popularity of the ad, identify the user's interest points and the adaptability of the ad in time, and ensure that the ad can be adjusted in time instead of waiting for a long time for result evaluation. The ad system analyzes whether the ad content matches the user's interest label based on the collected user behavior data (such as clicks, browsing, dwell time, etc.). Specifically, if the user does not show positive behavior to the ad (for example, low click rate), it can be inferred that the ad does not meet the user's interest needs. This judgment mechanism helps to evaluate the effectiveness of the ad in real time and optimize the ad placement strategy, avoiding the placement of uninteresting or ineffective ad content, thereby improving the matching degree of the ad and user experience. The process of quantifying behavior feedback is usually to convert the user's interactive behavior (such as clicks, dwell time, interaction times, etc.) into a quantitative interest coefficient. This coefficient can be a floating value to represent the user's interest intensity in a specific ad or ad type. By quantifying behavior feedback, the ad system can accurately capture the dynamic changes in user interest, and adjust the ad content according to different interest levels to improve the personalization and accuracy of the ad. This process is a decision-making process that uses a set of interest coefficient thresholds to determine whether the ad meets user needs. When the interest coefficient is lower than the preset threshold, it means that the ad content no longer attracts users, and the system will mark the ad as "not matching". By setting the threshold, the ad system can filter out inefficient ads in time, avoid wasting ad resources, and ensure the relevance of ad content. The choice of threshold is crucial to the system's effectiveness, and it needs to be adjusted based on the actual situation of the user group, ad type, and placement platform. When the ad does not meet the user's interest needs, the system needs to further analyze the causal characteristics of the ad. For example, the ad content itself, the display method, the time period, etc. may affect the user's reaction. Causal analysis helps the system find out which specific characteristics have failed to attract users, and then identifies the driving factors that affect the effectiveness of the ad. This causal analysis helps the ad system identify which specific factors cause the ad to not meet user interest, thereby providing a basis for adjusting the ad strategy. It improves the intelligence level of the ad system and can further refine the ad content through data analysis. According to the driving factors analyzed, the ad system identifies the problems in the ad content or form and adjusts the strategy accordingly. For example, if the ad's creativity is insufficient or the placement time is not suitable, the ad system can automatically adjust the display method or content of the ad to improve its appeal. By analyzing the reasons why the ad does not meet the user's interest needs, the system can automatically optimize the ad content and improve the matching degree of the ad and user interest. This adjustment can improve the effectiveness of the ad and the ROI (return on investment) of the ad placement.After identifying the specific reasons why the advertising content does not meet the user's needs, the advertising system will apply adjustment strategies to modify the advertising content or reselect the delivery time. This may include updating the content idea, adjusting the user segmentation, changing the advertising frequency, etc. By continuously adjusting the advertising strategy, the advertising delivery is more refined, thereby improving the performance of the advertisement. The personalization and accuracy of the advertisement are greatly improved, which can improve the user's interaction rate and ultimately increase the advertising revenue or achieve the target.

[0066] In an embodiment, the analyzing the advertising data causal features, identifying the driving factors formed by the target users, the method comprises:

[0067] Extracting the behavior data, influence variables and advertising data of the multi-modal historical data;

[0068] Based on the extracted results, a hierarchical causal diagram is constructed to determine all causal paths of the multi-modal historical data;

[0069] Obtain the revenue conversion rate of all causal paths, and calculate the weight coefficient of each causal path according to the revenue conversion rate;

[0070] Add a virtual intervention test to each causal path, and calculate the revenue conversion rate after the intervention;

[0071] According to the revenue conversion rate before the intervention and the revenue conversion rate after the intervention, identify the explicit driving factors of the causal path;

[0072] Add a virtual intervention test to each causal path, map the weight coefficient to a low-dimensional vector, and according to the mapping result, identify the implicit driving factors of the causal path;

[0073] Based on the explicit driving factors and implicit driving factors, determine the driving factors.

[0074] As mentioned above, data is extracted from various sources, including users' historical behavior (e.g., clicks, browsing, purchases), external influencing variables (e.g., season, holidays, promotional activities, etc.), and the advertising data itself (such as ad types, display periods, creative content, etc.). Multimodal data can provide comprehensive information for user profiling from multiple dimensions. By extracting multi-dimensional data, the influence of users' diverse behaviors and external environmental factors can be more accurately captured. In particular, a comprehensive understanding of advertising push behavior helps to avoid misleading the advertising system's judgment of user interest due to a single data source, and reduces the phenomenon of mispushing caused by short-term factors such as promotional activities. The hierarchical causal graph (Hierarchical Causal Graph) is used to construct the causal relationship between multimodal data. By establishing the causal path between variables, it is analyzed which variables will affect the advertising effect and how they will affect each other. For example, user purchase behavior may be influenced by both seasonal discounts and advertising display time, and the hierarchical causal graph can help determine the causal relationship between these factors. The hierarchical causal graph provides a structured way to help the advertising system understand complex causal relationships and avoid overly simplified models. For example, the system can distinguish between user purchase behavior due to short-term promotional activities and long-term interest, thus avoiding misjudgment of users' long-term needs. Return on investment (ROI) refers to the measure of advertising effectiveness, i.e., the conversion rate between advertising investment and user behavior (such as purchase, click, etc.). By calculating the ROI of each causal path, it can be determined which causal paths contribute more to the advertising effect. The weight coefficient is based on the conversion rate to weight the causal path, reflecting its degree of influence on the final advertising effect. According to the ROI and weight coefficient of each causal path, the system can clearly identify which causal factors play a dominant role in user behavior. This helps to more accurately identify which factors are long-term drivers of user purchase behavior and which factors are short-term and occasional, thus avoiding the error of over-reliance on short-term factors for advertising push. Virtual intervention test (Virtual Intervention Test) refers to observing the change in causal paths after intervention (such as assuming a change in a variable) by simulating intervention. By calculating the ROI before and after intervention, the system can identify which causal paths are key drivers. Through this intervention, the advertising system can simulate different scenarios to further understand which causal paths have a greater impact on advertising effectiveness in actual applications, and thus optimize advertising strategies. If a path has a significant change after intervention, it may be a potential key driver. Explicit driving factors are factors that directly and obviously affect user behavior, such as the creative content of the ad itself, the timing of ad display, etc. By comparing the changes in ROI before and after intervention, explicit driving factors that directly affect the advertising effect in the causal path can be identified.By identifying the explicit drivers, the advertising system can pinpoint which factors are the key drivers of user behavior changes, and thus improve the ad display strategy. For example, the system can identify that a certain specific ad content or display time directly increases user engagement and purchase rate. The low-dimensional vector mapping is a step to convert the high-dimensional causal path weight coefficients into a more concise vector representation through dimensionality reduction techniques. This step helps the system to convert complex causal relationships into a more easily analyzed and processed format. This mapping helps to extract the most influential factors from a large number of causal paths and reveals potential implicit drivers. The low-dimensional vector after dimensionality reduction facilitates the extraction of meaningful implicit driving factors in large-scale advertising data. Implicit driving factors are factors that are not easily perceived but have a profound impact on advertising effectiveness, such as users' potential interests or long-term trends. Implicit driving factors refer to factors that are not easily observed or measured, related to users' emotional preferences, long-term purchase habits, or social influences. Through low-dimensional vector mapping, these difficult-to-observe implicit factors can be identified. By identifying implicit driving factors, the advertising system can gain a deeper understanding of users' long-term interests and needs, thus avoiding short-term misjudgments caused by seasonal promotional activities. For example, the system may find that certain ad placement factors (such as time, method, etc.) do not directly lead to purchase behavior, but they subtly influence users' long-term interests, thus improving the accuracy of ad recommendations. The comprehensive analysis of explicit and implicit driving factors helps to form a comprehensive user interest model. Ultimately, through the comprehensive evaluation of these driving factors, the advertising system can more accurately adjust the ad placement strategy and optimize the user experience.

[0075] In an embodiment, after the step of determining the reason for not meeting the target user interest demand, the method further comprises:

[0076] Analyzing the interest change parameters of the target user to resolve the reason for not meeting the target user interest demand;

[0077] Re-predicting the target user's current interest label according to the interest change parameters;

[0078] Generating the advertising data adjustment strategy based on the re-predicted interest label and the reason for not meeting the target user interest demand.

[0079] As mentioned above, identifying why the target user's interest needs are not met in the advertisement pushing or recommendation system can be achieved by comparing the user's historical behavior, advertisement feedback (such as click rate, purchase rate), and the predetermined interest model. If the advertisement content does not match the user's interest needs, it may be due to the low matching degree of the advertisement content itself with the user's needs, or due to external factors such as the time and frequency of advertisement placement, which weaken the influence of the advertisement. Interest change parameters refer to various indicators that measure the change in user interest, such as the upward or downward trend of a user's interest in a certain type of goods or service. Interest changes can be inferred from user behavior data, such as browsing history, purchase records, search queries, and reactions to advertisements (click, skip, ignore, etc.). The analysis of interest changes not only focuses on short-term fluctuations, but also considers long-term trends. Analyzing the user's interest change parameters can help the system identify the dynamic trend of the user's current interest, rather than just the static interest preference. This is crucial for adjusting the advertisement strategy, as user interest is constantly changing, and the advertisement system needs to adjust its prediction model accordingly to improve the relevance and effectiveness of the advertisement. Interest labels are category labels assigned by the advertisement recommendation system based on user interest (e.g., sports enthusiasts, tech geeks, fashionistas, etc.). Re-predicting the target user's interest label is based on the analysis of their interest change parameters, and the user's interest label is updated and adjusted in real time through machine learning models (such as collaborative filtering, content recommendation, deep learning models, etc.) in the recommendation system. This process will determine whether the user's interest has changed based on their recent behavior and historical trends, and reassign the interest label. Based on the new interest label and the identified reasons for not meeting the user's needs, the advertisement system will adjust the placement strategy of the advertisement data. These adjustments may include optimizing the display time of the advertisement, adjusting the content of the advertisement creative, re-setting the frequency of the advertisement, adjusting the target audience group, etc. The core of the advertisement strategy adjustment is to ensure that the advertisement content can meet the user's needs to the greatest extent based on the user's dynamic interest label and preference. The purpose of generating an advertisement data adjustment strategy is to improve the relevance, appeal, and effectiveness of the advertisement. By combining the user's real-time interest needs with the advertisement system's strategy, the advertisement display is more accurate, reducing the push of ineffective advertisements, thereby improving the advertisement conversion rate and reducing user churn. Through such strategy adjustment, the advertisement can effectively avoid the lag response of a single push strategy to changes in user needs.

[0080] In an embodiment, the interest tags include content-level tags, form-level tags, and scenario-level tags. By comprehensively analyzing these three types of tags, the advertisement recommendation system can generate more refined user portraits. For example, a certain user may be interested in "travel" content (content-level tags), prefer to receive information through "video advertisements" (form-level tags), and often view travel-related content during "afternoon on weekends" (scenario-level tags). Based on these tags, the system can push "video advertisements" related to "weekend travel". By monitoring the changes of these tags in real time, the advertisement system can timely adjust the advertisement delivery strategy. For example, if the user's interest tags change from "technology" to "health", the advertisement system can quickly update the content-level tags and preferentially recommend related advertisements such as fitness and health products, rather than continue to deliver irrelevant technology advertisements. The combination of content-level, form-level, and scenario-level tags makes the advertisement recommendation more personalized, not only focusing on the user's long-term interests, but also optimizing according to the user's current situation. For example, by identifying the user's outing scenario through scenario-level tags, the system can push advertisements related to outings, catering, and entertainment, while at home, it may push home product or video entertainment advertisements.

[0081] Referring to Figure 3 In the embodiments of the present application, a user interest prediction device based on multi-modal data is also provided, comprising:

[0082] The acquisition module 1 is configured to acquire multi-modal historical data of a target user.

[0083] The identification module 2 is configured to identify influence variables in the multi-modal historical data.

[0084] The segmentation module 3 is configured to segment the influence variables from the multi-modal historical data according to the identification results.

[0085] The construction module 4 is configured to construct a demand analysis model based on the segmentation results.

[0086] The input module 5 is configured to acquire behavior data of the target user in real time, input the behavior data into the interest prediction model, and output a demand data set corresponding to the current behavior data of the target user through the demand analysis model.

[0087] The prediction module 6 is configured to analyze the demand data set based on the timestamp based on the output results, and predict the current interest tags of the target user.

[0088] The sending module 7 is configured to generate advertisement data according to the interest tags and send the advertisement data to the target user.

[0089] As described above, it can be understood that each component of the user interest prediction device based on multi-modal data proposed in the present application can realize the functions of any one of the user interest prediction methods based on multi-modal data as described above, and the specific structure will not be described again.

[0090] Referring to Figure 4 , the present application also provides a computer device, which can be a server, and the internal structure thereof can be as shown in Figure 4 . The computer device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is used to store monitoring data and other data. The network interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement a user interest prediction method based on multi-modal data.

[0091] The processor executes the user interest prediction method based on multi-modal data as described above, including: obtaining multi-modal historical data of a target user; identifying influence variables in the multi-modal historical data; segmenting the influence variables from the multi-modal historical data according to the identification result; constructing a demand analysis model based on the segmentation result; obtaining behavior data of the target user in real time, inputting the behavior data into the interest prediction model, outputting a demand data set corresponding to the current behavior data of the target user through the demand analysis model; based on the output result, analyzing the demand data set based on a timestamp, and predicting the current interest label of the target user; generating advertisement data according to the interest label and sending it to the target user.

[0092] An embodiment of the present application also provides a computer readable storage medium having a computer program stored thereon, which is executed by a processor to implement a user interest prediction method based on multi-modal data, including the steps of: obtaining multi-modal historical data of a target user; identifying influence variables in the multi-modal historical data; segmenting the influence variables from the multi-modal historical data according to the identification result; constructing a demand analysis model based on the segmentation result; obtaining behavior data of the target user in real time, inputting the behavior data into the interest prediction model, outputting a demand data set corresponding to the current behavior data of the target user through the demand analysis model; based on the output result, analyzing the demand data set based on a timestamp, and predicting the current interest label of the target user; generating advertisement data according to the interest label and sending it to the target user.

[0093] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, databases, or other media in this application and in examples provided herein, unless specifically stated otherwise, can include non-volatile and / or volatile memory. Non-volatile memory can include, for example, read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include, for example, random access memory (RAM), or external cache memory. As an illustration and not a limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus DRAM (RDRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0094] It should be noted that, in this document, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, device, article, or method that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, device, article, or method. Without more limitations, an element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, device, article, or method that includes the element.

[0095] The above description is only the preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A user interest prediction method based on multimodal data, characterized in that, The method includes: Obtain multimodal historical data of the target user; Identify the influencing variables in the multimodal historical data; Based on the identification results, the influencing variables are segmented from the multimodal historical data; Based on the segmentation results, a requirements analysis model is constructed. Real-time acquisition of target user behavior data, input of the behavior data into the demand analysis model, and output of the demand dataset corresponding to the target user's current behavior data through the demand analysis model; Based on the output results, the demand dataset is analyzed using timestamps, and the current interest tags of the target user are predicted. Based on the interest tags, advertising data is generated and sent to the target user; After the step of generating advertising data based on the interest tags and sending it to the target user, the method includes: receiving behavioral feedback from the target user in real time; determining whether the advertising data meets the target user's interest needs based on the behavioral feedback; quantifying the behavioral feedback to obtain the target user's interest coefficient; if the interest coefficient is lower than a threshold, determining that the advertising data does not meet the target user's interest needs; analyzing the causal characteristics of the advertising data based on the fact that the advertising data does not meet the target user's interest needs, and identifying the driving factors that led to the target user's interest; determining the reasons for not meeting the target user's interest needs based on the driving factors, and generating an adjustment strategy for the advertising data based on the reasons for not meeting the target user's interest needs; adjusting the advertising data based on the adjustment strategy; the step of... The method for analyzing the causal characteristics of the advertising data and identifying the driving factors for the formation of the target users includes: extracting behavioral data, influencing variables, and advertising data from the multimodal historical data; constructing a hierarchical causal graph based on the extraction results to determine all causal paths of the multimodal historical data; obtaining the revenue conversion rate of all causal paths and calculating the weight coefficient of each causal path based on the revenue conversion rate; adding a virtual intervention test to each causal path and calculating the revenue conversion rate after the intervention; identifying the explicit driving factors of the causal path based on the revenue conversion rate before and after the intervention; adding a virtual intervention test to each causal path, mapping the weight coefficients to a low-dimensional vector, and identifying the implicit driving factors of the causal path based on the mapping results; and determining the driving factors based on the explicit and implicit driving factors.

2. The user interest prediction method based on multimodal data according to claim 1, characterized in that, The steps of analyzing the demand dataset based on timestamps and predicting the current interest tags of the target user include: Analyze the scene characteristics of the behavioral data based on the behavioral data; Based on the analysis results, predict the interest change characteristics of the target user in relation to the scene features; The interest change features, scene features, and behavioral data are timestamped and then fused together to obtain joint features. The joint features are input into a preset classification model, and the classification model predicts the target user's interest tags based on the joint features.

3. The user interest prediction method based on multimodal data according to claim 2, characterized in that, The step of generating advertising data based on the interest tags and sending it to the target user includes: Obtain the multimodal historical data; Based on the multimodal historical data, the revenue conversion rate of all transmission paths is determined, and the weight coefficient of each transmission path is calculated based on the revenue conversion rate. Acquire scene features and advertising data from current behavioral data; Based on the scene characteristics and advertising data, analyze the conflicting paths in all transmission paths; Based on the analysis results, available transmission paths are determined, and the transmission cost of the available transmission paths is calculated. Based on the calculation results, the available transmission path with the lowest transmission cost and the highest weight coefficient is selected as the transmission path for this operation. The advertising data is sent to the target user based on the determined transmission path.

4. The user interest prediction method based on multimodal data according to claim 1, characterized in that, After determining the reasons why the target user's interest needs are not met, the method further includes: Analyze the reasons why the target user's interest needs are not met, and analyze the target user's interest change parameters; Re-predict the target user's current interest tags based on the interest change parameters; An adjustment strategy for the advertising data is generated based on the re-predicted interest tags and the reasons why the target user's interest needs are not met.

5. The user interest prediction method based on multimodal data according to claim 1, characterized in that, The interest tags include content-level tags, format-level tags, and scenario-level tags.

6. A user interest prediction device based on multimodal data, used in the method described in any one of claims 1-5, characterized in that, include: The acquisition module is used to acquire multimodal historical data of the target user; The identification module is used to identify the influencing variables in the multimodal historical data; The segmentation module is used to segment the influencing variables from the multimodal historical data based on the identification results; Build modules are used to construct requirements analysis models based on the segmentation results; The input module is used to acquire the target user's behavior data in real time, input the behavior data into the demand analysis model, and output the demand dataset corresponding to the target user's current behavior data through the demand analysis model. The prediction module is used to analyze the demand dataset based on the output results and timestamps, and to predict the current interest tags of the target user. The sending module is used to generate advertising data based on the interest tags and send it to the target user.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images