User interest prediction method and device based on multi-modal data

Abstract

The invention relates to the field of advertisement recommendation, in particular to a user interest prediction method and device based on multi-modal data, and the method comprises the steps: obtaining multi-modal historical data of a target user; identifying influence variables in the multi-modal historical data; according to an identification result, segmenting the influence variable from the multi-modal historical data; constructing a demand analysis model based on a segmentation result; acquiring behavior data of the target user in real time, inputting the behavior data into the interest prediction model, and outputting a demand data set corresponding to the current behavior data of the target user through the demand analysis model; analyzing the demand data set based on the output result and the timestamp, and predicting the current interest tag of the target user; and generating advertisement data according to the interest label and sending to the target user. And interest prediction is more accurate through multi-angle data integration. Key variables influencing user interests are recognized, irrelevant information is removed through feature selection, and the precision of an interest prediction model can be remarkably improved.

Description

Technical Field

[0001] This application relates to the technical field of advertising recommendation, and particularly to a method and device for predicting user interests based on multi-modal data. Background Art

[0002] Predicting user interests from multi-modal data refers to integrating data from different sources and in different forms (such as text, images, voice, video, user behavior data, etc.) to more accurately predict and understand users' interests, preferences, and needs. This process combines multiple data modalities (i.e., multiple types of information) for comprehensive analysis to improve the effectiveness of recommendation systems, advertising placement, personalized services, etc.

[0003] In the prior art, products are usually pushed based on users' short-term purchase behaviors. For example, when a user purchases a certain product, the system may push similar products based on this behavior. However, if this is just a one-time purchase by the user due to discounts or promotions, the system is likely to misjudge this behavior as the user's long-term interest. If a user purchases a product due to seasonal discounts or promotions (such as air conditioners in summer and down jackets in winter), the system may mistakenly think that the user will prefer these products in future needs and keep pushing similar products later. This can lead to the situation where the user is no longer interested in the product in actual demand, but the system keeps pushing similar products, resulting in advertising redundancy and reducing the user experience. Secondly, the prior art usually focuses on content and personalized recommendation, but pays insufficient attention to the adaptability of the advertising transmission method. Different advertising contents are suitable for different transmission channels and display methods (such as push notifications, emails, pop-ups, social platforms, etc.). If a unified transmission method is adopted, it is likely to lead to the deterioration of the user experience and even miss advertisements that meet the user's needs.

[0004] Therefore, there are deficiencies in the prior art and improvements are needed. Summary of the Invention

[0005] To solve one or several problems in the prior art, the main objective of this application is to provide a method and device for predicting user interests based on multi-modal data.

[0006] To achieve the above-mentioned invention objective, this application proposes a method for predicting user interests based on multi-modal data, and the method includes:

[0007] Obtain multi-modal historical data of a target user;

[0008] Identify influencing variables in the multi-modal historical data;

[0009] According to the identification result, segment the influencing variables from the multi-modal historical data;

[0010] Construct a requirements analysis model based on the segmentation results;

[0011] Obtain the behavior data of the target user in real time, input the behavior data into the interest prediction model, and output the requirements dataset corresponding to the current behavior data of the target user through the requirements analysis model;

[0012] Based on the output results, analyze the requirements dataset based on the timestamp and predict the current interest tags of the target user;

[0013] Generate advertisement data according to the interest tags and send it to the target user.

[0014] An embodiment of the present application further provides a user interest prediction device based on multimodal data, including:

[0015] An acquisition module, configured to acquire the multimodal historical data of the target user;

[0016] An identification module, configured to identify the influencing variables in the multimodal historical data;

[0017] A segmentation module, configured to segment the influencing variables from the multimodal historical data according to the identification results;

[0018] A construction module, configured to construct a requirements analysis model based on the segmentation results;

[0019] An input module, configured to acquire the behavior data of the target user in real time, input the behavior data into the interest prediction model, and output the requirements dataset corresponding to the current behavior data of the target user through the requirements analysis model;

[0020] A prediction module, configured to analyze the requirements dataset based on the timestamp and predict the current interest tags of the target user based on the output results;

[0021] A sending module, configured to generate advertisement data according to the interest tags and send it to the target user.

[0022] The present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in any one of the above are implemented.

[0023] The present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in any one of the above are implemented.

[0024] The user interest prediction method and device based on multimodal data according to the embodiments of the present application can comprehensively capture the interests and needs of users from multiple dimensions (such as text, images, behaviors, etc.) by obtaining multimodal historical data. This multi-angle data integration makes interest prediction more accurate. Identifying the key variables that affect user interests and removing irrelevant information through feature selection can significantly improve the accuracy of the interest prediction model. By finely analyzing the influencing factors, the model is more in line with the actual needs of users. By obtaining user behavior data in real time, the system can dynamically adjust user interest prediction to ensure that the recommended content matches the real-time needs of users, thereby improving the user experience. The demand analysis model can provide more accurate personalized recommendations according to the current interest needs of users by deeply mining user behaviors and interests, optimize advertising and content push, and avoid information overload and irrelevant recommendations. Combining timestamp information for analyzing the trend of interest changes enables the prediction to not only accurately reflect the current interest but also understand the changing needs of users over time, thus achieving more timely recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a schematic flowchart of a user interest prediction method based on multimodal data according to an embodiment of the present application;

[0026] Figure 2 is a schematic flowchart of a user interest prediction method based on multimodal data according to an embodiment of the present application;

[0027] Figure 3 is a schematic block diagram of the structure of a user interest prediction device based on multimodal data according to an embodiment of the present application;

[0028] Figure 4 is a schematic block diagram of the structure of a computer device according to an embodiment of the present application.

[0029] The realization, functional features and advantages of the objectives of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0031] Refer to Figure 1 , in an embodiment of the present application, a user interest prediction method based on multimodal data is provided, and the method includes:

[0032] S1. Obtain the multimodal historical data of the target user;

[0033] S2. Identify the influencing variables in the multimodal historical data;

[0034] S3. According to the recognized results, segment the influencing variables from the multimodal historical data;

[0035] S4. Based on the segmentation results, construct a demand analysis model;

[0036] S5. Obtain the behavior data of the target user in real time, input the behavior data into the interest prediction model, and output the demand dataset corresponding to the current behavior data of the target user through the demand analysis model;

[0037] S6. Based on the output results, analyze the demand dataset based on the timestamp and predict the current interest tags of the target user;

[0038] S7. Generate advertisement data according to the interest tags and send it to the target user.

[0039] As described in the above steps S1 - S3, by collecting various behavior data of users, a comprehensive user historical dataset is constructed. Multimodal data includes text (such as search records, comments, social media content), images (such as photos uploaded by users or product images clicked), audio, video, and behavior data (such as clicks, purchases, browsing, etc.). Through these data, multi - angle information of user behavior can be obtained. By integrating data of different modalities, a comprehensive understanding of users' interests, behaviors, and needs can be achieved. Combining information from different sources can improve the prediction accuracy of users' interests and avoid biases that may exist in single - modality data. Influencing variables refer to those data elements that play a key role in predicting users' interests, behaviors, or needs. In this step, by analyzing and processing multimodal data, features that have a greater impact on interest prediction are identified. For example, users' click frequencies, comment sentiments, browsing durations, visual attractiveness of product images, etc. can all be influencing variables. By identifying important variables, interference from irrelevant data can be avoided, the input of analysis can be streamlined, and the calculation efficiency can be improved. Identifying the key factors affecting interests helps to establish a more targeted interest prediction model, thereby improving the prediction accuracy. Segmenting influencing variables means extracting key influencing factors from multimodal data and performing feature selection or transformation on these data. This may include performing feature engineering on user behavior data, selecting the features most relevant to interest prediction, and segmenting important variables. After segmenting the influencing variables, interference from irrelevant data can be reduced, and focus can be placed on information helpful for prediction, thereby improving the precision and efficiency of the model.

[0040] As described in the above steps S4 - S7, the requirements analysis model is an analysis framework constructed based on the segmented impact variables, which is used to capture the potential requirements of users in the current context. Usually, the requirements analysis model can utilize machine learning or deep learning methods, such as regression analysis, classification models, or neural networks, to learn the patterns of user requirements from historical data. This model can effectively capture the potential requirements of users and help predict the content or products that users may be interested in at a certain point in time. Based on the requirements analysis model, the system can better provide personalized recommendations or services according to the user's interests and needs, enhancing the user experience. Real - time acquisition of behavioral data means continuously collecting the behavioral information of users, such as clicks, browsing, searches, purchases, etc., when users interact with the platform. These data are promptly input into the interest prediction model so that the model can update the prediction of the user's interest in real - time according to the current behavior. Real - time acquisition of the user's behavioral data can ensure that the update of the interest prediction model is timely and can reflect the current interest changes of users. Through real - time data, the model can dynamically adjust the interest tags to promptly adapt to the interest changes of users and provide instant personalized recommendations. This process maps the real - time acquired user behavioral data to a requirements dataset through the requirements analysis model. The requirements dataset represents the interest requirements of users at the current moment. The requirements dataset may contain a set of tags or features that reflect the content, products, or services currently preferred by users. Through the requirements dataset output by the requirements analysis model, the current interests and needs of users can be accurately mapped, avoiding inaccurate recommendations caused by unclear user requirements. According to the precise requirements provided by the requirements dataset, the conversion rate of advertisements or recommendations can be increased, optimizing the advertising placement effect. Analyzing according to the requirements dataset and timestamp information can understand the changing trend of user interests and predict the current interest tags. The timestamp plays a crucial role here. It helps to identify the laws of user interest changes over time. For example, users may be interested in news in the morning and entertainment content in the evening. Using the timestamp for the time - series analysis of requirements data helps to capture the time dynamics of user interests, making the prediction more accurate and personalized. Through the time analysis of the requirements dataset, the changing interests of users can be captured in real - time, maintaining the timeliness and relevance of recommendations. According to the predicted user interest tags, the system can generate relevant advertisement data. The advertisement data is based on the user's interest tags and requirements data, and specifically pushes advertisement content that matches the user's interests to improve the click - through rate and conversion rate of advertisements. The advertisement content generated according to the interest tags can ensure a high degree of match between the advertisement and the user's interests, enhancing the user's willingness to click and interact. By pushing more relevant advertisements, the user's acceptance of advertisements is increased, thereby enhancing the user's satisfaction and the advertising revenue of the platform.

[0041] As described above, by obtaining multi-modal historical data, it is possible to comprehensively capture users' interests and needs from multiple dimensions (such as text, images, behaviors, etc.). This multi-angle data integration makes interest prediction more accurate. Identifying the key variables that affect users' interests and removing irrelevant information through feature selection can significantly improve the accuracy of the interest prediction model. By finely analyzing the influencing factors, the model is closer to users' actual needs. By obtaining users' behavior data in real time, the system can dynamically adjust the prediction of users' interests to ensure that the recommended content matches users' real-time needs, thereby enhancing the user experience. The demand analysis model can provide more accurate personalized recommendations based on users' current interest needs by deeply mining users' behaviors and interests, optimize the advertising and content push, and avoid information overload and irrelevant recommendations. Analyzing the trend of interest changes in combination with timestamp information enables the prediction to not only accurately reflect the current interests but also understand users' changing needs over time, thus achieving more timely recommendations.

[0042] Referring to Figure 2 , in one embodiment, the steps of analyzing the demand data set based on the timestamp and predicting the current interest tags of the target user include:

[0043] S61. Analyze the scenario features of the behavior data according to the behavior data;

[0044] S62. Based on the analysis results, predict the interest change features of the scenario features for the target user;

[0045] S63. Align the timestamps of the interest change features, scenario features, and behavior data, and perform feature fusion on the processed interest change features, scenario features, and behavior data features to obtain joint features;

[0046] S64. Input the joint features into a preset classification model, and use the classification model to predict the interest tags of the target user according to the joint features.

[0047] As described in the above steps, behavioral data is multi-dimensional data generated during the interaction between users and the platform (such as clicks, browsing, purchases, etc.). These behaviors are usually related to specific scenarios (for example, the behaviors of users in different scenarios such as shopping, browsing news, and watching videos are different). By analyzing the scenario features of behavioral data, the aim is to extract the environmental and contextual factors related to specific behaviors. For example, the behavioral patterns of users at a specific time, location, and device may reflect their current interests or needs. Scenario features include time (such as morning, noon, evening), geographical location (such as workplace, home), device (such as mobile phone, computer), etc., and these factors have an impact on user behavior. Analyzing these scenario features can help understand the background of behaviors, and then infer the changes in users' interests in different scenarios. By analyzing scenario features, the potential needs behind users' behaviors can be captured more accurately. For example, when a user is in a shopping scenario, they may show interests related to goods, while in a social scenario, they may show interests in entertainment content. By analyzing the relationship between scenario features and user behaviors, the changing trend of users' interests in that scenario can be inferred. This change is usually temporal, that is, users' interests do not remain constant, but change over time and with the switching of scenarios. This prediction can dynamically capture the fluctuations and changes in users' interests, generating more accurate interest predictions not only based on users' current behaviors but also combined with contextual factors such as time and location. Using these changing trends, the system can timely adjust the recommendation strategy to improve the real-time performance and accuracy of personalized recommendations. When processing user behavioral data, timestamp alignment is an important step because users' behaviors are time-dependent. Timestamp alignment processing refers to synchronizing all relevant data (interest change features, scenario features, behavioral data) based on timestamps to ensure the consistency of time information for each feature. This step can ensure that the model can accurately understand the correlation between each feature at the same time point when making interest predictions. Behavioral data, interest change features, and scenario features may have different time frequencies or sampling intervals, and timestamp alignment can align these features within the same time frame. Timestamp alignment processing ensures that the model can reflect short-term and long-term interest changes. Through timestamp alignment processing, the system can capture the rapid changes in users' interests in the short term and the trends on a long time scale. This step helps the model understand the interest state of users at a certain time point and enhances the accuracy of the prediction results. Fusing interest change features, scenario features, and behavioral data features to form a joint feature is to solve the problem of complementarity between features. Each individual feature dimension reveals different aspects of users' interests, and through feature fusion, the system can integrate information from multiple dimensions to obtain a more comprehensive interest prediction. Interest change features reveal the dynamic process of users' interests changing over time. Scenario features describe the background and environment of users' behaviors, helping to understand users' interest preferences in different situations.Behavior data features show the way users interact with the platform and are the direct basis for interest prediction. Classification models (such as logistic regression, support vector machines, deep learning models, etc.) can infer the current user's interest labels by learning existing user behavior data and labels. The core task of a classification model is to classify based on feature data and predict the most likely current interest labels of the user. The preset classification model is usually trained based on historical data. By comparing the combined features of the current user, the model can output the predicted interest labels. Through the output of the classification model, accurate personalized recommendations can be provided for users to ensure that the recommended content is highly relevant to the user's current interests.

[0048] In one embodiment, the step of generating advertisement data according to the interest label and sending it to the target user includes:

[0049] Obtain the multimodal historical data;

[0050] According to the multimodal historical data, determine the revenue conversion rate of all transmission paths, and calculate the weight coefficient of each transmission path according to the revenue conversion rate;

[0051] Obtain the scenario features and advertisement data of the current behavior data;

[0052] According to the scenario features and advertisement data, analyze the conflicting paths existing in all transmission paths;

[0053] Based on the analysis results, determine the available transmission paths and calculate the transmission costs of the available transmission paths;

[0054] According to the calculation results, select the available transmission with the lowest transmission cost and the highest weight coefficient as the transmission path for this time;

[0055] Send the advertisement data to the target user based on the determined transmission path.

[0056] As described above, multimodal historical data includes a user's past click behavior, purchase records, browsing history, social media activities, etc. By comprehensively analyzing these historical behaviors, the system can more accurately understand the changes in user interests and their behavior patterns in different environments. These historical data will help the system understand the preferences and behavior logics of users in different scenarios. Multimodal data can reflect multiple behavior dimensions of users, helping to build a more accurate user interest model. Through the integration of multi-source information, it is possible to better infer the current needs of users, thereby optimizing the advertising push strategy. The transmission path refers to the dissemination route through which advertising data reaches users from the platform, which can be multiple channels, such as recommendation engines, social media, email pushes, etc. The revenue conversion rate of each channel represents the effect that this path can bring in past placements (such as click-through rate, purchase rate, etc.). The weight coefficient reflects the relative importance of each path. The revenue conversion rate is based on the statistical analysis of historical data and measures the success rate of different transmission paths (such as different advertising recommendation methods). Paths with high conversion rates may be given higher weights. The weight coefficient is adjusted according to the conversion rate to help select the most suitable advertising dissemination method among multiple transmission paths. By calculating the revenue conversion rate and the weight coefficient, the most effective transmission path can be selected to improve the conversion effect of advertisements. Selecting the path with a higher weight coefficient for advertising transmission can ensure that limited advertising resources are used for the channels that are most likely to bring high returns. The current behavior data reflects some recent behaviors of users (such as browsing pages, clicking on advertisements, interacting with the platform, etc.). The scenario characteristics describe the specific background in which user behaviors occur, such as time, location, device, emotional state, etc. By obtaining the current behavior data and relevant scenario characteristics, the advertising system can better understand the needs of users at a certain moment. Scenario characteristics help to understand the background of user behaviors. For example, when a user watches videos on their mobile phone at night, the advertising content may tend to be entertainment and consumer goods; while during the day at work, the advertisements may be more inclined towards office software or news information. Advertising data includes information such as the content of the advertisement itself, the target user group, and the advertising budget. Combining with the current scenario characteristics of users can provide a basis for the selection and customization of advertising content. By combining the current behaviors and scenario characteristics of users, advertising placement can more accurately match the immediate needs of users. Make the advertisement more in line with the interests and scenarios of users, thereby reducing user interference and enhancing the acceptance of advertisements. During the advertising placement process, conflicts or repetitions may occur between multiple transmission paths. For example, if a user has already received similar advertisements on other channels, the new advertising path may conflict with the previous advertising path, resulting in duplicate advertising placements or content conflicts. The analysis of conflict paths is to avoid duplicate advertising content placements or to avoid pushing similar or irrelevant advertising content through different paths, thereby causing user dissatisfaction or a decline in advertising effects. By identifying and excluding conflict paths, the system can ensure the diversity and effectiveness of advertising placement.Reduce the repeated appearance of the same advertisement content to avoid annoying users. After excluding conflicting paths, the advertisement can reach users more effectively and avoid wasting resources. After excluding conflicting paths, the system needs to evaluate the costs of the remaining available transmission paths. Each transmission path may have different costs. For example, some advertising channels may require payment, while other channels may be free. By evaluating the costs of these paths, the system can select the path with the highest cost-effectiveness. Transmission costs include factors such as the cost of advertising placement, the resources used by the platform, and possible time delays. The system needs to weigh and choose the most suitable advertising placement channel based on these costs. By calculating the transmission costs, the system can select an advertising path with lower costs and higher benefits, avoiding ineffective expenditures. Ensure that advertising placement can achieve the best results at the lowest cost, thereby increasing the overall return on investment in advertising. After calculating the costs and weight coefficients of each path, the system needs to select the transmission path with the lowest cost and the highest weight coefficient as the final advertising transmission method. This selection is based on comprehensive benefits, that is, while ensuring efficient transmission, minimizing costs as much as possible. Selecting a path with a higher weight coefficient means that the system selects the path that is most likely to bring a higher conversion rate for placement, while controlling costs to ensure the economy of placement. This decision ensures the optimal allocation of advertising resources and reduces unnecessary waste. By selecting the most suitable transmission path, it is possible to maximize the advertising effect while reducing costs. It can ensure that the advertisement is transmitted through the most suitable channel and at the most economical cost, enhancing the return on investment in advertising. Finally, based on the above steps, after determining the optimal transmission path, the advertisement data is transmitted to the target user. This process involves sending the advertisement content to the user through the selected transmission channel. The selection of the transmission path has been made based on the principle of maximizing the conversion rate and minimizing the cost, and the advertisement will be pushed to the user through the most effective channel. The advertisement can reach the target user quickly and effectively in the most suitable way, thereby increasing the advertisement's exposure rate, click-through rate, and conversion rate.

[0057] In one embodiment, after the step of generating advertisement data according to the interest tags and sending it to the target user, the method includes:

[0058] Receiving the behavioral feedback of the target user in real time;

[0059] Judging whether the advertisement data meets the interest needs of the target user according to the behavioral feedback of the target user;

[0060] Quantifying the behavioral feedback to obtain the interest coefficient of the target user;

[0061] If the interest coefficient is lower than the threshold, it is judged that the advertisement data does not meet the interest needs of the target user;

[0062] Based on the fact that the advertising data does not meet the interest needs of the target user, analyze the causal characteristics of the advertising data and identify the driving factors formed by the target user;

[0063] According to the driving factors, determine the reasons for not meeting the interest needs of the target user, and generate an adjustment strategy for the advertising data based on the reasons for not meeting the interest needs of the target user;

[0064] Adjust the advertising data based on the adjustment strategy.

[0065] As described above, after the advertisement is launched, real-time monitoring and receiving of the behavioral feedback of target users are carried out, such as click-through rate, browsing duration, interaction behavior, etc. These feedback data reflect the users' reactions and interest levels towards the advertisement content. Obtaining feedback in real time can help the advertisement system quickly understand the popularity of the advertisement, promptly identify the users' interest points and the adaptability of the advertisement, ensuring that the advertisement placement can be adjusted immediately, rather than waiting for a long-term result evaluation. The advertisement system analyzes whether the advertisement content matches the interest tags of the target users based on the collected user behavior data (such as clicks, browsing, dwell time, etc.). Specifically, if the user does not show positive behavior towards the advertisement (such as a low click-through rate), it can be inferred that the advertisement does not meet the users' interest needs. This judgment mechanism helps to immediately evaluate the effectiveness of the advertisement and optimize the advertisement placement strategy, avoiding the placement of uninteresting or ineffective advertisement content, thereby improving the matching degree of the advertisement and the user experience. The process of quantifying behavioral feedback usually converts the users' interaction behaviors (such as clicks, dwell time, number of interactions, etc.) into a quantitative interest coefficient. This coefficient can be a floating value, used to represent the interest intensity of the target user towards a specific advertisement or advertisement type. By quantifying behavioral feedback, the advertisement system can accurately capture the dynamic changes in users' interests, and thus adjust the advertisement content according to different interest levels, enhancing the personalization and accuracy of the advertisement. This process is a decision-making process, using a set interest coefficient threshold to determine whether the advertisement meets the users' needs. When the interest coefficient is lower than the preset threshold, it means that the advertisement content no longer attracts users, and the system will mark the advertisement as "mismatched". By setting the threshold, the advertisement system can timely filter out inefficient advertisements, avoid wasting advertisement resources, and ensure the relevance of the advertisement content. The selection of the threshold is crucial for the system effect, and it needs to be adjusted based on the actual situations of the user group, advertisement type, and placement platform. When the advertisement does not meet the interest needs of the target users, the system needs to further analyze the causal characteristics of the advertisement. For example, factors such as the advertisement content itself, display method, time period, etc. may affect the users' reactions. Causal analysis helps the system find out which specific characteristics have not successfully attracted users, and then identify the driving factors affecting the advertisement effect. This causal analysis helps the advertisement system identify which specific factors cause the advertisement not to meet the users' interests, thereby providing a basis for adjusting the advertisement strategy. It improves the intelligence level of the advertisement system and can further refine the advertisement content through data analysis. Based on the analyzed driving factors, the advertisement system identifies problems in the advertisement content or form and formulates adjustment strategies accordingly. For example, if the advertisement lacks creativity or the placement time is inappropriate, the advertisement system can automatically adjust the display method or content of the advertisement to improve the attractiveness. By analyzing the reasons for not meeting the users' interest needs, the system can automatically optimize the advertisement content and improve the matching degree between the advertisement and the users' interests. This adjustment can improve the effectiveness of the advertisement and the ROI (return on investment) of the advertisement placement.After identifying the specific reasons why the advertisement content does not meet the user's needs, the advertisement system will apply adjustment strategies to modify the advertisement content or reselect the delivery time. This may include updates to content creativity, adjustments to user segmentation, changes in advertisement frequency, etc. By continuously adjusting the advertisement strategy, the advertisement delivery becomes more refined, thereby improving the performance of the advertisement. The personalization and precision of the advertisement are greatly enhanced, which can increase the user interaction rate and ultimately increase advertisement revenue or achieve the goal.

[0066] In one embodiment, for analyzing the causal characteristics of the advertisement data and identifying the driving factors formed by the target users, the method includes:

[0067] Extract the behavioral data, influencing variables, and advertisement data of the multimodal historical data;

[0068] Based on the extraction results, construct a hierarchical causal graph to determine all causal paths of the multimodal historical data;

[0069] Obtain the revenue conversion rates of all causal paths, and calculate the weight coefficients of each causal path according to the revenue conversion rates;

[0070] Conduct a virtual intervention test on each causal path and calculate the revenue conversion rate after the intervention;

[0071] Based on the revenue conversion rate before the intervention and the revenue conversion rate after the intervention, identify the explicit driving factors of the causal path;

[0072] Conduct a virtual intervention test on each causal path, map the weight coefficients to low-dimensional vectors, and based on the mapping results, identify the implicit driving factors of the causal path;

[0073] Based on the explicit driving factors and implicit driving factors, determine the driving factors.

[0074] As mentioned above, data is extracted from multiple sources, including historical user behaviors (e.g., clicks, browsing, purchases), external influencing variables (e.g., seasons, holidays, promotions, etc.), and advertising data itself (e.g., advertising type, display time, creative content, etc.). Multimodal data can provide comprehensive information for user portraits from multiple dimensions. By extracting multi-dimensional data, the diverse behaviors of users and the influence of external environmental factors can be captured more accurately. In particular, a comprehensive understanding of advertising push behavior can help avoid a single data source misleading the advertising system to misjudge user interests and reduce the phenomenon of false pushes caused by short-term factors such as promotions. Use a hierarchical causal graph to construct causal relationships between multimodal data. By establishing causal paths between variables, analyze which variables affect advertising effects and how they affect each other. For example, user purchasing behavior may be affected by both seasonal discounts and advertising display time, and a hierarchical causal graph can help determine the causal relationship between these factors. A hierarchical causal graph provides a structured way to help advertising systems understand complex causal relationships and avoid overly simplified models. For example, the system can distinguish between a user's purchase behavior due to a short-term promotion and long-term interests, so as to no longer misjudge the user's long-term needs. Revenue conversion rate (ROI) refers to the measure of advertising effectiveness, that is, the conversion ratio between advertising investment and user behavior (such as purchases, clicks, etc.). By calculating the revenue conversion rate of each causal path, it is possible to determine which causal paths contribute more to the advertising effect. The weight coefficient is a weighted causal path based on the conversion rate, reflecting its degree of influence on the final advertising effect. According to the revenue conversion rate and weight coefficient of each causal path, the system can clearly identify which causal factors play a leading role in user behavior. This helps to more accurately identify which factors are long-term drivers of user purchasing behavior and which factors are short-term and sporadic, thereby avoiding erroneous advertising pushes that rely too much on short-term factors. Virtual Intervention Test refers to observing the effect of changes in causal paths after intervention by simulating intervention (for example, assuming changes in a certain variable). By calculating the revenue conversion rate before and after the intervention, the system can identify which causal paths are key drivers. Through this intervention, the advertising system can simulate different scenarios to further understand which causal paths have a greater impact on advertising effectiveness in actual applications, and then optimize advertising strategies. If a path changes significantly after the intervention, it may be a potential key driver. Explicit drivers refer to factors that directly and obviously affect user behavior, such as the creative content of the advertisement itself, the timing of the advertisement display, etc. By comparing the changes in revenue conversion rate before and after the intervention, it is possible to identify explicit drivers that have a direct impact on the advertising effect in the causal path.By identifying explicit driving factors, the advertising system can more precisely pinpoint which factors are crucial in driving changes in user behavior, and then improve the advertising display strategy. For example, the system can identify that a specific advertising content or display period will directly increase user engagement and purchase rate. Low-dimensional vector mapping is the process of transforming high-dimensional causal path weight coefficients into a more concise vector representation through dimensionality reduction techniques. This step helps the system convert complex causal relationships into a format that is easier to analyze and process. Such mapping helps extract the most influential factors from a large number of causal paths and reveals potential implicit driving factors. The low-dimensional vectors after dimensionality reduction facilitate the extraction of meaningful implicit driving factors from large-scale advertising data. Implicit driving factors are those factors that are not easily detectable but have a profound impact on advertising effects, such as users' latent interests or long-term trends. Implicit driving factors refer to those factors that are not easily directly observable or measurable and are related to factors such as users' emotional preferences, long-term purchase habits, or social influences. Through low-dimensional vector mapping, these difficult-to-intuitively-discover implicit factors can be identified. By identifying implicit driving factors, the advertising system can gain a deeper understanding of users' long-term interests and needs, thus avoiding short-term misjudgments caused by seasonal promotional activities. For example, the system may find that certain advertising placement factors (such as time, method, etc.) do not directly lead to purchase behavior, but they will subtly influence users' long-term interests, thereby improving the accuracy of advertising recommendations. The comprehensive analysis of explicit and implicit driving factors helps form a comprehensive user interest model. Ultimately, through the comprehensive evaluation of these driving factors, the advertising system can more precisely adjust the advertising placement strategy and optimize the user experience.

[0075] In one embodiment, after the step of determining the reason for not meeting the target user's interest requirements, the method further includes:

[0076] Analyze the reason for not meeting the target user's interest requirements and analyze the interest change parameters of the target user;

[0077] Re-predict the current interest tags of the target user according to the interest change parameters;

[0078] Generate the advertising data adjustment strategy based on the re-predicted interest tags and the reason for not meeting the target user's interest requirements.

[0079] As described above, to identify why the target user's interest needs are not met in the advertisement push or recommendation system, it can be achieved by comparing the user's historical behavior, advertisement feedback (such as click-through rate, purchase rate), and the predefined interest model. If the advertisement content does not match the user's interest needs, it may be due to the low matching degree of the advertisement content itself with the user's needs, or due to external factors such as the time and frequency of advertisement placement, which weaken the influence of the advertisement. The interest change parameter refers to various indicators that measure the change of user interest, such as the upward or downward trend of the user's interest in a certain category of goods or services. The change of interest can be inferred from the user's behavior data, such as browsing history, purchase records, search queries, and responses to advertisements (click, skip, ignore, etc.). The analysis of interest change not only focuses on short-term fluctuations but also considers long-term trends. Analyzing the user's interest change parameters can help the system identify the dynamic change trend of the user's current interest, rather than just the static interest preference. This is crucial for adjusting the advertisement strategy because the user's interest is constantly changing, and the advertisement system needs to continuously adjust its prediction model according to these changes to improve the relevance and effectiveness of the advertisement. The interest tag is a category tag assigned by the advertisement recommendation system according to the user's interest (for example, sports enthusiasts, technology fans, fashionistas, etc.). Re-predicting the interest tags of the target user is based on the analysis of their interest change parameters, and through machine learning models (such as collaborative filtering, content recommendation, deep learning models, etc. in the recommendation system) to update and adjust the user's interest tags in real time. This process will judge whether the user's interest has changed according to the user's recent behavior and historical trends, and re-assign the interest tags. According to the new interest tags and the identified reasons for not meeting the user's needs, the advertisement system will adjust the placement strategy of the advertisement data. These adjustments may include: optimizing the display time of the advertisement, adjusting the content of the advertisement creative, re-setting the advertisement frequency, adjusting the target audience group, etc. The core of the advertisement strategy adjustment is to ensure that the advertisement content can meet the user's needs to the greatest extent according to the user's dynamic interest tags and preferences. The purpose of generating the advertisement data adjustment strategy is to improve the relevance, attractiveness, and effectiveness of the advertisement. By combining the user's real-time interest needs with the strategy of the advertisement system, the display of the advertisement is more accurate, reducing the push of ineffective advertisements, thereby increasing the advertisement conversion rate and reducing user churn. Through this strategy adjustment, the advertisement can effectively avoid the lag response of a single push strategy to the change of user needs.

[0080] In one embodiment, the interest tags include content-level tags, form-level tags, and scenario-level tags. By comprehensively analyzing these three types of tags, the advertising recommendation system can generate a more refined user profile. For example, a certain user may be interested in "travel" content (content-level tag), prefer to receive information through "video ads" (form-level tag), and often view travel-related content during "weekend afternoons" (scenario-level tag). Based on these tags, the system can push "video ads" related to "weekend travel". By monitoring the changes of these tags in real time, the advertising system can adjust the advertising placement strategy in a timely manner. For example, if the user's interest tags change from "technology" to "health", the advertising system can quickly update the content-level tags and give priority to recommending relevant ads such as fitness and health products, rather than continuing to place irrelevant technology ads. The combination of content-level, form-level, and scenario-level tags makes advertising recommendations more personalized, not only paying attention to the user's long-term interests, but also optimizing according to the user's current situation. For example, by identifying the user's out-of-home scenario through the scenario-level tag, the system can push ads related to going out, dining, and entertainment, while when at home, it may push ads for home products or audio-visual entertainment.

[0081] Referring to Figure 3 , an apparatus for predicting user interests based on multimodal data is further provided in an embodiment of the present application, including:

[0082] An acquisition module 1, configured to acquire multimodal historical data of a target user;

[0083] An identification module 2, configured to identify influencing variables in the multimodal historical data;

[0084] A segmentation module 3, configured to segment the influencing variables from the multimodal historical data according to the identification result;

[0085] A construction module 4, configured to construct a demand analysis model based on the segmentation result;

[0086] An input module 5, configured to acquire the behavior data of the target user in real time, input the behavior data into the interest prediction model, and output a demand data set corresponding to the current behavior data of the target user through the demand analysis model;

[0087] A prediction module 6, configured to analyze the demand data set based on a time stamp based on the output result, and predict the current interest tags of the target user;

[0088] A sending module 7, configured to generate advertisement data according to the interest tags and send it to the target user.

[0089] As described above, it can be understood that each component of the user interest prediction device based on multimodal data proposed in this application can implement the functions of any of the above-mentioned user interest prediction methods based on multimodal data, and the specific structure will not be elaborated herein.

[0090] Referring Figure 4 , an embodiment of this application further provides a computer device, which may be a server, and its internal structure may be as Figure 4 shown. The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as monitoring data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a user interest prediction method based on multimodal data.

[0091] The above-mentioned processor executes the above-mentioned user interest prediction method based on multimodal data, including: obtaining multimodal historical data of a target user; identifying influencing variables in the multimodal historical data; according to the identification result, segmenting the influencing variables from the multimodal historical data; based on the segmentation result, constructing a demand analysis model; obtaining the behavior data of the target user in real time, inputting the behavior data into the interest prediction model, and outputting a demand data set corresponding to the current behavior data of the target user through the demand analysis model; based on the output result, analyzing the demand data set based on a timestamp, and predicting the current interest label of the target user; generating advertisement data according to the interest label and sending it to the target user.

[0092] An embodiment of this application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements a user interest prediction method based on multimodal data, including the steps of: obtaining multimodal historical data of a target user; identifying influencing variables in the multimodal historical data; according to the identification result, segmenting the influencing variables from the multimodal historical data; based on the segmentation result, constructing a demand analysis model; obtaining the behavior data of the target user in real time, inputting the behavior data into the interest prediction model, and outputting a demand data set corresponding to the current behavior data of the target user through the demand analysis model; based on the output result, analyzing the demand data set based on a timestamp, and predicting the current interest label of the target user; generating advertisement data according to the interest label and sending it to the target user.

[0093] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium provided in this application and used in the embodiments can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0094] It should be noted that in this document, the term "including", "comprising", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, device, article, or method including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such a process, device, article, or method. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, device, article, or method including that element.

[0095] The above are only the preferred embodiments of this application, and do not limit the patent scope of this application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of this application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of this application.

Claims

1. A user interest prediction method based on multimodal data, characterized in that: The method comprises: Obtain multimodal historical data of target users; identifying influencing variables in the multimodal historical data; According to the identification result, segmenting the influencing variables from the multimodal historical data; Based on the segmentation results, build a demand analysis model; Acquire the behavior data of the target user in real time, input the behavior data into the interest prediction model, and output the demand data set corresponding to the current behavior data of the target user through the demand analysis model; Based on the output result, the demand data set is analyzed based on the timestamp, and the current interest tag of the target user is predicted; Advertising data is generated according to the interest tag and sent to the target user.

2. The method for predicting user interests based on multimodal data according to claim 1, characterized in that: The steps of analyzing the demand data set based on the timestamp and predicting the current interest tag of the target user include: Analyzing scene features of the behavior data according to the behavior data; Based on the analysis result, predict the interest change characteristics of the scene features to the target user; Performing timestamp alignment processing on the interest change features, scene features, and behavior data, and fusing the processed interest change features, scene features, and behavior data features to obtain joint features; The joint features are input into a preset classification model, and the classification model is used to predict the interest tags of the target user based on the joint features.

3. The method for predicting user interests based on multimodal data according to claim 2, characterized in that: The step of generating advertising data according to the interest tag and sending it to the target user comprises: Acquiring the multimodal historical data; Determine the revenue conversion rate of all transmission paths according to the multimodal historical data, and calculate the weight coefficient of each transmission path according to the revenue conversion rate; Obtain scene features and advertising data of current behavior data; Analyzing conflicting paths among all transmission paths according to the scene characteristics and the advertisement data; Based on the analysis results, available transmission paths are determined, and transmission costs of the available transmission paths are calculated; According to the calculation results, the available transmission with the lowest transmission cost and the highest weight coefficient is selected as the transmission path for this time; The advertisement data is sent to the target user based on the determined transmission path.

4. The method for predicting user interests based on multimodal data according to claim 1, characterized in that: After the step of generating advertisement data according to the interest tag and sending it to the target user, the method includes: Receive behavioral feedback from the target user in real time; According to the behavior feedback of the target user, determining whether the advertisement data meets the interest needs of the target user; quantifying the behavioral feedback to obtain an interest coefficient of the target user; If the interest coefficient is lower than a threshold, it is determined that the advertisement data does not meet the interest requirements of the target user; Based on the fact that the advertisement data does not satisfy the interest needs of the target users, analyzing the causal characteristics of the advertisement data, and identifying the driving factors for the formation of the target users; Determine the reason why the interest requirements of the target user are not met according to the driving factors, and generate the advertisement data adjustment strategy according to the reason why the interest requirements of the target user are not met; The advertisement data is adjusted based on the adjustment policy.

5. The method for predicting user interests based on multimodal data according to claim 4, characterized in that: The method of analyzing the causal characteristics of the advertising data and identifying the driving factors for the formation of the target users comprises: extracting behavioral data, influencing variables and advertising data from the multimodal historical data; Based on the extracted results, a hierarchical causal graph is constructed to determine all causal paths of multimodal historical data; Obtaining the revenue conversion rates of all causal paths, and calculating the weight coefficient of each causal path according to the revenue conversion rates; Each causal path was tested by adding a virtual intervention and the benefit conversion rate after the intervention was calculated; Identify the explicit driving factors of the causal path based on the benefit conversion rate before and after the intervention; Performing a virtual intervention test on each causal path, mapping the weight coefficient into a low-dimensional vector, and identifying the implicit driving factor of the causal path according to the mapping result; Based on the explicit driving factors and the implicit driving factors, the driving factors are determined.

6. The method for predicting user interests based on multimodal data according to claim 4, characterized in that: After the step of determining the reason why the target user's interest requirements are not met, the method further includes: Analyze the reasons why the interest requirements of the target user are not met, and analyze the interest change parameters of the target user; Re-predicting the current interest tag of the target user according to the interest change parameter; The advertisement data adjustment strategy is generated based on the re-predicted interest tags and the reasons why the interest requirements of the target user are not met.

7. The user interest prediction method based on multimodal data according to claim 1, characterized in that: The interest tags include content-level tags, form-level tags and scene-level tags.

8. A user interest prediction device based on multimodal data, characterized in that: include: An acquisition module is used to acquire multimodal historical data of target users; An identification module, used to identify influencing variables in the multimodal historical data; A segmentation module, used for segmenting the influencing variables from the multimodal historical data according to the identification result; A construction module is used to construct a demand analysis model based on the segmentation results; An input module, used to obtain the behavior data of the target user in real time, input the behavior data into the interest prediction model, and output a demand data set corresponding to the current behavior data of the target user through a demand analysis model; A prediction module, configured to analyze the demand data set based on the output result and the timestamp, and predict the current interest tag of the target user; A sending module is used to generate advertising data according to the interest tag and send it to the target user.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images