Enterprise marketing intelligent recommendation method and system based on deep learning

Abstract

The invention provides an enterprise marketing intelligent recommendation method and system based on deep learning, and the method comprises the steps: constructing an interest heat map according to the historical behavior data of a user, and predicting the future interest trend of the user; obtaining a state space in combination with the interest heat map, the interest trend tag and an external environment signal; generating a candidate recommended action set for each user according to the state space; inputting the state space into a pre-trained strategy network based on reinforcement learning to evaluate the strategy value of each candidate action in the candidate recommendation action set in the current state, and selecting a recommendation action with the optimal long-term return; and real execution of the recommendation behavior is completed for the recommendation action of the optimal long-term return, and after the response of the user occurs, the feedback behavior of the user is collected so as to carry out system optimization. The method has good adaptability and popularization value in complex marketing tasks with sparse user behaviors or obvious interest drifting.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of enterprise marketing intelligent recommendation, and particularly relates to an enterprise marketing intelligent recommendation method and system based on deep learning. BACKGROUND

[0002] With the continuous evolution of digital marketing means, enterprises need more intelligent and accurate recommendation systems to improve the conversion effect of marketing activities in the face of the increasingly complex user behavior and the diversification of media channels. Traditional recommendation systems are mostly based on collaborative filtering, content tag matching, matrix decomposition or click rate prediction models based on shallow neural networks. These methods mainly focus on the static goal of "what content to recommend", and model the similarity between users and content to predict the products or marketing materials that users may be interested in. Although this method has achieved certain results in general e-commerce or information recommendation scenarios, it still faces many challenges in enterprise-level marketing scenarios.

[0003] Firstly, enterprise marketing emphasizes conversion orientation rather than click orientation. The recommendation system not only needs to recommend "content", but also must make intelligent decisions on "when to recommend" and "through what channel to recommend", because inappropriate or improper media touch methods may lead to user aversion and even cancellation of attention. However, existing recommendation technologies almost stay in the single-dimensional perspective of "content recommendation", lacking the ability to jointly model the recommendation time point, channel response, and user acceptance preference. Secondly, user interest has obvious time sequence dynamics. The conventional model only uses short-term behavior characteristics for static matching, which is difficult to capture the evolution path of user preference and cannot make early prediction before interest migration or fluctuation. Thirdly, enterprises often face serious data sparsity and delayed feedback problems in actual marketing: low user behavior frequency, and lag of conversion events, which makes it difficult for traditional supervised learning recommendation framework to learn effective causal relationships. In addition, the repetition and frequent push of recommended content in multi-channel marketing scenarios can also cause "user fatigue", reducing the overall marketing effect, and existing systems generally lack the perception and control mechanism of marketing "aversion" or "interference cost".

[0004] In summary, the existing marketing recommendation technology still stays in the static recommendation stage with goods or content as the core, lacking dynamic interest modeling, strategy time point optimization, multi-channel distribution control, and marketing feedback closed-loop adjustment mechanism. There is a significant gap between this reality and the actual needs of enterprise marketing. Therefore, there is an urgent need for a new type of recommendation system based on user behavior time sequence characteristics, interest trend evolution, multi-channel response difference, and feedback optimization capability, which can achieve accurate, efficient, and controllable marketing recommendation behavior in complex scenarios. SUMMARY

[0005] The purpose of the present application is to propose a deep learning-based enterprise marketing intelligent recommendation method and system, which automatically identifies the current interest state and its change trend of the user behavior trajectory, thereby predicting the most appropriate recommendation opportunity, and dynamically matching the optimal recommendation method based on the response characteristics of different channels, solving the single problem of traditional recommendation systems that only focus on recommended content.

[0006] To achieve the above purpose, in the first aspect of the present application, a deep learning-based enterprise marketing intelligent recommendation method is provided, which comprises the following steps:

[0007] An interest heat map is constructed according to the historical behavior data of the user, and the future interest trend of the user is predicted, and an interest trend label is generated for each user based on the interest trend; wherein the interest heat map is obtained based on the weighting of the historical behavior data of the user; the future interest trend of the user is obtained by predicting the interest heat map using a weighted sliding window method;

[0008] The interest heat map and the interest trend label are combined with external environmental signals, and trend regularization is performed to obtain a state space; the trend regularization is obtained by smoothing regularization of the predicted future interest trend of the user;

[0009] According to the state space, a candidate recommendation action set containing recommended content, push channel and recommended time point is generated for each user, and the push time point is optimized by combining the interest heat map, the interest trend label and the push channel, so as to select the most appropriate recommendation opportunity for each user;

[0010] The state space is input into a pre-trained reinforcement learning-based policy network to evaluate the policy value of each candidate action in the candidate recommendation action set under the current state, and the recommendation action with the optimal long-term return is selected; wherein the pre-trained reinforcement learning-based policy network combines the immediate positive feedback of whether the user clicks or converts the recommendation action, measures the value of the recommendation action in improving user retention or brand preference in the long term, and generates the policy value of each candidate action under the current state according to the push frequency cost;

[0011] The real execution of the recommended behavior of the recommendation action with the optimal long-term return is completed, and the user feedback behavior is collected after the user response occurs, which is used to train the reinforcement learning-based policy network for system optimization.

[0012] Further, the historical behavior data includes click actions, browsing actions and purchase actions.

[0013] Further, the interest trend label specifically includes:

[0014] If the interest trend is higher than the current interest, mark as rising;

[0015] If the interest trend is lower than the current interest, mark as falling;

[0016] If the interest trend is equal to the current interest, mark as stable.

[0017] Further, the external environment signal includes time information, holidays and promotion activities;

[0018] The time information is used to reflect the timing of user behavior;

[0019] The holidays are represented by binary coding to indicate whether the user behavior occurs during holidays or not;

[0020] The promotion activities are used to indicate whether there are promotion activities in a certain period of time or not;

[0021] The external environment signal maps different external signals into vectors through an embedding layer, and is fused into the state space with the interest heat map and the interest trend label.

[0022] Further, the combination of the interest heat map, the interest trend label and the push channel optimizes the push time point, and selects the most suitable recommendation opportunity for each user, represented as:

[0023]

[0024] Where, t k is the recommended time point, and the time point is selected to optimize the recommendation effect; α k and β k are weight factors in the time point optimization, used to balance the user activity and the influence of seasonality; is the time pool.

[0025] Further, the selected recommendation action set is also provided with a diversity loss function, which is used to avoid too many repeated recommendations in the candidate recommendation set, and the diversity loss function is represented as:

[0026]

[0027] Where, is the diversity loss function, which is used to encourage the diversity of the recommendation action. represents the similarity between the recommended content and ; represents that the push channel is not repeated with ; represents that the recommended time point with no repetition;

[0028] wherein the similarity is reduced to avoid recommending repetitive content.

[0029] Further, the instant positive feedback is based on historical click / conversion rate; the value of the recommended action in improving user retention or brand preference in the long term is approximated by the number of active times or repeat purchase behavior of the user in the next 7 days; and the push frequency cost is accumulated by tracking the number of times the user receives in the last 7 days.

[0030] Further, the step of completing the recommended behavior of the recommended action with the optimal long-term return and collecting user feedback behavior after user response occurs for training the reinforcement learning-based policy network for system optimization comprises:

[0031] receiving the recommended action with the optimal long-term return, including recommended content r * , push channel c * , and recommended time point t * ;

[0032] preparing the recommended content r * as a push template; selecting a push channel c * ; performing a recommendation task at a recommended time point t * or its tolerance range; and synchronously recording metadata;

[0033] All relevant behaviors of the user after receiving the recommended push will be collected and structured as a feedback signal group, including whether to click the recommended content, whether to complete the key conversion, whether to make any response, whether to actively close or shield or report the push, and the time difference between the sending of the push to the user and the first response;

[0034] All feedback information of the executed recommended behavior is sorted into a triple, including the current user state vector, the recommended action with the optimal long-term return, and the feedback signal.

[0035] Based on the triple, the system is optimized.

[0036] Further, the recommended content includes a certain product or activity; and the push channel includes App push and SMS.

[0037] In a second aspect of the present application, a deep learning-based enterprise marketing intelligent recommendation system is provided, which comprises:

[0038] a user heat analysis module configured to construct an interest heat map based on historical behavior data of the user, predict a future interest trend of the user, and generate an interest trend label for each user based on the interest trend, wherein the interest heat map is obtained by weighting the historical behavior data of the user, and the future interest trend of the user is predicted based on the interest heat map using a weighted sliding window method;

[0039] a state space construction module configured to combine the interest heat map and the interest trend label, and an external environment signal, and perform trend regularization to obtain a state space, wherein the trend regularization is obtained by smoothing and regularizing the predicted future interest trend of the user;

[0040] a selected recommendation action generation module configured to generate a candidate recommendation action set for each user based on the state space, including recommended content, a push channel, and a recommended time point, and optimize the recommended time point by combining the interest heat map, the interest trend label, and the push channel, to select the most suitable recommendation time for each user;

[0041] an optimal recommendation action selection generation module configured to input the state space into a pre-trained reinforcement learning-based policy network to evaluate the policy value of each candidate action in the candidate recommendation action set in the current state, and select a recommendation action with an optimal long-term return, wherein the pre-trained reinforcement learning-based policy network combines immediate positive feedback after the user clicks or converts the recommendation action, measures the value of the recommendation action in improving user retention or brand preference in the long term, and generates the policy value of each candidate action in the current state based on the push frequency cost;

[0042] a marketing feedback and recommendation execution module configured to complete real execution of the recommendation behavior of the recommendation action with the optimal long-term return, and collect user feedback behavior after the user response occurs, to train the reinforcement learning-based policy network for system optimization.

[0043] The present application has at least the following beneficial technical effects:

[0044] The application provides a deep learning intelligent recommendation method with user dynamic interest recognition, marketing content and push channel joint recommendation decision, intelligent selection of recommendation time point and optimization of marketing feedback response capability. The method can automatically recognize the interest state and its change trend of the user based on the user behavior track, so as to predict the most suitable recommendation time, and dynamically match the optimal recommendation mode based on the response characteristics of different channels, solve the single problem of traditional recommendation system only focusing on the recommended content. On this basis, the application further introduces a multi-dimensional feedback modeling mechanism, the system can adjust and update the strategy according to the user's conversion behavior, adverse signal and marketing execution cost, so as to realize the accurate, efficient and robust enterprise-level intelligent recommendation capability.

[0045] The application builds a complete user behavior modeling system and recommendation decision optimization logic, so that the system can consider the three targets of recommendation relevance, acceptance probability and conversion effect, and realize the change from static recommendation to strategic recommendation. The system is not only suitable for high-frequency user behavior data scene, but also has good adaptability and popularization value in complex marketing tasks with sparse user behavior or obvious interest drift, and has significant technical progress and practical application significance. BRIEF DESCRIPTION OF DRAWINGS

[0046] The application is further described by using the drawings, but the embodiments in the drawings do not constitute any limitation on the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the following drawings.

[0047] Figure 1 The application discloses an enterprise marketing intelligent recommendation method based on deep learning

[0048] Figure 2 The application discloses an enterprise marketing intelligent recommendation system framework based on deep learning. DETAILED DESCRIPTION

[0049] The embodiments of the application are described in detail below, and the examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the application, and cannot be understood as a limitation on the application.

[0050] Embodiment one

[0051] As shown in Figure 1 The application discloses an enterprise marketing intelligent recommendation method based on deep learning, which comprises:

[0052] S1. Construct an interest heat map based on the user's historical behavior data, predict the user's future interest trends, and generate an interest trend label for each user based on the interest trends; wherein the interest heat map is obtained by weighting the user's historical behavior data; the user's future interest trends are predicted based on the interest heat map using a weighted sliding window method.

[0053] Specifically, the goal of this step is to construct a dynamic interest heat map based on the user's historical behavior data and predict future interest trends. Through this process, the present invention can capture changes in user interests and provide data support for subsequent recommendation system decisions. Ultimately, by generating an interest heat map and performing trend prediction, the present invention can accurately predict changes in user interests, providing a basis for subsequent recommendation decisions.

[0054] Furthermore, the construction of interest heat map:

[0055] The interest heat map is used to show the user's interest in different products, services or topics at different time periods. To generate this heat map, the present invention needs to perform weighted processing based on the user's historical behavior to reflect the different effects of different behaviors on the intensity of interest.

[0056] Among them, the interest heat map T i,j Indicates the user's interest intensity in product or topic i at time point j.

[0057] Furthermore, when calculating interest intensity, the present invention weights the user's behavior data. Assuming that a user has multiple behaviors (such as clicks, browsing, purchases, etc.) within a certain period of time, the interest intensity can be calculated as follows:

[0058]

[0059] Among them, T i,j : The user's interest in product or topic i at time point j.

[0060] w k : Represents the weighting coefficients of different behavior types (such as click, browse, purchase, etc.). These weighting coefficients reflect the contribution of different behaviors to user interests.

[0061] h i,j,k : Represents the quantitative value of the kth behavior of the user on product i at time point j (such as the number of clicks, browsing time, etc.).

[0062] The purpose of this formula is to quantify the impact of different user behaviors on interest by using a weighted approach. For example, purchasing behavior might be given a higher weight, while click behavior might be given a medium weight, and browsing behavior a lower weight.

[0063] Further, interest trend prediction:

[0064] After the construction of the interest heat map T i,j After the construction of the interest heat map T

[0065] The goal of the present application is to predict the interest intensity of the user on product i at the next time point j+1

[0066]

[0067] wherein, represents the predicted interest heat of the user on product i at time point j+1.

[0068] T i,j-k represents the interest heat of the user on product i in the past k time points.

[0069] α k : weighting coefficient, representing the influence degree of past interest data on future interest prediction.

[0070] wherein, the purpose of the formula is to predict the future interest value according to the historical interest data and the weighting coefficient (α k ), to ensure that the interest trend can consider the recent changes in user behavior. For example, if the user's interest in a product has been rising in the recent period, the system can predict that the interest may continue to rise in the future period.

[0071] Further, interest trend label generation:

[0072] Based on the interest heat prediction result, the present application generates an interest trend label τ u for each user, marking the change trend of the current interest of the user. The generation of the interest trend label is based on the following rules:

[0073] If the future interest is higher than the current interest T i,j , it is marked as rising.

[0074] If the future interest is lower than the current interest T i,j , it is marked as falling.

[0075] If they are equal or the change is not large, it is marked as stable.

[0076] This label will help the subsequent steps to determine which stage of interest the user is currently in, thereby providing a basis for the decision-making of the recommendation system.

[0077] Through this step, the present application uses the user's historical behavior data and interest prediction model to construct a dynamic interest heat map and predict the user's interest trend. This not only provides real-time interest data support for the recommendation system, but also helps the system to make more accurate push and strategy adjustment in the future according to the changing trend of user interest.

[0078] S2, combine the interest heat map and interest trend label, and external environment signal, and perform trend regularization to obtain a state space; the trend regularization is obtained by smoothing regularization on the predicted future interest trend of the user.

[0079] Specifically, the core goal of this step is to combine the interest heat map T and interest trend label τ u generated in step 1 with external environment signals to construct a state space s t that can be used for reinforcement learning model, and input these information into the subsequent recommendation strategy generation. This step combines the changing trend of dynamic interest and the influence of external environment on user behavior, thereby providing more comprehensive and accurate input for the recommendation system.

[0080] Further, in step 1, the present application constructs the interest heat map T and predicts the changing trend of user interest. Based on these data, the key task of this step is to construct a comprehensive state space that integrates interest heat, trend label and external environment signal as the input of subsequent reinforcement learning.

[0081] The present application can combine user interest, interest change trend and external environment signal into a high-dimensional state vector s t in the following way:

[0082] s t =f(T,τ u ,e t ) (3)

[0083] Where s t : comprehensive state vector, represents the comprehensive information of user's interest state and external environment factors at time point t.

[0084] T: user's interest heat map for a product or topic at time point t.

[0085] τ u : interest trend label, describes the changing trend of user interest (such as rising or falling).

[0086] e tExternal environment signals, including timestamp, holiday, promotion, etc.

[0087] f: a mapping function that fuses the interest heat map, interest trend label and external environment signals into a state vector.

[0088] It can be understood that in this way, the application not only reflects the dynamic changes of user interest, but also combines the influence of external factors on interest, thereby constructing a complete state space.

[0089] where the external environment signal e t is an important innovation in this step. External factors (such as time information, holidays, promotions, etc.) will significantly affect the user's interest pattern, so these factors need to be encoded into information suitable for model processing. In order to achieve this, the application encodes the external environment signal as follows:

[0090] Time information: The timestamp t reflects the time sequence of user behavior. For example, the interest pattern of holidays or weekdays is usually different, so the application can use time series encoding (such as periodic features or time of day) to represent the influence of time.

[0091] Holiday signal: Special events such as holidays can affect user behavior and interest. The application uses binary encoding to represent whether the user behavior occurs on a holiday. For example, if the current date is a holiday, the value of this item is 1, and the value is 0 when it is not a holiday.

[0092] Promotion: Promotions can cause dramatic fluctuations in user interest. The application can introduce a promotion flag to indicate whether there is a promotion in a certain time period.

[0093] These environment signals are mapped into vectors through embedding layers (such as word embedding technology or simple numerical encoding), and are fused into the state space s t along with the interest heat map and trend label.

[0094] Further, in order to further enhance the generalization ability of the state space in the reinforcement learning process, the application introduces an innovative regularization mechanism. Traditional reinforcement learning methods often rely on static state space modeling, but in actual applications, the user's interest changes often have strong dynamics. Therefore, the application needs a method to deal with the overfitting problem in the state space, especially when there is noisy data. For this purpose, the application proposes an interest trend regularization term to control the dynamic changes in the state space and avoid excessive reliance on past behavior data.

[0095] The application can introduce a trend regularization term to ensure that the state vector s tThe interest trend in the state space s

[0096]

[0097] The regularization term of interest trend constrains the smoothness of interest change.

[0098] The predicted interest heat value represents the interest in product or topic i at time point t.

[0099] λ: Regularization strength hyperparameter, controls the strength of smoothness constraint.

[0100] The role of this regularization term is to force the model to consider the smooth change of interest in the state space, avoiding the model's excessive dependence on short-term behavioral fluctuations, thereby improving the system's generalization ability.

[0101] By constructing a rich state space s t This step provides multi-dimensional input data for the subsequent reinforcement learning model, including not only the historical change of user interest, but also the influence of external environmental factors.

[0102] S3, according to the state space, generate a set of candidate recommendation actions for each user, including recommended content, push channel and recommended time point, and optimize the push time point by combining the interest heat map, interest trend label and push channel, to select the most suitable recommendation timing for each user.

[0103] Specifically, the core goal of this step is to generate a set of candidate recommendation actions based on the state space s t generated in step S2 , which will include recommended content, recommended push channel and push time point. The generation of the candidate recommendation action set not only considers the state change of user interest, but also integrates external environmental factors (such as holidays, promotional activities, etc.), thereby providing a diverse selection space for subsequent reinforcement learning strategy optimization. In practical applications, these candidate recommendation actions will be used to select the optimal recommendation strategy to achieve a personalized and dynamic recommendation system.

[0104] In step 2, the invention constructs a state space s t which contains information such as user interest change and external environmental impact. This state space will serve as the basis for generating recommendation actions. In this step, the invention combines this information with the optimization of recommended content, channel and time point to create a multi-dimensional set of candidate recommendation actions A tAnd ensure that the generated recommended actions can fully cover user needs, while improving the timeliness and accuracy of recommendations.

[0105] Further, the candidate recommended action set is generated:

[0106] Based on the state space s of step S2 t The application generates a candidate recommended action set A for each user t Each recommended action includes the following three elements:

[0107] Recommended content r i : Products, services or activities that users are interested in. According to the user's interest heat map T and trend label τ u The recommendation system selects content that matches the user's interest from the content pool .

[0108] Recommended channel c j : Choose the most suitable push channel. The push channel can be App push, SMS, email, etc., and the most suitable channel is selected according to the user's historical behavior and preferences.

[0109] Recommended time point t k : The time point of the push. The system can analyze the user's active period to choose the right time to push the recommended content.

[0110] Candidate recommended action set A t Can be generated by the following formula:

[0111]

[0112] Where A t : Recommended action candidate set, containing a collection of multiple recommended actions.

[0113] r i : Recommended content, selected from the content pool .

[0114] c j : Push channel, selected from the channel pool .

[0115] t k : Recommended time point, selected from the time pool .

[0116] In this way, the application generates multiple candidate recommended actions for users, each of which not only reflects the changes in user interest, but also optimizes the actual effect of the recommendation according to the push channel and time point.

[0117] Further, the innovation of this step is the multi-dimensional recommendation action generation: the generated recommendation action contains multi-dimensional information of content, channel and time point, so that the recommendation is not only based on single interest data, but also considers user behavior, environmental signals and timeliness, so as to realize truly personalized recommendation. In addition, dynamic timeliness: by selecting the appropriate push time point t k for each recommendation action, the system can ensure that the recommendation achieves the maximum effect within the user's active period, avoiding missing the best recommendation opportunity.

[0118] Further, the optimization of content, channel and time point:

[0119] In generating candidate recommendation actions, the present application needs to further optimize the content, channel and time point of each recommendation action to ensure that it meets the user's needs to the greatest extent.

[0120] Content optimization: according to the interest heat map T and trend label τ u generated by the user in step 1, the present application preferentially selects those contents with high or rising user interest values. Content selection not only depends on historical behavior, but also combines trend prediction of user interest. In this way, the recommended content will be consistent with the user's future interest changes, avoiding recommending outdated or irrelevant content.

[0121] Channel adaptability: the selection of push channel c j is based on the user's historical interaction data. By analyzing the user's response on different channels (such as the conversion rate of push through the App is higher than that of SMS push), the present application can select the most suitable push channel for each user. In particular, during certain special periods (such as holidays, promotional activities), the system will preferentially select the channel with the best effect.

[0122] Time point optimization: time point selection is completed by analyzing the user's active time period. Based on the user's activity patterns at different time points (such as the difference in activity level in the morning and evening), the present application can select the most suitable recommendation time for each user. The present application optimizes the recommendation time point in the following way:

[0123]

[0124] Where, t k : recommended time point, select the time point that optimizes the recommendation effect. α k and β k : weight factors in time point optimization, used to balance the influence of user activity and seasonality.

[0125] This formula selects a time point t that can best improve the conversion rate by combining user activity and seasonal factors.k , thereby ensuring the timeliness and accuracy of recommendations.

[0126] The innovation in this area lies in optimizing timing: by incorporating activity and seasonality factors, we optimize the timeliness of recommendations, ensuring they arrive during the user's most active hours, thereby increasing conversion rates. Furthermore, we personalize the matching of channels and content: based on historical user behavior and responses, the system selects the most appropriate recommendation channel for each user, while prioritizing content that the user is likely to be interested in, improving the accuracy of recommendations.

[0127] Furthermore, the diversity and exploration of candidate recommended actions:

[0128] To prevent the recommendation system from becoming overly rigid about a specific type of content or push method, this paper introduces a diversity constraint when generating candidate recommended action sets. This constraint aims to encourage exploration, prevent the recommendation system from falling into information cocooning or recommendation fatigue, and ensure that users receive a more diverse range of recommendations.

[0129] Specifically, the present invention can introduce a diversity loss function to avoid excessive duplication of recommendations in the candidate recommendation set. For example, the present invention aims to ensure that in the candidate recommendation action set, different push channels, content types, and recommendation timings can fully cover user needs without excessive overlap in a certain dimension:

[0130]

[0131] in, Diversity loss function is used to encourage the diversity of recommended actions.

[0132] sim(r i ,r j ):Indicates recommended content i and r j By reducing the similarity, the system can avoid recommending duplicate content.

[0133] Make sure to push channel c i with c j No repetition.

[0134] Ensure the recommended time t i With t j No repetition.

[0135] It can be understood that through this loss function, the system can guide diverse exploration when generating candidate recommended actions, ensuring the multi-dimensional coverage of the recommendation set, thereby avoiding recommendation fatigue caused by a single recommendation model.

[0136] The innovation of this block is added later, including diversity constraints: by introducing a diversity loss function, the invention not only optimizes the relevance of recommended content, but also ensures the diversity of recommendations, avoiding the system from falling into the situation of over-fitting recommendations. And encourage exploration and innovation: through appropriate exploration mechanisms, the system can ensure the accuracy of recommendations while providing more potential content that meets user needs, avoiding users from falling into a single information situation.

[0137] By generating a multi-dimensional candidate recommendation action set A t , this step provides a rich and personalized selection space for subsequent recommendation strategy optimization. The system combines user interest, external environmental signals, push time points, and diversity constraints to ensure the accuracy, timeliness, and diversity of recommended content. Ultimately, the recommendation system can make optimal recommendations based on multi-dimensional data, thereby improving the conversion rate and user satisfaction of recommendations.

[0138] S4, input the state space into a pre-trained reinforcement learning-based policy network to evaluate the policy value of each candidate action in the candidate recommendation action set under the current state, and select the recommended action with the optimal long-term return; wherein the pre-trained reinforcement learning-based policy network combines immediate positive feedback after the user clicks or converts the recommended action, measures the value of the recommended action in improving user retention or brand preference in the long term, and generates the policy value of each candidate action under the current state.

[0139] Specifically, the goal of this step is to evaluate the potential value of each candidate action under the current state based on the state vector s t generated in step S2 and the candidate recommendation action set A t constructed in step 3, and to select the recommended action with the optimal long-term return The policy network is responsible for modeling the optimal mapping of user state to recommended behavior, and is the core of the personalized and dynamic recommendation strategy in the entire patent solution. Its task is to learn rather than execute or feedback, which will be handled by subsequent steps.

[0140] Further, unlike traditional recommendations that only maximize click-through rate, this step designs a multi-dimensional target in the modeling process and customizes modeling for key factors in the enterprise marketing recommendation scenario, so that the policy network can not only output recommended actions that users are interested in, but also consider the dynamic balance of marketing resources and frequency.

[0141] Further, the input and output of the policy network are defined as follows:

[0142] The policy network takes the current user state vector s t as input, and outputs from the candidate recommendation action set At each action a t = <r i , c j , t k > of the value estimation Q(s t , a t ) and select an optimal action

[0143] State vector s t contains:

[0144] High-frequency interest features from the interest heat map T;

[0145] Interest evolution direction from the trend label τ u ;

[0146] Push sensitivity from the environment signal e t (e.g. whether it is a holiday, whether it is a promotion);

[0147] User's current push history (for frequency control).

[0148] Candidate recommendation action set A t is a set composed of multiple triplets, each action a t = <r i , c j , t k > represents recommended content, push channel, and recommended time point respectively.

[0149] Further, the value function is modeled:

[0150] To train the strategy network, the present application defines a strategy value function Q(s t , a t ) to measure the long-term expected return of action a t in the current state. This function not only considers direct benefits such as click conversion, but also integrates user acceptance and marketing product control and other system cost items, so as to realize more practical demand-oriented strategy learning:

[0151]

[0152] wherein, represents the immediate positive feedback of whether the user clicks or converts the recommended action, which can be modeled according to the historical click / conversion rate.

[0153] The value of the recommended action in promoting user retention or brand preference in the long term can be approximately represented by the number of active times or repeat purchase behavior of the user within the next 7 days.

[0154] Push frequency cost, prevent recommendations too frequent, the system can track the user received the number of cumulative modeling within 7 days.

[0155] λ1, λ2: control the weight of long-term return and frequency penalty hyperparameters.

[0156] This function enables the strategy network to identify actions that convert well in the short term but harm user relationships in the long term, thus tending to select recommendations that are optimal for users in the long term.

[0157] Optimal action selection logic:

[0158] After completing action evaluation, the system determines the final strategy output by finding the action with the highest value in the current state among all candidate actions:

[0159]

[0160] where, s t represents the optimal recommendation action in the current state, including recommended content r * , channel c * , and time point t * .

[0161] The strategy does not directly execute the action, but rather passes the decision to the next step to be implemented in combination with the feedback mechanism.

[0162] This step is the core layer of strategy modeling in the patent, which completes the task of converting complex user states into decidable recommendation behaviors through reinforcement learning, and is the key logical bridge connecting the front-end state modeling and the back-end recommendation execution. The strategy learned by it not only considers user behavior data, but also includes system resources and market dynamic factors, making the recommendation system have real-time intelligence, strategy robustness and business controllability, and highly match the actual needs of enterprise intelligent marketing scenarios.

[0163] S5, the real execution of the optimal long-term return recommendation action completes the recommendation behavior, and after the user response occurs, the user feedback behavior is collected for training the strategy network based on reinforcement learning to optimize the system.

[0164] Specifically, the goal of this step is to complete the real execution of the optimal recommendation action selected by the strategy network in step S4, and after the user response occurs, collect its feedback behavior signal to build structured feedback data for subsequent strategy training. This step is the interface layer connecting intelligent decision-making and real user interaction in the entire system, which not only bears the responsibility of scheduling and executing the recommendation action, but also is responsible for the conversion of behavior signals to trainable data, and is the closed-loop establishment node in the patent solution.

[0165] Further, the recommended action execution mechanism:

[0166] The recommended system receives the optimal recommended action output in step 4 After that, call the actual push scheduling module in the marketing system to perform the following operations:

[0167] The recommended content r * is prepared as a push template (such as a certain product or activity);

[0168] Select the push channel c * (such as App push, SMS, WeChat service notification, etc.);

[0169] At the recommended time point t * or its tolerance range, execute the recommended task;

[0170] Synchronize the push log, touch situation, device response code and other metadata for subsequent behavior mapping.

[0171] It can be understood that the key of this stage is not modeling, but accurately, efficiently and unbiasedly executing the selected optimal recommended strategy, and ensuring that all behavior events have traces in the system.

[0172] Further, user behavior feedback collection and structured representation:

[0173] All related behaviors of the user after receiving the recommended push will be collected and structured as a feedback signal group b t , including but not limited to the following dimensions:

[0174] Whether to click the recommended content (Boolean type);

[0175] Whether to complete the key conversion (such as purchase, reservation, etc.);

[0176] Whether to make no response;

[0177] Whether to actively close, shield or report the push;

[0178] The time difference between the push sent to the user and the first response.

[0179] All signals will be standardized as structured behavior labels for subsequent reward function design, but this step does not directly calculate the reward function itself.

[0180] Further, the feedback data organization and the connection of the strategy network:

[0181] All feedback information of the executed recommended behavior will be written into the behavior log table or experience record pool for policy training in the following triple format:

[0182]

[0183] Wherein, s t : user state, from step S2;

[0184] The recommended action actually executed in this step, from step S4;

[0185] b t : the set of user feedback signals collected in this step.

[0186] In the subsequent policy training process, another step (not the responsibility of this step) will read the triple, apply the reward function to b t , and generate a numerical training signal. But the goal of this step is to ensure the completeness, accuracy and structure of the feedback signal, and to provide high-quality input for training.

[0187] This step revolves around the two core tasks of execution of the optimal recommended action and collection of behavior feedback, ensuring that the recommended results can be implemented.

[0188] Embodiment two

[0189] As shown in Figure 2 , the embodiment of the application also provides an enterprise marketing intelligent recommendation system based on deep learning, which comprises:

[0190] A user heat analysis module 101 is configured to construct an interest heat map according to historical behavior data of a user, predict a future interest trend of the user, and generate an interest trend label for each user based on the interest trend; wherein the interest heat map is obtained by weighting the historical behavior data of the user; and the future interest trend of the user is obtained by predicting the future interest trend of the user based on the interest heat map using a weighted sliding window method.

[0191] A state space construction module 102 is configured to combine the interest heat map and the interest trend label, and an external environment signal, and perform trend regularization to obtain a state space; the trend regularization is obtained by smoothing and regularizing the predicted future interest trend of the user.

[0192] A recommended action generation module 103 is configured to generate a set of candidate recommended actions for each user based on the state space, including recommended content, a push channel, and a recommended time point, and optimize the recommended time point by combining the interest heat map, the interest trend label, and the push channel, to select the most suitable recommended time for each user.

[0193] The optimal recommended action selection generation module 104 is configured to input the state space into a pre-trained reinforcement learning-based policy network to evaluate a policy value of each candidate action in the set of candidate recommended actions under the current state, and select a recommended action with an optimal long-term return; wherein the pre-trained reinforcement learning-based policy network generates the policy value of each candidate action under the current state in combination with immediate positive feedback after the user clicks or converts the recommended action, value of the recommended action in improving user retention or brand preference in the long term, and a push frequency cost.

[0194] The marketing feedback and recommended execution module 105 is configured to complete real execution of the recommended behavior of the recommended action with the optimal long-term return, and collect user feedback behavior after user response occurs, to train the reinforcement learning-based policy network for system optimization.

[0195] The above describes specific embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from the order in the embodiments and still achieve the desired result. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order shown or in consecutive order to achieve the desired results. In certain implementations, multi-task processing and parallel processing are possible or can be advantageous.

[0196] The system, device, module or unit described in the above embodiments can be specifically implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer. Specifically, the computer may, for example, be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0197] For the sake of description, the above device is described in various units by function. Of course, the functions of each unit can be implemented in the same or more software and / or hardware when implementing the present specification.

[0198] Those skilled in the art will appreciate that the embodiments of the present specification can be provided as a method, a system, or a computer program product. Therefore, the embodiments of the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0199] The specification is presented with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flow or multiple flows and / or blocks Figure 1 one or more flow or multiple flows and / or blocks

[0200] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flow or multiple flows and / or blocks Figure 1 one or more flow or multiple flows and / or blocks

[0201] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flow or multiple flows and / or blocks Figure 1 one or more flow or multiple flows and / or blocks

[0202] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0203] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory, etc. The memory is an example of computer readable media.

[0204] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0205] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to encompass non-exclusive inclusion, such that processes, methods, articles or devices that comprise a list of elements not only include those elements, but also include other elements not expressly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0206] The specification can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The specification can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0207] Each embodiment in the specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0208] Finally, it should be noted that: the embodiment of the application discloses the enterprise marketing intelligent recommendation platform based on deep learning disclosed only for the preferred embodiment of the application, only for the description of the technical scheme of the application, not for its limitation; although the application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand; the technical scheme recorded in the foregoing embodiments can be modified, or some technical features can be replaced; and the modification or replacement does not make the essence of the corresponding technical scheme deviate from the spirit and scope of the technical scheme of the embodiments of the application.

Claims

1. An enterprise marketing intelligent recommendation method based on deep learning, characterized by: The method comprises the following steps: An interest heat map is constructed based on the user's historical behavior data, and the user's future interest trends are predicted, and an interest trend label is generated for each user based on the interest trends; wherein the interest heat map is obtained by weighting the user's historical behavior data; and the user's future interest trends are predicted based on the interest heat map using a weighted sliding window method; Combining the interest heat map and interest trend labels, as well as external environmental signals, and performing trend regularization to obtain a state space; the trend regularization is obtained by performing smoothing regularization on the predicted future interest trend of the user; Based on the state space, a set of candidate recommended actions is generated for each user, including recommended content, push channel, and recommendation time. The push time is optimized based on the interest heat map, interest trend tags, and push channel to select the most appropriate recommendation time for each user. Inputting the state space into a pre-trained reinforcement learning-based policy network to evaluate the policy value of each candidate action in the candidate recommended action set in the current state and select the recommended action with the best long-term return; wherein the pre-trained reinforcement learning-based policy network combines the immediate positive feedback after whether the user clicks or converts the recommended action, the value of the recommended action in improving user retention or brand preference in the long term, and the push frequency cost to generate the policy value of each candidate action in the current state; The recommended action with the optimal long-term reward is used to complete the actual execution of the recommended behavior, and after the user responds, the user feedback behavior is collected to train the reinforcement learning-based policy network for system optimization.

2. The enterprise marketing intelligent recommendation method based on deep learning according to claim 1 is characterized in that: The historical behavior data includes click actions, browsing actions and purchase actions.

3. The enterprise marketing intelligent recommendation method based on deep learning according to claim 1 is characterized in that: The interest trend tags specifically include: If the interest trend is higher than the current interest, it is marked as rising; If the interest trend is lower than the current interest, it is marked as declining; If the interest trend is equal to the current interest, it is marked as stable.

4. The enterprise marketing intelligent recommendation method based on deep learning according to claim 1 is characterized in that: The external environmental signals include time information, holidays and promotional activities; The time information is used to reflect the temporal nature of user behavior; The holidays are represented by binary codes to indicate whether the user behavior occurs on a holiday; The promotion activity is used to indicate whether there is a promotion activity within a certain time period; The external environment signal maps different external signals into vectors through the embedding layer and is integrated into the state space with the interest heat map and interest trend labels.

5. The enterprise marketing intelligent recommendation method based on deep learning according to claim 1 is characterized in that: The push timing is optimized by combining the interest heat map, interest trend tags, and push channels to select the most appropriate recommendation time for each user, which is expressed as: Among them, t k The time point for recommendation is to select the time point that optimizes the recommendation effect; α k and β k It is the weight factor in time point optimization, used to balance the influence of user activity and seasonality. For the time pool.

6. The enterprise marketing intelligent recommendation method based on deep learning according to claim 1, characterized in that: The recommended action set is also provided with a diversity loss function to avoid excessive duplication of recommendations in the candidate recommendation set. The diversity loss function is expressed as: in, is a diversity loss function used to encourage the diversification of recommended actions; Indicates recommended content and similarity between Indicates ensuring the push channel and No repetition; Indicates ensuring the recommended time point and No repetition; The similarity is reduced to avoid recommending duplicate content.

7. The enterprise marketing intelligent recommendation method based on deep learning according to claim 1, characterized in that: The instant positive feedback is obtained based on historical click / conversion rates; the value of the recommendation action in improving long-term user retention or brand preference is measured based on the approximate number of active times or repeat purchases of the user in the next 7 days; the push frequency cost is accumulated by tracking the number of times the user has received it in the past 7 days.

8. The enterprise marketing intelligent recommendation method based on deep learning according to claim 1 is characterized in that: The steps of actually executing the recommended action with the optimal long-term reward and collecting user feedback after the user responds to the action to train the reinforcement learning-based policy network for system optimization include: Receive the recommended action with the optimal long-term reward, including the recommended content r * , push channel c * and recommended time t * ; The recommended content * Prepare to push template; select push channel c * At the recommended time t * or perform the recommended task within its tolerance range; and simultaneously record metadata; All relevant user behaviors after receiving a recommendation push notification will be collected and structured into a feedback signal group, including: whether the user clicked on the recommendation content, whether the user completed a key conversion, whether the user did not respond, whether the user actively closed, blocked, or reported the push notification, and the time difference between the push notification and the user's first response; The feedback information of all executed recommended actions is organized into a triple, including the current user state vector, the recommended action with the current optimal long-term return, and the feedback signal; System optimization is performed based on the triples.

9. The enterprise marketing intelligent recommendation method based on deep learning according to claim 8, characterized in that: The recommended content includes a certain product or activity; the push channel includes App push and SMS.

10. Enterprise marketing intelligent recommendation system based on deep learning, characterized by: The system comprises: A user heat analysis module is used to construct an interest heat map based on the user's historical behavior data, predict the user's future interest trends, and generate an interest trend label for each user based on the interest trends; wherein the interest heat map is obtained by weighting the user's historical behavior data; the user's future interest trends are predicted based on the interest heat map using a weighted sliding window method; A state space construction module is used to combine the interest heat map and interest trend labels, as well as external environmental signals, and perform trend regularization to obtain a state space; the trend regularization is obtained by smoothing and regularizing the predicted future interest trend of the user; A recommended action selection generation module is used to generate a set of candidate recommended actions for each user based on the state space, including recommended content, push channel, and recommendation time, and optimize the push time in combination with the interest heat map, interest trend tags, and push channel to select the most appropriate recommendation time for each user; An optimal recommended action selection generation module is configured to input the state space into a pre-trained reinforcement learning-based policy network to evaluate the policy value of each candidate action in the candidate recommended action set in its current state and select the recommended action with the best long-term return. The pre-trained reinforcement learning-based policy network generates the policy value of each candidate action in its current state by combining the immediate positive feedback of whether the user clicks or converts the recommended action, the value of the recommended action in improving user retention or brand preference in the long term, and the push frequency cost. The marketing feedback and recommendation execution module is used to complete the actual execution of the recommended action with the optimal long-term return, and collect user feedback behavior after the user responds, which is used to train the reinforcement learning-based strategy network for system optimization.

Images