Enterprise marketing intelligent recommendation method and system based on deep learning

Abstract

The application provides an enterprise marketing intelligent recommendation method and system based on deep learning, which comprises the following steps: constructing an interest heat map according to historical behavior data of a user, and predicting the future interest trend of the user; combining the interest heat map, the interest trend label and external environment signals to obtain a state space; generating a candidate recommendation action set for each user according to the state space; inputting the state space into a pre-trained reinforcement learning-based policy network to evaluate the policy value of each candidate action in the candidate recommendation action set under the current state, and selecting a recommendation action with the optimal long-term return; and completing the real execution of the recommendation action of the optimal long-term return, and collecting user feedback behavior after the user response occurs, so as to optimize the system. The application also has good adaptability and popularization value in complex marketing tasks with sparse user behavior or obvious interest drift.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of enterprise marketing intelligent recommendation, and particularly relates to an enterprise marketing intelligent recommendation method and system based on deep learning. BACKGROUND

[0002] With the continuous evolution of digital marketing means, enterprises need more intelligent and accurate recommendation systems to improve the conversion effect of marketing activities in the face of the increasingly complex user behavior and the diversification of media channels. Traditional recommendation systems are mostly based on collaborative filtering, content tag matching, matrix decomposition or click rate prediction models based on shallow neural networks. These methods mainly focus on the static goal of "what content to recommend", and model the similarity between users and content to predict the products or marketing materials that users may be interested in. Although this method has achieved certain results in general e-commerce or information recommendation scenarios, it still faces many challenges in enterprise-level marketing scenarios.

[0003] Firstly, enterprise marketing emphasizes conversion orientation rather than click orientation. The recommendation system not only needs to recommend "content", but also must make intelligent decisions on "when to recommend" and "through what channel to recommend", because inappropriate or improper media touch methods may lead to user aversion and even cancellation of attention. However, existing recommendation technologies almost stay in the single-dimensional perspective of "content recommendation", lacking the ability to jointly model the recommendation time point, channel response, and user acceptance preference. Secondly, user interest has obvious time sequence dynamics. The conventional model only uses short-term behavior characteristics for static matching, which is difficult to capture the evolution path of user preference and cannot make early prediction before interest migration or fluctuation. Thirdly, enterprises often face serious data sparsity and delayed feedback problems in actual marketing: low user behavior frequency, and lag of conversion events, which makes it difficult for traditional supervised learning recommendation framework to learn effective causal relationships. In addition, the repetition and frequent push of recommended content in multi-channel marketing scenarios can also cause "user fatigue", reducing the overall marketing effect, and existing systems generally lack the perception and control mechanism of marketing "aversion" or "interference cost".

[0004] In summary, the existing marketing recommendation technology still stays in the static recommendation stage with goods or content as the core, lacking dynamic interest modeling, strategy time point optimization, multi-channel distribution control, and marketing feedback closed-loop adjustment mechanism. There is a significant gap between this reality and the actual needs of enterprise marketing. Therefore, there is an urgent need for a new type of recommendation system based on user behavior time sequence characteristics, interest trend evolution, multi-channel response difference, and feedback optimization capability, which can achieve accurate, efficient, and controllable marketing recommendation behavior in complex scenarios. SUMMARY

[0005] The purpose of the present application is to propose a deep learning-based enterprise marketing intelligent recommendation method and system, which automatically identifies the current interest state and its change trend of the user behavior trajectory, thereby predicting the most appropriate recommendation opportunity, and dynamically matching the optimal recommendation method based on the response characteristics of different channels, solving the single problem of traditional recommendation systems that only focus on recommended content.

[0006] To achieve the above purpose, in the first aspect of the present application, a deep learning-based enterprise marketing intelligent recommendation method is provided, which comprises the following steps:

[0007] An interest heat map is constructed according to the historical behavior data of the user, and the future interest trend of the user is predicted, and an interest trend label is generated for each user based on the interest trend; wherein the interest heat map is obtained based on the weighting of the historical behavior data of the user; the future interest trend of the user is obtained by predicting the interest heat map using a weighted sliding window method;

[0008] The interest heat map and the interest trend label are combined with external environmental signals, and trend regularization is performed to obtain a state space; the trend regularization is obtained by smoothing regularization of the predicted future interest trend of the user;

[0009] According to the state space, a candidate recommendation action set containing recommended content, push channel and recommended time point is generated for each user, and the push time point is optimized by combining the interest heat map, the interest trend label and the push channel, so as to select the most appropriate recommendation opportunity for each user;

[0010] The state space is input into a pre-trained reinforcement learning-based policy network to evaluate the policy value of each candidate action in the candidate recommendation action set under the current state, and the recommendation action with the optimal long-term return is selected; wherein the pre-trained reinforcement learning-based policy network combines the immediate positive feedback of whether the user clicks or converts the recommendation action, measures the value of the recommendation action in improving user retention or brand preference in the long term, and generates the policy value of each candidate action under the current state according to the push frequency cost;

[0011] The real execution of the recommended behavior of the recommendation action with the optimal long-term return is completed, and the user feedback behavior is collected after the user response occurs, which is used to train the reinforcement learning-based policy network for system optimization.

[0012] Further, the historical behavior data includes click actions, browsing actions and purchase actions.

[0013] Further, the interest trend label specifically includes:

[0014] If the interest trend is higher than the current interest, mark as rising;

[0015] If the interest trend is lower than the current interest, mark as falling;

[0016] If the interest trend is equal to the current interest, mark as stable.

[0017] Further, the external environment signal includes time information, holidays and promotion activities;

[0018] The time information is used to reflect the timing of user behavior;

[0019] The holidays are represented by binary coding to indicate whether the user behavior occurs during holidays or not;

[0020] The promotion activities are used to indicate whether there are promotion activities in a certain period of time or not;

[0021] The external environment signal maps different external signals into vectors through an embedding layer, and is fused into the state space with the interest heat map and the interest trend label.

[0022] Further, the combination of the interest heat map, the interest trend label and the push channel optimizes the push time point, and selects the most suitable recommendation opportunity for each user, represented as:

[0023]

[0024] Where, t k is the recommended time point, and the time point is selected to optimize the recommendation effect; α k and β k are weight factors in the time point optimization, used to balance the user activity and the influence of seasonality; is the time pool.

[0025] Further, the selected recommendation action set is also provided with a diversity loss function, which is used to avoid too many repeated recommendations in the candidate recommendation set, and the diversity loss function is represented as:

[0026]

[0027] Where, is the diversity loss function, which is used to encourage the diversity of the recommendation action. represents the similarity between the recommended content and ; represents that the push channel is not repeated with ; represents that the recommended time point with no repetition;

[0028] wherein the similarity is reduced to avoid recommending repetitive content.

[0029] Further, the instant positive feedback is based on historical click / conversion rate; the value of the recommended action in improving user retention or brand preference in the long term is approximated by the number of active times or repeat purchase behavior of the user in the next 7 days; and the push frequency cost is accumulated by tracking the number of times the user receives in the last 7 days.

[0030] Further, the step of completing the recommended behavior of the recommended action with the optimal long-term return and collecting user feedback behavior after user response occurs for training the reinforcement learning-based policy network for system optimization comprises:

[0031] receiving the recommended action with the optimal long-term return, including recommended content r * , push channel c * , and recommended time point t * ;

[0032] preparing the recommended content r * as a push template; selecting a push channel c * ; performing a recommendation task at a recommended time point t * or its tolerance range; and synchronously recording metadata;

[0033] All relevant behaviors of the user after receiving the recommended push will be collected and structured as a feedback signal group, including whether the recommended content is clicked, whether the key conversion is completed, whether any response is made, whether the push is actively closed or shielded or reported, and the time difference between the sending of the push to the user and the first response;

[0034] All feedback information of the executed recommended behavior is sorted into a triple, including the current user state vector, the recommended action with the optimal long-term return, and the feedback signal.

[0035] Based on the triple, the system is optimized.

[0036] Further, the recommended content includes a certain product or activity; and the push channel includes App push and SMS.

[0037] In a second aspect of the present application, a deep learning-based enterprise marketing intelligent recommendation system is provided, which comprises:

[0038] a user heat analysis module configured to construct an interest heat map based on historical behavior data of the user, predict a future interest trend of the user, and generate an interest trend label for each user based on the interest trend, wherein the interest heat map is obtained by weighting the historical behavior data of the user, and the future interest trend of the user is predicted based on the interest heat map using a weighted sliding window method;

[0039] a state space construction module configured to combine the interest heat map and the interest trend label, and an external environment signal, and perform trend regularization to obtain a state space, wherein the trend regularization is obtained by smoothing and regularizing the predicted future interest trend of the user;

[0040] a selected recommendation action generation module configured to generate a candidate recommendation action set for each user based on the state space, including recommended content, a push channel, and a recommended time point, and optimize the recommended time point by combining the interest heat map, the interest trend label, and the push channel, to select the most suitable recommendation time for each user;

[0041] an optimal recommendation action selection generation module configured to input the state space into a pre-trained reinforcement learning-based policy network to evaluate the policy value of each candidate action in the candidate recommendation action set in the current state, and select a recommendation action with an optimal long-term return, wherein the pre-trained reinforcement learning-based policy network combines immediate positive feedback after the user clicks or converts the recommendation action, measures the value of the recommendation action in improving user retention or brand preference in the long term, and generates the policy value of each candidate action in the current state based on the push frequency cost;

[0042] a marketing feedback and recommendation execution module configured to complete real execution of the recommendation behavior of the recommendation action with the optimal long-term return, and collect user feedback behavior after the user response occurs, to train the reinforcement learning-based policy network for system optimization.

[0043] The present application has at least the following beneficial technical effects:

[0044] The application provides a deep learning intelligent recommendation method with user dynamic interest recognition, marketing content and push channel joint recommendation decision, intelligent selection of recommendation time point and optimization of marketing feedback response capability. The method can automatically recognize the interest state and its change trend of the user based on the user behavior track, so as to predict the most suitable recommendation time, and dynamically match the optimal recommendation mode based on the response characteristics of different channels, solve the single problem of traditional recommendation system only focusing on the recommended content. On this basis, the application further introduces a multi-dimensional feedback modeling mechanism, the system can adjust and update the strategy according to the user's conversion behavior, adverse signal and marketing execution cost, so as to realize the accurate, efficient and robust enterprise-level intelligent recommendation capability.

[0045] The application builds a complete user behavior modeling system and recommendation decision optimization logic, so that the system can consider the three targets of recommendation relevance, acceptance probability and conversion effect, and realize the change from static recommendation to strategic recommendation. The system is not only suitable for high-frequency user behavior data scene, but also has good adaptability and popularization value in complex marketing tasks with sparse user behavior or obvious interest drift, and has significant technical progress and practical application significance. BRIEF DESCRIPTION OF DRAWINGS

[0046] The application is further described by using the drawings, but the embodiments in the drawings do not constitute any limitation on the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the following drawings.

[0047] Figure 1 The application discloses an enterprise marketing intelligent recommendation method based on deep learning

[0048] Figure 2 The application discloses an enterprise marketing intelligent recommendation system framework based on deep learning. DETAILED DESCRIPTION

[0049] The embodiments of the application are described in detail below, and the examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the application, and cannot be understood as a limitation on the application.

[0050] Embodiment one

[0051] As shown in Figure 1 The application discloses an enterprise marketing intelligent recommendation method based on deep learning, which comprises:

[0052] S1, constructing an interest heat map based on historical behavior data of a user, and predicting a future interest trend of the user, and generating an interest trend label for each user based on the interest trend; wherein the interest heat map is obtained based on weighting the historical behavior data of the user; and the future interest trend of the user is predicted based on the interest heat map using a weighted sliding window method.

[0053] Specifically, the goal of this step is to construct a dynamic interest heat map based on the user's historical behavior data, and predict the user's future interest trend. Through this process, the application can capture the changes in user interest and provide data support for subsequent recommendation system decision-making. Finally, by generating an interest heat map and predicting trends, the application can accurately predict changes in user interest and provide a basis for subsequent recommendation decisions.

[0054] Further, the construction of the interest heat map:

[0055] The interest heat map is used to represent the user's interest intensity in different products, services or topics at different time periods. In order to generate this heat map, the application needs to perform weighting processing according to the user's historical behavior, reflecting the different effects of different behaviors on interest intensity.

[0056] wherein the interest heat map T i,j represents the user's interest intensity in product or topic i at time point j.

[0057] Further, when calculating the interest intensity, the application weights the user's behavior data. Assuming that the user has multiple behaviors (such as clicking, browsing, purchasing, etc.) in a certain period of time, the interest heat can be calculated as follows:

[0058]

[0059] wherein T i,j : the interest heat of the user in product or topic i at time point j.

[0060] w k : represents the weighting coefficient of different behavior types (such as clicking, browsing, purchasing, etc.). These weighting coefficients reflect the contribution of different behaviors to user interest.

[0061] h i,j,k : represents the quantitative value of the kth behavior of the user in product i at time point j (such as the number of clicks, browsing time, etc.).

[0062] The purpose of this formula is to quantify the impact of different user behaviors on interest through weighting. For example, a purchase behavior may be given a higher weight, a click behavior may be given a medium weight, and a browsing behavior may be given a lower weight.

[0063] Further, interest trend prediction:

[0064] After the construction of the interest heat map T i,j After the construction of the interest heat map T

[0065] The goal of the present application is to predict the interest intensity of the user on product i at the next time point j+1

[0066]

[0067] wherein, represents the predicted interest heat of the user on product i at time point j+1.

[0068] T i,j-k represents the interest heat of the user on product i in the past k time points.

[0069] α k : weighting coefficient, representing the influence degree of past interest data on future interest prediction.

[0070] wherein, the purpose of the formula is to predict the future interest value according to the historical interest data and the weighting coefficient (α k ), to ensure that the interest trend can consider the recent changes in user behavior. For example, if the user's interest in a product has been rising in the recent period, the system can predict that the interest may continue to rise in the future period.

[0071] Further, interest trend label generation:

[0072] Based on the interest heat prediction result, the present application generates an interest trend label τ u for each user, marking the change trend of the current interest of the user. The generation of the interest trend label is based on the following rules:

[0073] If the future interest is higher than the current interest T i,j , it is marked as rising.

[0074] If the future interest is lower than the current interest T i,j , it is marked as falling.

[0075] If they are equal or the change is not large, it is marked as stable.

[0076] This label will help the subsequent steps to determine which stage of interest the user is currently in, thereby providing a basis for the decision-making of the recommendation system.

[0077] Through this step, the present application uses the user's historical behavior data and interest prediction model to construct a dynamic interest heat map and predict the user's interest trend. This not only provides real-time interest data support for the recommendation system, but also helps the system to make more accurate push and strategy adjustment in the future according to the changing trend of user interest.

[0078] S2, combine the interest heat map and interest trend label, and external environment signal, and perform trend regularization to obtain a state space; the trend regularization is obtained by smoothing regularization on the predicted future interest trend of the user.

[0079] Specifically, the core goal of this step is to combine the interest heat map T and interest trend label τ u generated in step 1 with external environment signals to construct a state space s t that can be used for reinforcement learning model, and input these information into the subsequent recommendation strategy generation. This step combines the changing trend of dynamic interest and the influence of external environment on user behavior, thereby providing more comprehensive and accurate input for the recommendation system.

[0080] Further, in step 1, the present application constructs the interest heat map T and predicts the changing trend of user interest. Based on these data, the key task of this step is to construct a comprehensive state space that integrates interest heat, trend label and external environment signal as the input of subsequent reinforcement learning.

[0081] The present application can combine user interest, interest change trend and external environment signal into a high-dimensional state vector s t in the following way:

[0082] s t =f(T,τ u ,e t ) (3)

[0083] Where s t : comprehensive state vector, represents the comprehensive information of user's interest state and external environment factors at time point t.

[0084] T: user's interest heat map for a product or topic at time point t.

[0085] τ u : interest trend label, describes the changing trend of user interest (such as rising or falling).

[0086] e tExternal environment signals, including timestamp, holiday, promotion, etc.

[0087] f: a mapping function that fuses the interest heat map, interest trend label and external environment signals into a state vector.

[0088] It can be understood that in this way, the application not only reflects the dynamic changes of user interest, but also combines the influence of external factors on interest, thereby constructing a complete state space.

[0089] where the external environment signal e t is an important innovation in this step. External factors (such as time information, holidays, promotions, etc.) will significantly affect the user's interest pattern, so these factors need to be encoded into information suitable for model processing. In order to achieve this, the application encodes the external environment signal as follows:

[0090] Time information: The timestamp t reflects the time sequence of user behavior. For example, the interest pattern of holidays or weekdays is usually different, so the application can use time series encoding (such as periodic features or time of day) to represent the influence of time.

[0091] Holiday signal: Special events such as holidays can affect user behavior and interest. The application uses binary encoding to represent whether the user behavior occurs on a holiday. For example, if the current date is a holiday, the value of this item is 1, and the value is 0 when it is not a holiday.

[0092] Promotion: Promotions can cause dramatic fluctuations in user interest. The application can introduce a promotion flag to indicate whether there is a promotion in a certain time period.

[0093] These environment signals are mapped into vectors through embedding layers (such as word embedding technology or simple numerical encoding), and are fused into the state space s t along with the interest heat map and trend label.

[0094] Further, in order to further enhance the generalization ability of the state space in the reinforcement learning process, the application introduces an innovative regularization mechanism. Traditional reinforcement learning methods often rely on static state space modeling, but in actual applications, the user's interest changes often have strong dynamics. Therefore, the application needs a method to deal with the overfitting problem in the state space, especially when there is noisy data. For this purpose, the application proposes an interest trend regularization term to control the dynamic changes in the state space and avoid excessive reliance on past behavior data.

[0095] The application can introduce a trend regularization term to ensure that the state vector s tThe interest trend in the state space s

[0096]

[0097] The regularization term of interest trend constrains the smoothness of interest change.

[0098] The predicted interest heat value represents the interest in product or topic i at time point t.

[0099] λ: Regularization strength hyperparameter, controls the strength of smoothness constraint.

[0100] The role of this regularization term is to force the model to consider the smooth change of interest in the state space, avoiding the model's excessive dependence on short-term behavioral fluctuations, thereby improving the system's generalization ability.

[0101] By constructing a rich state space s t This step provides multi-dimensional input data for the subsequent reinforcement learning model, including not only the historical change of user interest, but also the influence of external environmental factors.

[0102] S3, according to the state space, generate a set of candidate recommendation actions for each user, including recommended content, push channel and recommended time point, and optimize the push time point by combining the interest heat map, interest trend label and push channel, to select the most suitable recommendation timing for each user.

[0103] Specifically, the core goal of this step is to generate a set of candidate recommendation actions based on the state space s t generated in step S2, which will include recommended content, recommended push channel and push time point. The generation of the candidate recommendation action set not only considers the state change of user interest, but also integrates external environmental factors (such as holidays, promotional activities, etc.), thereby providing a diverse selection space for subsequent reinforcement learning strategy optimization. In practical applications, these candidate recommendation actions will be used to select the optimal recommendation strategy to achieve a personalized and dynamic recommendation system.

[0104] In step 2, the invention constructs a state space s t which contains information such as user interest change and external environmental impact. This state space will serve as the basis for generating recommendation actions. In this step, the invention combines this information with the optimization of recommended content, channel and time point to create a multi-dimensional set of candidate recommendation actions A tAnd ensure that the generated recommended actions can fully cover user needs, while improving the timeliness and accuracy of recommendations.

[0105] Further, the candidate recommended action set is generated:

[0106] Based on the state space s of step S2 t The application generates a candidate recommended action set A for each user t Each recommended action includes the following three elements:

[0107] Recommended content r i : Products, services or activities that users are interested in. According to the user's interest heat map T and trend label τ u The recommendation system selects content that matches the user's interest from the content pool .

[0108] Recommended channel c j : Choose the most suitable push channel. The push channel can be App push, SMS, email, etc., and the most suitable channel is selected according to the user's historical behavior and preferences.

[0109] Recommended time point t k : The time point of the push. The system can analyze the user's active period to choose the right time to push the recommended content.

[0110] Candidate recommended action set A t Can be generated by the following formula:

[0111]

[0112] Where A t : Recommended action candidate set, containing a collection of multiple recommended actions.

[0113] r i : Recommended content, selected from the content pool .

[0114] c j : Push channel, selected from the channel pool .

[0115] t k : Recommended time point, selected from the time pool .

[0116] In this way, the application generates multiple candidate recommended actions for users, each of which not only reflects the changes in user interest, but also optimizes the actual effect of the recommendation according to the push channel and time point.

[0117] Furthermore, the innovation of this design lies in the multi-dimensional generation of recommendation actions: the generated recommendation actions incorporate multi-dimensional information including content, channel, and time point, enabling recommendations to go beyond single interest data and comprehensively consider user behavior, environmental signals, and timeliness, thereby achieving truly personalized recommendations. Another aspect is dynamic timeliness: by selecting an appropriate push time point t for each recommendation action... k The system can ensure that recommendations achieve maximum effect during periods of user activity, avoiding missing the best recommendation opportunities.

[0118] Further optimizations include content, channels, and timing.

[0119] When generating candidate recommendation actions, this invention needs to further optimize the content, channel, and timing of each recommendation action to ensure that it meets user needs to the greatest extent.

[0120] Content optimization: Based on the user's interest heatmap T and trend tag τ generated in step 1. u This invention prioritizes content that users have high or rising interest in. Content selection relies not only on historical behavior but also on predictions of user interest trends. This ensures that recommended content aligns with future changes in user interests, avoiding the recommendation of outdated or irrelevant content.

[0121] Channel compatibility: Push channel c j The selection is based on users' historical interaction data. By analyzing users' responses on different channels (e.g., conversion rates via app push notifications are higher than via SMS push notifications), this invention can select the most suitable push channel for each user. In particular, during certain special periods (such as holidays or promotional events), the system will prioritize the channel with the best performance.

[0122] Timing Optimization: Timing selection is achieved by analyzing user activity periods. Based on user activity patterns at different times (such as differences in activity levels between morning and evening), this invention can select the most suitable recommendation time for each user. This invention optimizes the recommendation timing in the following ways:

[0123]

[0124] Among them, t k : Recommended timing: Select the time point that optimizes the recommendation effect. α k and β k : A weighting factor in point-in-time optimization, used to balance the effects of user activity and seasonality.

[0125] This formula combines user activity and seasonal factors to select the point t that best maximizes conversion rates.k This ensures the timeliness and accuracy of the recommendations.

[0126] The innovation here lies in the optimization of timing: by incorporating activity and seasonality factors, the timeliness of recommendations is optimized to ensure that recommendations are pushed during the user's most active periods, thereby improving the conversion rate. It also includes personalized matching of channels and content: based on user history and reactions, the system can select the most suitable recommendation channel for each user, while prioritizing content that the user may be interested in, thus improving the accuracy of recommendations.

[0127] Furthermore, the diversity and exploration of candidate recommendation actions:

[0128] To prevent the recommendation system from becoming overly reliant on a particular type of content or push method, this invention introduces a diversity constraint when generating candidate recommendation action sets. The purpose of this constraint is to encourage exploration, prevent the recommendation system from becoming trapped in information cocoons or experiencing recommendation fatigue, and ensure that users receive a wider variety of recommendations.

[0129] Specifically, this invention can introduce a diversity loss function to avoid excessive duplicate recommendations in the candidate recommendation set. For example, this invention aims to ensure that different push channels, content types, and recommendation times in the candidate recommendation action set can fully cover user needs without excessive overlap in any dimension.

[0130]

[0131] in, A diversity loss function is used to encourage a variety of recommended actions.

[0132] sim(r i ,r j ): Indicates recommended content r i and r j The similarity between them. By reducing the similarity, the system can avoid recommending duplicate content.

[0133] Ensure push channel c i With c j No repetition.

[0134] Ensure the recommended timing point t i With t j No repetition.

[0135] It is understandable that through this loss function, the system can guide diverse explorations when generating candidate recommendation actions, ensuring multi-dimensional coverage of the recommendation set, thereby avoiding recommendation fatigue caused by a single recommendation mode.

[0136] The innovations in this section were added later, including diversity constraints: by introducing a diversity loss function, this invention not only optimizes the relevance of recommended content but also ensures the diversity of recommendations, preventing the system from falling into a situation of overfitting. It also encourages exploration and innovation: through appropriate exploration mechanisms, the system can provide more potential content that meets user needs while ensuring recommendation accuracy, preventing users from falling into a situation of limited information.

[0137] By generating a multi-dimensional candidate recommendation action set A t This step provides a rich and personalized selection space for subsequent recommendation strategy optimization. By combining user interests, external environmental signals, push timing, and diversity constraints, the system ensures the accuracy, timeliness, and diversity of recommended content. Ultimately, the recommendation system can make optimal recommendation strategies based on multi-dimensional data, thereby improving recommendation conversion rates and user satisfaction.

[0138] S4. Input the state space into a pre-trained reinforcement learning-based policy network to evaluate the policy value of each candidate action in the current state, and select the recommended action with the best long-term return; wherein, the pre-trained reinforcement learning-based policy network combines the immediate positive feedback after the user clicks or converts the recommended action, measures the value of the recommended action in improving user retention or brand preference in the long term, and the cost of push frequency to generate the policy value of each candidate action in the current state.

[0139] Specifically, the goal of this step is to: based on the state vector s generated in step S2 t and the candidate recommendation action set A constructed in step 3 t By training a reinforcement learning policy network, the potential value of each candidate action in the current state is evaluated, thereby selecting the recommended action with the optimal long-term reward. This strategy network is responsible for modeling the optimal mapping between user state and recommendation behavior. It is the core strategy for achieving personalized and dynamic recommendations in the entire patent solution. Its task is to learn rather than execute or provide feedback, which will be handled by subsequent steps.

[0140] Furthermore, unlike traditional recommendations that only maximize click-through rates, this modeling process designs multi-dimensional objectives and customizes models for key factors in enterprise marketing recommendation scenarios. This enables the strategy network to not only output recommendation actions that users are interested in, but also to consider the dynamic balance between marketing resources and frequency.

[0141] Furthermore, the inputs and outputs of the policy network are defined as follows:

[0142] The policy network uses the current user state vector s t As input, the output is in the candidate recommended action set At Each action a t = <r i ,c j ,t k Value estimation Q(s) t ,a t And select the optimal action from them.

[0143] State vector s t Includes:

[0144] High-frequency interest features derived from the interest heatmap T;

[0145] From trend tag τ u The direction of interest evolution;

[0146] From environmental signal e t The sensitivity of push notifications (e.g., whether it is a holiday or a promotion);

[0147] The user's current push history (used for frequency control).

[0148] Candidate Recommended Action Set A t It is a set consisting of multiple triples, each action a t = <r i ,c j ,t k > These represent the recommended content, the push channel, and the recommendation time, respectively.

[0149] Furthermore, value function modeling:

[0150] To train the policy network, this invention defines a policy value function Q(s) t ,a t ), used to measure action a t The long-term expected return in the current state. This function not only considers direct revenue such as click-through rates and conversions, but also incorporates system costs such as user acceptance and marketing quality control, thereby achieving strategy learning that better meets practical needs.

[0151]

[0152] in, This indicates the immediate positive feedback after a user clicks or converts the recommended action, and can be modeled based on historical click / conversion rates.

[0153] The value of recommendation actions in improving user retention or brand preference in the long run can be approximated by the number of times users are active or repeat purchases in the next 7 days.

[0154] To reduce the cost of push notifications and prevent excessively frequent recommendations, the system can accumulate modeling by tracking the number of times a user receives notifications over the past 7 days.

[0155] λ1, λ2: Weighted hyperparameters that control long-term returns and frequency penalties.

[0156] This function enables the policy network to identify actions that generate good short-term conversions but damage user relationships in the long run, thus making it more inclined to choose the recommendation behavior that is optimal for users in the long run.

[0157] Optimal action selection logic:

[0158] After completing the action evaluation, the system determines the final policy output by finding the action with the highest value in the current state among all candidate actions:

[0159]

[0160] in, Indicates the current state s t The optimal recommended action, including the recommended content r, is determined. * Channel C * Time point t * .

[0161] The strategy does not execute the action directly, but instead delegates the decision to the next step in conjunction with a feedback mechanism.

[0162] This step is the core layer of strategy modeling in the patent. Through reinforcement learning, it transforms complex user states into decision-making recommendation behaviors, serving as a crucial logical bridge connecting front-end state modeling and back-end recommendation execution. The learned strategies not only consider user behavior data but also encompass system resources and market dynamics, enabling the recommendation system to possess real-time intelligence, strategy robustness, and commercial controllability, highly matching the actual needs of enterprise intelligent marketing scenarios.

[0163] S5. Perform the recommendation action with the best long-term reward in a real-world manner, and after the user response occurs, collect user feedback behavior to train the reinforcement learning-based policy network for system optimization.

[0164] Specifically, the goal of this step is to recommend the optimal action based on the policy network selected in step S4. The process involves executing the recommendation behavior and collecting user feedback signals after a response occurs, constructing structured feedback data for subsequent strategy training. This step serves as the interface layer connecting intelligent decision-making with real-world user interaction within the entire system. It is responsible for both scheduling and executing recommendation actions and converting behavioral signals into trainable data, thus forming the closed-loop establishment node in the patented solution.

[0165] Furthermore, the recommended action execution mechanism is as follows:

[0166] The recommendation system receives the optimal recommendation action output from step 4. Then, the actual push scheduling module in the marketing system is invoked to perform the following operations:

[0167] Recommended content r * (e.g., a product or promotion) will be prepared as a push notification template;

[0168] Select push channel c * (such as App push notifications, SMS messages, WeChat service notifications, etc.);

[0169] Recommended time point t * Or perform the recommended task within its tolerance range;

[0170] Synchronously record push logs, reach status, device response codes and other metadata for subsequent behavior mapping.

[0171] Understandably, the key at this stage is not modeling, but rather the accurate, efficient, and unbiased execution of the selected optimal recommendation strategy, and ensuring that all behavioral events are traceable within the system.

[0172] Furthermore, user behavior feedback collection and structured representation:

[0173] All user behaviors after receiving the recommendation push will be collected and structured as feedback signal group b. t Including but not limited to the following dimensions:

[0174] Whether to click on the recommended content (Boolean);

[0175] Whether key conversions (such as purchases, reservations, etc.) were completed;

[0176] Has no response been made?

[0177] Whether to actively turn off, block, or report this push notification;

[0178] The time difference between the push notification being sent to the user's first response.

[0179] All signals will be standardized into structured behavioral labels for use in the design of the subsequent reward function, but the reward function itself will not be calculated directly in this step.

[0180] Furthermore, the integration of feedback data organization with the policy network:

[0181] Feedback information for all executed recommendation actions will be written to the action log table or experience record pool used for policy training in the following triplet format:

[0182]

[0183] Among them, s t User status, from step S2;

[0184] The recommended action actually performed in this step comes from step S4;

[0185] b t The set of user feedback signals collected in this step.

[0186] In subsequent policy training, another step (not the responsibility of this step) will read the triplet and apply the reward function to b. t The signal is then transformed to generate a numerical training signal. However, the goal of this step is to ensure the integrity, accuracy, and structure of the feedback signal, providing high-quality input for training.

[0187] This step revolves around two core tasks: executing the optimal recommended action and collecting behavioral feedback, ensuring that the recommendation results can be implemented effectively.

[0188] Example 2

[0189] like Figure 2 As shown, this embodiment of the invention also provides an intelligent recommendation system for enterprise marketing based on deep learning, the system comprising:

[0190] The user popularity analysis module 101 is used to construct an interest heatmap based on the user's historical behavior data, predict the user's future interest trends, and generate interest trend tags for each user based on the interest trends; wherein the interest heatmap is obtained by weighting the user's historical behavior data; and the user's future interest trends are predicted based on the interest heatmap using a weighted sliding window method.

[0191] The state space construction module 102 is used to combine the interest heat map and interest trend tags, as well as external environmental signals, and perform trend regularization to obtain the state space; the trend regularization is obtained by smoothing the predicted user's future interest trend.

[0192] The recommended action generation module 103 is used to generate a set of candidate recommended actions for each user based on the state space, including recommended content, push channel and recommendation time point, and optimize the push time point by combining the interest heat map, interest trend tags and push channel to select the most suitable recommendation time for each user.

[0193] The optimal recommendation action selection generation module 104 is used to input the state space into a pre-trained reinforcement learning-based policy network to evaluate the policy value of each candidate action in the candidate recommendation action set in the current state and select the recommendation action with the best long-term return. The pre-trained reinforcement learning-based policy network combines the immediate positive feedback after the user clicks or converts the recommendation action, measures the value of the recommendation action in improving user retention or brand preference in the long term, and the cost of push frequency to generate the policy value of each candidate action in the current state.

[0194] The marketing feedback and recommendation execution module 105 is used to execute the recommendation action with the best long-term return, and after the user response occurs, it collects user feedback behavior to train the reinforcement learning-based policy network for system optimization.

[0195] The foregoing has described specific embodiments of this specification; other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than those shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily have to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0196] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0197] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0198] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0199] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0200] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0201] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0202] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0203] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0204] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0205] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0206] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0207] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0208] Finally, it should be noted that the enterprise marketing intelligent recommendation platform based on deep learning disclosed in the embodiments of the present invention is only a preferred embodiment of the present invention and is only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep learning-based intelligent recommendation method for enterprise marketing, characterized in that, The method includes the following steps: An interest heatmap is constructed based on users' historical behavior data, and future interest trends of users are predicted. An interest trend tag is generated for each user based on the interest trends. The interest heatmap is obtained by weighting the users' historical behavior data. The future interest trends of users are predicted based on the interest heatmap using a weighted sliding window method. The state space is obtained by combining the interest heatmap and interest trend tags, as well as external environmental signals, and performing trend regularization; the trend regularization is obtained by smoothing the predicted user's future interest trend. Based on the state space, a set of candidate recommendation actions is generated for each user, including recommended content, push channel, and push time. The push time is optimized by combining the interest heat map, interest trend tags, and push channel to select the most suitable recommendation time for each user. The state space is input into a pre-trained reinforcement learning-based policy network to evaluate the policy value of each candidate action in the current state, and select the recommended action with the best long-term return. The pre-trained reinforcement learning-based policy network combines the immediate positive feedback after the user clicks or converts the recommended action, measures the value of the recommended action in improving user retention or brand preference in the long term, and the cost of push frequency to generate the policy value of each candidate action in the current state. The recommendation action is executed based on the optimal long-term reward. After the user responds, the user feedback behavior is collected to train the reinforcement learning-based policy network for system optimization.

2. The enterprise marketing intelligent recommendation method based on deep learning according to claim 1, characterized in that, The historical behavioral data includes click actions, browsing actions, and purchase actions.

3. The enterprise marketing intelligent recommendation method based on deep learning according to claim 1, characterized in that, The interest trend tags specifically include: If the stated interest trend is higher than the current interest, it is marked as rising; If the interest trend is lower than the current interest, it is marked as declining; If the interest trend equals the current interest, it is marked as stable.

4. The enterprise marketing intelligent recommendation method based on deep learning according to claim 1, characterized in that, The external environmental signals include time information, holidays, and promotional activities; The time information is used to reflect the temporal sequence of user behavior; The holidays are indicated by binary encoding to show whether a user's behavior occurred on a holiday; The promotional activity is used to indicate whether a promotional activity exists within a certain time period; The external environment signals are mapped into vectors through an embedding layer and then fused into the state space along with the interest heatmap and interest trend labels.

5. The enterprise marketing intelligent recommendation method based on deep learning according to claim 1, characterized in that, The optimization of the push timing by combining the interest heatmap, interest trend tags, and push channels to select the most suitable recommendation time for each user is expressed as follows: ； in, For push notification timing, select the time point that optimizes the recommendation effect; and It serves as a weighting factor in point-in-time optimization, used to balance the impact of user activity and seasonality. For time pool.

6. The enterprise marketing intelligent recommendation method based on deep learning according to claim 1, characterized in that, The selection recommendation action set also includes a diversity loss function to avoid excessive duplicate recommendations in the candidate recommendation set. The diversity loss function is expressed as follows: ； in, This is a diversity loss function used to encourage diversity in recommended actions; Indicates recommended content and The similarity between them; This indicates that the push channel is being ensured. and No repetition; This indicates ensuring the timing of the push notification. and No repetition; This involves reducing similarity to avoid recommending duplicate content.

7. The enterprise marketing intelligent recommendation method based on deep learning according to claim 1, characterized in that, The instant positive feedback is obtained based on historical click / conversion rates; the value of the recommendation action in improving user retention or brand preference in the long term is approximately represented by the number of times the user is active or repurchased in the next 7 days; the cost of push frequency is obtained by tracking the number of times the user receives the push notifications in the past 7 days.

8. The enterprise marketing intelligent recommendation method based on deep learning according to claim 1, characterized in that, The steps of executing the recommendation behavior based on the optimal long-term reward and collecting user feedback behavior after the user response occurs to train the reinforcement learning-based policy network for system optimization include: The recommended action that receives the optimal long-term return includes the recommended content. Push Channel and push timing ; The recommended content Prepare the push template; select the push channel. At the time of push notification Perform recommended tasks within its tolerance range; and synchronously record metadata; All user behaviors after receiving the recommendation push will be collected and structured into a set of feedback signals, including: whether the user clicked on the recommended content, whether a key conversion was completed, whether no response was made, whether the user actively closed, blocked, or reported the push, and the time difference between the push and the user's first response. The feedback information of all executed recommendation actions is organized into triples, including the current user state vector, the recommendation action with the best long-term reward, and the feedback signal; System optimization is performed based on the triples.

9. The enterprise marketing intelligent recommendation method based on deep learning according to claim 8, characterized in that, The recommended content includes a specific product or activity; the push channels include App push and SMS.

10. A deep learning-based intelligent recommendation system for enterprise marketing, characterized in that: The system includes: The user popularity analysis module is used to construct an interest heatmap based on users' historical behavior data, predict users' future interest trends, and generate interest trend tags for each user based on the interest trends; wherein the interest heatmap is obtained by weighting the users' historical behavior data; and the users' future interest trends are predicted based on the interest heatmap using a weighted sliding window method. The state space construction module is used to combine the interest heatmap and interest trend tags, as well as external environmental signals, and perform trend regularization to obtain the state space; the trend regularization is obtained by smoothing the predicted user's future interest trend. The recommended action generation module is used to generate a set of candidate recommended actions for each user based on the state space, including recommended content, push channel and push time point, and optimize the push time point by combining the interest heat map, interest trend tags and push channel to select the most suitable recommendation time for each user. The optimal recommendation action selection generation module is used to input the state space into a pre-trained reinforcement learning-based policy network to evaluate the policy value of each candidate action in the candidate recommendation action set in the current state, and select the recommendation action with the best long-term return; wherein, the pre-trained reinforcement learning-based policy network combines the immediate positive feedback after whether the user clicks or converts the recommendation action, measures the value of the recommendation action in improving user retention or brand preference in the long term, and push frequency cost to generate the policy value of each candidate action in the current state; The marketing feedback and recommendation execution module is used to complete the actual execution of the recommendation behavior based on the optimal long-term return recommendation action, and to collect user feedback behavior after the user response occurs, which is used to train the reinforcement learning-based policy network for system optimization.

Images