Push strategy generation method and device, equipment and storage medium

Abstract

The application discloses a push strategy generation method and device, equipment and storage medium, and specifically discloses the following: obtaining full-link data corresponding to a target user; the full-link data comprises push funnel data and commercial value data; wherein, the push funnel data is used to represent the delivery, display and click conditions of a push message; the commercial value data is used to represent the retention and value conversion conditions of the target user for the push message; diagnosing and analyzing the full-link data to determine a loss link; wherein, the loss link is a link in the delivery, display, click, retention and value conversion links of the push message, and the user loss amount of the link exceeds a preset threshold; generating a plurality of candidate push strategies for improving the loss link; distributing and testing the plurality of candidate push strategies, obtaining feedback data corresponding to each candidate push strategy, and adjusting the traffic distribution proportion of each candidate push strategy according to the feedback data to generate a target push strategy.

Description

Technical Field

[0001] This invention relates to the field of Internet technology, and in particular to a method, apparatus, device and storage medium for generating push strategies. Background Technology

[0002] Currently, push notification operations for mobile applications typically rely on manual processes by operations personnel to determine the content and method of push notifications and evaluate their effectiveness. Specifically, operations personnel pre-determine the content and method of push notifications based on their experience, and after the push message is sent, they manually evaluate the effectiveness of the push based on the relevant data generated by the push, and manually adjust subsequent push notifications accordingly.

[0003] However, in the above-mentioned push operation methods, the information on which the manual evaluation of the push effect is based is limited and it is difficult to fully reflect the actual value generated by the push. This leads to a discrepancy between the evaluation results and the actual value generated by the push, making it difficult for the push content determined based on the evaluation results to accurately match the user's real needs. At the same time, the determination and adjustment of pushes both rely on manual completion by the operation personnel. The manual processing method is inefficient and has obvious lag, making it difficult to optimize pushes in a timely and effective manner. Summary of the Invention

[0004] The main objective of this invention is to provide a method, apparatus, device, and storage medium for generating push strategies, aiming to solve the problems of inaccurate push effect evaluation, delayed push adjustment, and low efficiency caused by the fact that the determination, evaluation, and adjustment of pushes all rely on manual completion by operators and the information on which the push effect evaluation is based is limited.

[0005] In a first aspect, embodiments of the present invention provide a method for generating a push strategy, comprising: Acquire full-link data corresponding to the target user; the full-link data includes push funnel data and commercial value data; wherein, the push funnel data is used to characterize the delivery, display and click status of push messages; the commercial value data is used to characterize the retention and value conversion status of the target user in response to the push messages; Diagnose and analyze the entire data chain to identify the loss links; wherein, the loss links are the links in the delivery, display, click, retention and value conversion of the push message where the user churn exceeds a preset threshold; Multiple candidate push strategies are generated to improve the loss process; wherein, the candidate push strategy includes push content and push time; The multiple candidate push strategies are distributed and tested to obtain feedback data corresponding to each candidate push strategy. Based on the feedback data, the traffic allocation ratio of each candidate push strategy is adjusted to generate the target push strategy.

[0006] Secondly, embodiments of the present invention provide a push strategy generation apparatus, comprising: The acquisition module is used to acquire full-link data corresponding to the target user; the full-link data includes push funnel data and commercial value data; wherein, the push funnel data is used to characterize the delivery, display and click status of push messages; the commercial value data is used to characterize the retention and value conversion status of the target user in response to the push messages; The determination module is used to perform diagnostic analysis on the entire link data to determine the loss links; wherein, the loss links are the links in the delivery, display, click, retention and value conversion links of the push message where the user churn exceeds a preset threshold; The first generation module is used to generate multiple candidate push strategies for improving the loss process; wherein, the candidate push strategy includes push content and push time; The second generation module is used to distribute and test the multiple candidate push strategies, obtain feedback data corresponding to each candidate push strategy, and adjust the traffic allocation ratio of each candidate push strategy according to the feedback data to generate a target push strategy.

[0007] Thirdly, embodiments of the present invention provide an electronic device, including: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the steps of the method described in the first aspect above.

[0008] Fourthly, embodiments of the present invention provide a computer-readable storage medium for storing computer-executable instructions that, when executed by a processor, implement the steps of the method described in the first aspect above.

[0009] Fifthly, embodiments of the present invention provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0010] The at least one technical solution provided by the embodiments of the present invention can achieve the following technical effects: In this embodiment of the invention, firstly, full-link data corresponding to the target user can be obtained. This full-link data includes push funnel data and commercial value data. The push funnel data characterizes the delivery, display, and click status of push messages, while the commercial value data characterizes the retention and value conversion status of target users in response to push messages. Then, diagnostic analysis is performed on the full-link data to identify the churn points. The churn points are those in the delivery, display, click, retention, and value conversion stages of push messages where user churn exceeds a preset threshold. Multiple candidate push strategies are generated to improve these churn points. Each candidate push strategy includes push content and push time. Distribution tests are conducted on the multiple candidate push strategies, and feedback data corresponding to each candidate push strategy is obtained. Based on the feedback data, the traffic allocation ratio of each candidate push strategy is adjusted to generate the target push strategy.

[0011] This invention, by acquiring end-to-end data including push funnel data and commercial value data, comprehensively reflects the actual value generated by push notifications, effectively addressing the core problems of limited information and inaccurate evaluation in existing push notification performance assessments. Based on the identified loss points determined through diagnostic analysis of the end-to-end data, candidate push strategies are generated. The traffic allocation ratio of each candidate strategy is adjusted through distribution testing and feedback data, automatically completing the determination and adjustment of push notifications without relying on manual intervention by operations personnel. This effectively achieves timely and efficient optimization of push notifications. Attached Figure Description

[0012] Figure 1 This is one of the flowcharts illustrating the push strategy generation method provided in an embodiment of the present invention; Figure 2 This is a second flowchart illustrating the push strategy generation method provided in an embodiment of the present invention. Figure 3 A schematic diagram of the module composition of the push strategy generation device 300 provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0014] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0015] Please see Figure 1 , Figure 1 This is one of the flowcharts illustrating the push strategy generation method provided in this embodiment of the invention, such as... Figure 1 As shown, the method includes the following steps: Step 102: Obtain the full-link data corresponding to the target user. The full-link data includes push funnel data and commercial value data. Push funnel data is used to characterize the delivery, display and click status of push messages, while commercial value data is used to characterize the retention and value conversion status of target users for push messages.

[0016] Step 104: Perform diagnostic analysis on the entire data chain to identify the loss links. The loss links are the links in the delivery, display, click, retention and value conversion of push messages where the user churn exceeds the preset threshold.

[0017] Step 106: Generate multiple candidate push strategies for improving the loss process. The candidate push strategies include push content and push time.

[0018] Step 108: Distribute and test multiple candidate push strategies, obtain feedback data for each candidate push strategy, adjust the traffic allocation ratio of each candidate push strategy based on the feedback data, and generate the target push strategy.

[0019] In this embodiment of the invention, firstly, full-link data corresponding to the target user can be obtained. This full-link data can include push funnel data and commercial value data. Specifically, the push funnel data can be used to characterize the delivery, display, and click status of push messages, while the commercial value data can be used to characterize the retention and value conversion status of the target user in response to push messages.

[0020] In one example, a complete push notification can sequentially go through multiple stages, including audience size extraction, audience feature matching, message sending, user receipt, user display, user click, retention, and value conversion. Among these stages, message sending, user receipt, user display, and user click are considered the shallower stages of the push notification process. Push funnel data can be used to characterize the delivery, display, and click status of push messages at these shallow stages, reflecting the target user's immediate response to the push message. Retention and value conversion are considered the deeper stages of the push notification process. Commercial value data can be used to characterize the target user's retention and value conversion of push messages, reflecting the value generated by the push message over a longer period. Retention can include next-day retention rate, long-term retention rate, etc., while value conversion can include in-app purchase amount, ad impressions, etc. For example, value conversion can manifest as monetization.

[0021] In mobile application push operations, push funnel data is often collected by the push side, while commercial value data is often located in a separate value conversion database. The two are scattered in different data sources and are difficult to correspond to the same push.

[0022] By acquiring end-to-end data, including push notification funnel data and monetization value data, the shallow and deep stages of push notifications can be mapped together. Push notification funnel data, belonging to the shallow funnel category, can include metrics such as message volume, delivery volume, impressions, and clicks. Monetization value data, belonging to the deep monetization value category, can include metrics such as next-day retention rate, long-term retention rate, in-app purchase amount, and ad impressions. In this way, end-to-end data can cover the entire chain from push notification delivery to retention and value conversion.

[0023] The delivery, display, and click of push messages can be understood as the push message reaching the user, being displayed to the user, and being clicked by the user, respectively. For example, the data access layer can connect to the client-side event tracking system, push channel gateway, and backend value conversion database to clean and align scattered multi-source data, obtaining end-to-end data under a unified identifier space. By acquiring end-to-end data, the complete value generated by a single push can be reflected, providing a data foundation for subsequent loss diagnosis, candidate push strategy generation, and traffic allocation adjustment.

[0024] In one example, when acquiring end-to-end data for a target user, client-side event tracking logs can be obtained. These logs record the target user's display and click events related to push notifications, as well as their in-app behavior events. In-app behavior events can include in-app purchase events and ad display events. Then, global identifiers can be injected into the push notification body and the client-side event tracking logs. These global identifiers can include audience segment identifiers, task identifiers, and timestamps. Based on these global identifiers, push notification funnel data can be correlated with monetization value data to obtain end-to-end data.

[0025] In this example, client-side event tracking logs can record in-app behavior events such as display events, click events, in-app purchase events, and ad display events that occur on the target user's client. Display events and click events correspond to push notification funnel data, while in-app purchase events and ad display events correspond to monetization value data. When sending push notifications to target users, a global identifier can be included in the message body and written to the client-side event tracking logs. This allows all user behavior events on the client, such as display, click, in-app purchase, and ad display, to be associated with this push notification through this global identifier. The global identifier can include an audience segment identifier, a task identifier, and a timestamp. The audience segment identifier identifies the target audience for this push notification; the task identifier identifies the push notification task; and the timestamp identifies the time of the push notification. Based on the global identifier, push notification funnel data can be associated with monetization value data, meaning that shallow delivery, display, and click data, along with deeper retention and value conversion data, can be mapped to the same push notification using the same global identifier, thus obtaining end-to-end data.

[0026] The message body of a push notification can be the main text of the push message. A global identifier can be included in the message body, and client-side event logs can record the target user's behavioral events on the client. For example, after a push notification is sent, the pusher can collect the delivery volume, impressions, and clicks of that batch of pushes to obtain push funnel data. After a user clicks the push notification and enters the application, the global identifier can be used to link in-app behavior logs and settlement data. Within a longer observation window, such as 24 to 72 hours, the next-day retention rate, in-app purchase conversion amount, and in-app ad display frequency of that batch of users can be continuously tracked and attributed to obtain commercial value data. Furthermore, the push funnel data can be correlated with the commercial value data to obtain end-to-end data.

[0027] Specifically, the in-app behavior logs are the behavior logs generated after a user clicks on a push notification and enters the app; the settlement data includes in-app purchase and ad display settlement data; the client-side event tracking system can collect target user behavior events on the client in real time; the push channel gateway can be used to send push messages; the backend value conversion database can be used to store value conversion data such as in-app purchases and ad displays; and the client-side event tracking logs are logs collected by the client-side event tracking system to record target user behavior events on the client. In this way, even if a user performs a deep-level behavior such as an in-app purchase or ad display a long time after clicking on the push notification, these deep-level behaviors can still be attributed to the corresponding push notification through global identifiers, thus obtaining end-to-end data.

[0028] In this embodiment of the invention, diagnostic analysis can be performed on the entire data chain to identify the loss-making stages. These loss-making stages can be the stages of push message delivery, display, click, retention, and value conversion where user churn exceeds a preset threshold.

[0029] Specifically, from delivery to retention and value conversion, push notifications go through the stages of delivery, display, click, retention, and value conversion. At each stage, a portion of users churns. By performing diagnostic analysis on the entire link data, we can identify the stages where user churn exceeds a preset threshold—the "loss stage." User churn can be the reduction in the number of users between adjacent stages. The diagnostic analysis involves analyzing the entire link data to determine the user churn situation at each stage. This analysis can utilize the natural language understanding and logical reasoning capabilities of a large language model. The preset threshold is a pre-set value used to judge whether user churn is abnormal. The loss stage is determined by comparing the user churn at each stage with the preset threshold. The loss stage is the stage in the push notification's delivery, display, click, retention, and value conversion stages where abnormal user churn occurs. The loss stage reflects at which stage the push notification experienced a large number of user churns; for example, the loss stage could be the click stage or the value conversion stage. The specific stage can be determined based on the user churn at each stage. It's understandable that only by first identifying the loss-making process can improvements be made to address that loss-making process when generating candidate push strategies.

[0030] In one example, before performing diagnostic analysis on the end-to-end data, the system can first select the first major language model that matches the routing and logical reasoning type, and the second major language model that matches the content generation type, from among several pre-set major language models, based on the type of processing task corresponding to the end-to-end data. The first major language model and the second major language model are different.

[0031] Specifically, diagnostic analysis of end-to-end data can be classified as a logical reasoning type of processing task, while the generation of candidate push strategies can be classified as a content generation type of processing task. Based on the type of processing task, the first major language model that matches the logical reasoning type can be used for diagnostic analysis, and the second major language model that matches the content generation type can be used for candidate push strategy generation. Furthermore, the first major language model and the second major language model can be different models, thereby enabling dynamic routing to different major language models based on the task type.

[0032] The pre-built large language models can be multiple pre-deployed large language models, which can be models trained on large-scale corpora and possess natural language understanding and generation capabilities. The type of processing task can be determined based on the characteristics of the processing task. For example, the characteristics of a diagnostic analysis task can be causal attribution and logical deduction across multi-dimensional data; the characteristics of a candidate push strategy generation task can be multi-style creative brainstorming and copywriting. In addition, the type of processing task can also be determined in conjunction with response time, computing power cost, or token consumption limit. Response time can be the time required for the large language model to process the task, computing power cost can be the computing resource cost required for the large language model to process the task, and token consumption limit can be the limit on the number of tokens consumed by the large language model when processing the task.

[0033] Logical reasoning tasks can correspond to diagnostic analysis tasks; content generation tasks can correspond to candidate push strategy generation tasks. For example, logical reasoning tasks can be routed to large language models with logical reasoning capabilities; content generation tasks can be routed to large language models with content generation capabilities. By routing different large language models to different types of processing tasks, diagnostic analysis can be performed by models suitable for logical reasoning, while candidate push strategy generation can be performed by models suitable for content generation.

[0034] In one example, when performing diagnostic analysis on end-to-end data, the first major language model can be used. The first major language model obtained from the routing can perform the diagnostic analysis on the end-to-end data, thus allowing the major language model with logical reasoning capabilities to complete the diagnostic analysis.

[0035] In another example, when performing diagnostic analysis on end-to-end data, the end-to-end data can be compared with historical baseline data to identify the loss links. Root cause analysis of these loss links can then be performed to determine the corresponding improvement directions. Historical baseline data refers to data from historical push tasks of the same type as the push message. In other words, when identifying loss links, the end-to-end data of the current push task can be compared with data from historical similar push tasks. Based on the comparison results, the loss links where user churn exceeds a preset threshold can be identified. Then, root cause analysis can be performed on these loss links to determine the corresponding improvement directions. Root cause analysis can deduce the causes of the loss links, and improvement directions can be directions for improving the loss links. Historical baseline data can be data from historical similar push tasks and can be compared with the end-to-end data of the current push task to identify links in the current push task where user churn is abnormal. For example, the end-to-end data of a certain push task can be compared with data from historical similar push tasks to identify the loss links where user churn exceeds a preset threshold, and then root cause analysis can be performed on these loss links.

[0036] For example, an anomaly can be identified where click-through rates meet targets but in-app purchase conversions are zero and average ad impressions per user are extremely low. This suggests that the push notification copy contains exaggerated claims, causing users to feel misled and quickly exit, resulting in clickbait-style clickbait tactics that negatively impact user retention. Therefore, the improvement direction can be determined by providing push notification copy with substantial content value. Another example is targeting users who are highly active during their morning commute. The root cause can be identified as a mismatch between the push notification time and their active time. Therefore, the improvement direction can be determined by matching the user's morning reading preferences and adjusting the delivery timing. The improvement direction can be determined based on the results of root cause analysis, guiding the generation of candidate push notification strategies. Through root cause analysis, not only can the stage at which the loss occurs be identified, but also the corresponding improvement direction can be determined.

[0037] In this embodiment of the invention, multiple candidate push strategies can be generated to improve the user churn stage. Each candidate push strategy can include push content and push time. That is, for a identified churn stage, multiple candidate push strategies can be generated to improve that stage. Each candidate push strategy can include push content and push time. The push content can be used to improve the user churn problem corresponding to the churn stage, and the push time can be used to specify the delivery time of the candidate push strategy. The push content can be the text of the push message, and the push time can be the delivery time of the push message. Generating multiple candidate push strategies is for the purpose of subsequently determining the target push strategy from among the multiple candidate push strategies.

[0038] In one example, when generating multiple candidate push strategies to improve the loss process, these strategies can be generated based on the improvement direction. Specifically, corresponding candidate push strategies can be generated based on the improvement direction determined by root cause analysis, ensuring that the candidate push strategies match the root causes of the loss process.

[0039] In one example, when generating multiple candidate push strategies to improve the loss process, a second large language model can be used to generate these strategies. Specifically, the second large language model obtained from the routing can perform the generation of candidate push strategies, thus the large language model with content generation capabilities completes the generation of candidate push strategies.

[0040] In one example, when generating multiple candidate push strategies to improve the loss reduction process, the target user's demographic characteristics can be determined based on the target user's user profile data. These demographic characteristics can include the target user's static attributes and dynamic behavioral preferences. These characteristics can then be converted into prompts, and based on these prompts, multiple candidate push strategies matching the target user's demographic characteristics can be generated.

[0041] Specifically, the characteristics of target users can be determined based on their user profile data. Static attributes within these characteristics can be attributes that remain unchanged over a relatively long period, such as device type and registration duration. Dynamic behavioral preferences can be preferences that change over time or in behavior, such as activity habits during specific time periods and content preferences. These characteristics can be converted into prompt words, and then, based on these prompt words, multiple candidate push strategies matching the target user's characteristics can be generated. The user profile characteristics can represent the target user's attributes and preferences. Converting these characteristics into prompt words involves organizing static attributes and dynamic behavioral preferences into prompt words in natural language. User profile data can be pre-collected data related to the target user. Prompt words can be natural language prompts generated based on user characteristics, used to guide a large language model in generating candidate push strategies. For example, for users with churn warnings due to several consecutive days of inactivity, a recall message providing substantial content value can be generated; for users who are highly active during their morning commute, a push message matching their morning news preferences can be generated. It's understandable that the characteristics of a user group with no activity for several consecutive days (as indicated by churn warnings) differ from those of a user group that is highly active during the morning commute. Therefore, the candidate push strategies generated for these two groups will also differ, enabling differentiated push notifications for different user groups. In other words, for target users with different user characteristics, a candidate push strategy matching their characteristics is generated. Furthermore, a lightweight parsing model can be used to convert the push text generated from prompt words into specific delivery time parameters, so that the candidate push strategy includes both the push content and the push time. Here, the lightweight parsing model can be a model used to parse natural language text into structured parameters.

[0042] In this embodiment of the invention, multiple candidate push strategies can be distributed and tested to obtain feedback data corresponding to each candidate push strategy. Based on the feedback data, the traffic allocation ratio of each candidate push strategy can be adjusted to generate a target push strategy. Specifically, multiple candidate push strategies can be distributed to users for testing, feedback data corresponding to each candidate push strategy can be obtained, and then the traffic allocation ratio of each candidate push strategy can be adjusted based on the feedback data to tilt traffic towards the candidate push strategy with better feedback data, thereby generating a target push strategy. The distribution test can involve distributing multiple candidate push strategies to different subsets of users for testing, and obtaining feedback data corresponding to each candidate push strategy through the distribution test. The feedback data can be used to characterize the push effect of each candidate push strategy. The feedback data can be the behavioral data of target users towards each candidate push strategy obtained during the distribution test. The target push strategy can be the push strategy determined from multiple candidate push strategies, and the traffic allocation ratio can be the proportion of traffic allocated to each candidate push strategy. Adjusting the traffic allocation ratio can tilt traffic towards the candidate push strategy with better feedback data. For example, multiple candidate push strategies can first be distributed to a portion of users for small-scale testing, and the traffic allocation ratio of each candidate push strategy can be adjusted based on the feedback data obtained from the test. For example, a multi-armed slot machine algorithm can be used to distribute multiple candidate push strategies to different user subsets. A small proportion of exploratory traffic can be allocated to conduct parallel testing of multiple candidate push strategies. Based on the feedback data, the comprehensive evaluation value of each candidate push strategy can be calculated, and the winning probability distribution corresponding to each candidate push strategy can be updated. Then, based on the updated winning probability distribution, the traffic of inefficient candidate push strategies can be automatically reduced, and the main traffic can be tilted to the candidate push strategy with the highest comprehensive evaluation value, thereby generating the target push strategy. By obtaining feedback data through distribution testing and adjusting the traffic allocation ratio based on the feedback data, the target push strategy can be determined from multiple candidate push strategies.

[0043] In one example, feedback data includes data characterizing the clicks, retention, and conversion rates of target users for each candidate push strategy. When adjusting the traffic allocation ratio of each candidate push strategy based on the feedback data, a comprehensive evaluation value for each candidate push strategy can be determined based on the feedback data, and the traffic allocation ratio of each candidate push strategy can be adjusted based on the comprehensive evaluation value. The comprehensive evaluation value can be used to characterize the comprehensive revenue corresponding to the candidate push strategy. The comprehensive revenue can be a joint reward determined based on clicks, retention, and conversion rates. Correspondingly, the feedback data can include clicks, retention, and conversion rates, and the joint reward, i.e., the comprehensive revenue, can be determined based on these factors. The comprehensive evaluation value can be used to characterize the comprehensive revenue corresponding to the candidate push strategy. Based on the comprehensive evaluation value, the traffic allocation ratio of each candidate push strategy can be adjusted to tilt traffic towards candidate push strategies with higher comprehensive evaluation values. The joint reward can be understood as combining data from multiple dimensions such as clicks, retention, and conversion rates into a single reward value; the comprehensive revenue can be a value determined by combining feedback data from multiple dimensions such as clicks, retention, and conversion rates; and the comprehensive evaluation value can be a single value used to characterize the comprehensive revenue corresponding to the candidate push strategy. For example, click metrics can include click-through rate (CTR), retention metrics can include retention rate, and conversion metrics can include in-app purchase amount and ad impressions. Clicks, retention, and conversions can be multiple dimensions representing the effectiveness of candidate push strategies. It can be understood that the feedback data includes not only shallow data such as clicks but also deeper data such as retention and conversions. This allows for the evaluation and adjustment of candidate push strategies based on overall returns, avoiding evaluation based solely on shallow data like clicks and improving the accuracy of candidate push strategy evaluation.

[0044] In one example, for instance, such as Figure 2As shown, the overall process of the push strategy generation method can include three steps: Step 1 is the data access module, which can collect user characteristics, including static profiles (i.e., static attributes) and dynamic profiles (i.e., dynamic behavioral preferences), and connect the entire data chain, including exposure, clicks, retention, in-app purchases, and ad frequency; Step 2 is the AIAgent strategy center, which can dynamically route to the inference model to perform root cause diagnosis of the entire chain of failures and output a structured diagnostic report; Step 3 is the content and distribution module, which can dynamically route to the generation model to produce user-specific strategy packages, and combine reinforcement learning to conduct small-scale testing and full-scale traffic allocation. Step 1 can pass structured context prompts to Step 2, and Step 2 can pass diagnostic reports and optimization instructions to Step 3. Real-time conversion feedback data can be fed back to Step 1 through a closed loop. The above three steps are sequentially connected, and the output of the previous step can be used as the input of the next step, thus forming a closed-loop process from data access, diagnosis, generation to distribution optimization. Through this closed-loop process, continuous optimization of the push strategy can be achieved.

[0045] In this embodiment of the invention, firstly, full-link data corresponding to the target user can be obtained. This full-link data includes push funnel data and commercial value data. The push funnel data characterizes the delivery, display, and click status of push messages, while the commercial value data characterizes the retention and value conversion status of target users in response to push messages. Then, diagnostic analysis is performed on the full-link data to identify the churn points. The churn points are those in the delivery, display, click, retention, and value conversion stages of push messages where user churn exceeds a preset threshold. Multiple candidate push strategies are generated to improve these churn points. Each candidate push strategy includes push content and push time. Distribution tests are conducted on the multiple candidate push strategies, and feedback data corresponding to each candidate push strategy is obtained. Based on the feedback data, the traffic allocation ratio of each candidate push strategy is adjusted to generate the target push strategy.

[0046] This invention, by acquiring end-to-end data including push funnel data and commercial value data, comprehensively reflects the actual value generated by push notifications, effectively addressing the core problems of limited information and inaccurate evaluation in existing push notification performance assessments. Based on the identified loss points determined through diagnostic analysis of the end-to-end data, candidate push strategies are generated. The traffic allocation ratio of each candidate strategy is adjusted through distribution testing and feedback data, automatically completing the determination and adjustment of push notifications without relying on manual intervention by operations personnel. This effectively achieves timely and efficient optimization of push notifications.

[0047] Figure 3 The push strategy generation device 300 shown can achieve Figure 1 The method described in the embodiment achieves the same technical effect, and can be specifically referred to in the above description. Figure 1 The description of the push strategy generation method in the illustrated embodiment will not be repeated here. The push strategy generation device 300 includes: The acquisition module 301 is used to acquire the full-link data corresponding to the target user; the full-link data includes push funnel data and commercial value data; wherein, the push funnel data is used to characterize the delivery, display and click status of push messages; the commercial value data is used to characterize the retention and value conversion status of the target user in response to the push messages; The determination module 302 is used to perform diagnostic analysis on the full-link data to determine the loss links; wherein, the loss links are the links in the delivery, display, click, retention and value conversion links of the push message where the user churn exceeds a preset threshold; The first generation module 303 is used to generate multiple candidate push strategies for improving the loss process; wherein, the candidate push strategy includes push content and push time; The second generation module 304 is used to perform distribution tests on the multiple candidate push strategies, obtain feedback data corresponding to each candidate push strategy, and adjust the traffic allocation ratio of each candidate push strategy according to the feedback data to generate a target push strategy.

[0048] Optionally, the device further includes ( Figure 3 (not shown in the image) The routing module 305 is used to, before performing diagnostic analysis on the end-to-end data, route from a set of multiple pre-set large language models a first large language model that matches the logical reasoning type and a second large language model that matches the content generation type, based on the type of the processing task corresponding to the end-to-end data; wherein the first large language model and the second large language model are different. The determining module 302 is used for: The first major language model is used to perform diagnostic analysis on the entire data chain; The first generation module 303 is used for: The second major language model is used to generate the multiple candidate push strategies.

[0049] Optionally, the acquisition module 301 is used for: Obtain client-side event tracking logs; wherein, the client-side event tracking logs are used to record the target user's display events, click events, and in-app behavior events in response to the push message; the in-app behavior events include in-app purchase events and ad display events; A global identifier is injected into the message body of the push message and the client-side event logs; the global identifier includes a user group identifier, a task identifier, and a timestamp. Based on the global identifier, the push funnel data is associated with the commercial value data to obtain the full-link data.

[0050] Optionally, the feedback data includes data characterizing the target user's clicks, retention, and value conversion for each of the candidate push strategies; The second generation module 304 is used for: Based on the feedback data, a comprehensive evaluation value for each candidate push strategy is determined, and the traffic allocation ratio for each candidate push strategy is adjusted according to the comprehensive evaluation value; wherein, the comprehensive evaluation value is used to characterize the comprehensive revenue corresponding to the candidate push strategy, and the comprehensive revenue is a joint reward determined based on the click, retention and value conversion situation.

[0051] Optionally, the first generation module 303 is used to: Based on the user profile data of the target users, the demographic characteristics of the target users are determined; wherein, the demographic characteristics include the static attributes and dynamic behavioral preferences of the target users; The characteristics of the population are converted into prompt words; Based on the prompt words, multiple candidate push strategies are generated that match the demographic characteristics of the target user.

[0052] Optionally, the determining module 302 is used to: The entire link data is compared with historical baseline data to determine the lost link; wherein, the historical baseline data is the data of historical push tasks of the same type as the push task corresponding to the push message; Root cause analysis was performed on the aforementioned loss process to determine the corresponding improvement directions. The first generation module 303 is used for: Based on the improvement direction, multiple candidate push strategies are generated to improve the loss process.

[0053] Figure 4 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present invention. Please refer to it. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0054] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0055] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0056] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a push strategy generation device at the logical level. The processor executes the program stored in memory and specifically performs the following operations: Acquire full-link data corresponding to the target user; the full-link data includes push funnel data and commercial value data; wherein, the push funnel data is used to characterize the delivery, display and click status of push messages; the commercial value data is used to characterize the retention and value conversion status of the target user in response to the push messages; Diagnose and analyze the entire data chain to identify the loss links; wherein, the loss links are the links in the delivery, display, click, retention and value conversion of the push message where the user churn exceeds a preset threshold; Multiple candidate push strategies are generated to improve the loss process; wherein, the candidate push strategy includes push content and push time; The multiple candidate push strategies are distributed and tested to obtain feedback data corresponding to each candidate push strategy. Based on the feedback data, the traffic allocation ratio of each candidate push strategy is adjusted to generate the target push strategy.

[0057] The above is as described in the present invention. Figure 1The push strategy generation method disclosed in the illustrated embodiments can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in one or more embodiments of the present invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in one or more embodiments of the present invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0058] The electronic device can also perform Figure 1 The method for generating the push strategy will not be described in detail here.

[0059] This invention also provides a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 1 or Figure 2 The method of the illustrated embodiment will not be described in detail here.

[0060] This invention also provides a computer program product stored in a storage medium and executed by at least one processor to implement... Figure 1 or Figure 2 The method of the illustrated embodiment will not be described in detail here.

[0061] Of course, in addition to the software implementation, the electronic device of the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0062] In summary, the above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of the present invention should be included within the scope of protection of one or more embodiments of the present invention.

[0063] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0064] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined in this embodiment of the invention, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0065] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0066] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A method for generating a push strategy, characterized in that, include: Acquire full-link data corresponding to the target user; the full-link data includes push funnel data and commercial value data; wherein, the push funnel data is used to characterize the delivery, display and click status of push messages; the commercial value data is used to characterize the retention and value conversion status of the target user in response to the push messages; Diagnose and analyze the entire data chain to identify the loss links; wherein, the loss links are the links in the delivery, display, click, retention and value conversion of the push message where the user churn exceeds a preset threshold; Multiple candidate push strategies are generated to improve the loss process; wherein, the candidate push strategy includes push content and push time; The multiple candidate push strategies are distributed and tested to obtain feedback data corresponding to each candidate push strategy. Based on the feedback data, the traffic allocation ratio of each candidate push strategy is adjusted to generate the target push strategy.

2. The method according to claim 1, characterized in that, Before performing diagnostic analysis on the end-to-end data, the method further includes: Based on the type of processing task corresponding to the full-link data, from multiple pre-set large language models, the first large language model that matches the routing and logical reasoning type, and the second large language model that matches the content generation type; wherein, the first large language model and the second large language model are different; The diagnostic analysis of the entire data chain includes: The first major language model is used to perform diagnostic analysis on the entire data chain; The generation of multiple candidate push strategies for improving the loss process includes: The second major language model is used to generate the multiple candidate push strategies.

3. The method according to claim 1, characterized in that, The acquisition of the full-link data corresponding to the target user includes: Obtain client-side event tracking logs; wherein, the client-side event tracking logs are used to record the target user's display events, click events, and in-app behavior events in response to the push message; the in-app behavior events include in-app purchase events and ad display events; A global identifier is injected into the message body of the push message and the client-side event logs; the global identifier includes a user group identifier, a task identifier, and a timestamp. Based on the global identifier, the push funnel data is associated with the commercial value data to obtain the full-link data.

4. The method according to claim 1, characterized in that, The feedback data includes data characterizing the target user's clicks, retention, and value conversion for each of the candidate push strategies; The step of adjusting the traffic allocation ratio of each candidate push strategy based on the feedback data includes: Based on the feedback data, a comprehensive evaluation value for each candidate push strategy is determined, and the traffic allocation ratio for each candidate push strategy is adjusted according to the comprehensive evaluation value; wherein, the comprehensive evaluation value is used to characterize the comprehensive revenue corresponding to the candidate push strategy, and the comprehensive revenue is a joint reward determined based on the click, retention and value conversion situation.

5. The method according to claim 1, characterized in that, The generation of multiple candidate push strategies for improving the loss process includes: Based on the user profile data of the target users, the demographic characteristics of the target users are determined; wherein, the demographic characteristics include the static attributes and dynamic behavioral preferences of the target users; The characteristics of the population are converted into prompt words; Based on the prompt words, multiple candidate push strategies are generated that match the demographic characteristics of the target user.

6. The method according to claim 1, characterized in that, The diagnostic analysis of the entire data chain includes: The entire link data is compared with historical baseline data to determine the lost link; wherein, the historical baseline data is the data of historical push tasks of the same type as the push task corresponding to the push message; Root cause analysis was performed on the aforementioned loss process to determine the corresponding improvement directions. The generation of multiple candidate push strategies for improving the loss process includes: Based on the improvement direction, multiple candidate push strategies are generated to improve the loss process.

7. A push strategy generation device, characterized in that, include: The acquisition module is used to acquire full-link data corresponding to the target user; the full-link data includes push funnel data and commercial value data; wherein, the push funnel data is used to characterize the delivery, display and click status of push messages; the commercial value data is used to characterize the retention and value conversion status of the target user in response to the push messages; The determination module is used to perform diagnostic analysis on the entire link data to determine the loss links; wherein, the loss links are the links in the delivery, display, click, retention and value conversion links of the push message where the user churn exceeds a preset threshold; The first generation module is used to generate multiple candidate push strategies for improving the loss process; wherein, the candidate push strategy includes push content and push time; The second generation module is used to distribute and test the multiple candidate push strategies, obtain feedback data corresponding to each candidate push strategy, and adjust the traffic allocation ratio of each candidate push strategy according to the feedback data to generate a target push strategy.

8. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store computer-executable instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 6.

Images