Marketing operation canvas batch decision-making method based on big data

Abstract

The invention discloses a marketing operation canvas batch decision-making method based on big data. According to the method, a breadth-first search algorithm is adopted to disassemble strategy nodes of a marketing operation canvas into L1 to Ln rule layers according to a hierarchical relationship, and a rule set is converted into executable SQL statements through a pre-compiling technology and stored in a distributed cache system. According to the system, an Apache Spark Structured Streaming distributed flow calculation engine is used for executing rule calculation according to a hierarchical sequence, a calculation result of an (n-1) th layer is used as input in an nth layer rule, and parallel processing is achieved through intersection operation. And a calculation result is converted into a key value pair format of the user ID and the hit instruction set, and is output to a Kafka message queue after distributed aggregation. In addition, the system also integrates the Flink CDC technology to realize incremental calculation optimization, and only the downstream hierarchy of the change strategy node is recalculated, so that the resource utilization efficiency is remarkably improved. The technical scheme is suitable for large-scale precision marketing scenes of industries such as finance and e-commerce.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data analysis and intelligent marketing decision, and particularly relates to a marketing operation canvas batch decision method based on big data. BACKGROUND

[0002] As a core tool for precise marketing of enterprises, the marketing operation canvas is widely used in customer touch scenarios in the financial, e-commerce, communication and other industries. The existing marketing decision system generally adopts a user dimension-based layer-by-layer traversal calculation mode, that is, a complete decision tree traversal process is executed for each user respectively, and an individualized marketing strategy is generated through a serialized processing mode. This traditional technical architecture faces large-scale user groups, and the system needs to calculate the path selection of each user in the decision tree independently, resulting in a linear growth feature of O(nxm) of the calculation complexity, where n represents the total number of users and m represents the total number of strategy nodes. As the user scale of enterprises expands from millions to tens of millions, the calculation bottleneck problem of the traditional scheme is increasingly prominent, and the time consumption of single decision calculation is extended from seconds to minutes or even hours, which seriously restricts the real-time response capability of marketing activities.

[0003] The traditional marketing decision system exposes three core technical defects when dealing with large-scale user groups, which seriously affects the practicability and economic benefits of the system. First, the problem of low calculation efficiency, the existing system adopts a single-user traversal mechanism, and each user needs to independently execute a complete strategy tree calculation process, which cannot fully utilize the parallel processing capability of modern distributed computing framework. Actual test data shows that the decision response delay in the scenario of tens of millions of users is generally more than 5 seconds, which is far from meeting the business needs of real-time marketing scenarios. Second, the problem of resource waste is prominent, when the marketing strategy is adjusted, the traditional system needs to re-execute full calculation for all users, even if the strategy change only affects a few nodes, the decision path of all users must be recalculated. This full recalculation mode leads to a large amount of redundant consumption of computing resources, and actual monitoring data shows that a single strategy iteration usually occupies more than 85% of CPU resources. Third, the lack of real-time performance, the existing system generally adopts a synchronous serial message pushing mechanism, and the decision result needs to be transmitted to each marketing channel one by one, and the transmission delay is more than 500 milliseconds, which is easy to form a system bottleneck in high-concurrency scenarios, directly affecting the touch effect of marketing activities.

[0004] In actual industry application scenarios, the limitations of traditional marketing decision-making technology have caused serious business impact and operational cost problems. In the business scenario of reaching tens of millions of users per day, the traditional technical solution forces the enterprise to expand the server cluster size by more than 3 times, and can only barely maintain the decision-making processing time of several hours, greatly increasing the investment cost and operation complexity of the IT infrastructure. E-commerce platforms face even more severe challenges during large-scale promotional activities. The user loss rate due to decision-making delay is as high as 12%, directly affecting the marketing conversion effect and enterprise revenue. More critically, the flexibility of strategy iteration is a problem. The strategy adjustment of the traditional system requires downtime maintenance, which affects business continuity for an average of 2 hours each time, making it impossible for the enterprise to quickly respond to market changes and competitive situations. These technical constraints not only affect the effectiveness of marketing activities, but also limit the competitive advantage of the enterprise in the process of digital transformation, and there is an urgent need for a new technical solution that can balance computing efficiency, resource optimization, and real-time response. SUMMARY

[0005] Based on the above purpose, the present application provides a marketing management canvas batch decision-making method based on big data, comprising the following steps: Receiving a marketing management canvas configuration containing a root management plan node and multiple strategy nodes, the strategy nodes having a hierarchical relationship, and each node being configured with guest group label intersection and union rules; Using a breadth-first search algorithm to decompose the strategy nodes into L1 to Ln rule layers according to the hierarchical relationship, wherein nodes in the same layer can be processed in parallel, and there is a data dependency relationship between different layers; Precompiling the rule set of each strategy node into an executable SQL statement and storing it in a distributed cache system; Executing rule calculation in hierarchical order through a distributed stream computing engine, wherein the nth layer rule takes the calculation result of the n-1th layer as the input data set, and generates the user filtering result of the current layer through intersection operation; Converting the calculation result into a key-value pair format of user ID and hit instruction set, and outputting it to a message queue system after distributed aggregation processing; Setting instruction priority according to user value level, distributing instructions to different message queue partitions through a custom partitioning strategy, and realizing the push of batch decision-making instructions.

[0006] Further, the step of decomposing the strategy nodes according to the hierarchical breadth-first search algorithm specifically includes: Starting from the root management plan node, a mapping table of node ID and parent-child relationship is established; Traverse all strategy nodes in hierarchical order through a queue data structure, and collect nodes in the same layer into the same rule layer; Bitwise operation is used to mark the processed nodes to avoid repeated traversal, ensuring that each policy node is assigned to only one unique rule level.

[0007] Further, the rule set pre-compilation step specifically includes: Parse the customer tag intersection and union rules in the policy node, extract the condition expressions and logical operators; Convert the condition expressions into the WHERE clause syntax of standard SQL, and perform bracket priority processing on the composite conditions; Generate a complete SQL statement containing table joins, filter conditions, and projection fields, and perform execution plan optimization; Store the pre-compiled SQL statements in the Redis distributed cache system in the form of key-value pairs, supporting dynamic updating and version management of rules.

[0008] Further, the distributed stream computing engine uses the Apache Spark Structured Streaming framework, and the specific computing process includes: Store user attribute data in HDFS shards according to user ID hash values, with each shard size controlled within the range of 128MB to 256MB; Read the pre-compiled SQL statements in the order of rule levels, and perform associated queries between the current level rules and the calculation results of the previous level; Perform predicate pushdown and column pruning optimization on the SQL execution plan through the Catalyst optimizer to reduce data transmission volume; Use micro-batch processing mode to execute computing tasks, with a batch interval set to 5 seconds to 30 seconds to balance computing latency and system throughput.

[0009] Further, the above method further includes an incremental calculation optimization step: Real-time monitor changes in policy node configuration through Flink CDC technology; Build a dependency graph between policy nodes, and when a node change is detected, automatically identify the affected downstream level range; Only re-execute rule compilation and calculation processing for the affected policy nodes, and retain the historical calculation results of unchanged nodes; Merge and update the incremental calculation results with the stock results to achieve quick effect of policy changes.

[0010] Further, the custom partitioning strategy specifically includes: Divide the instructions into three priority queues according to user value levels, with VIP user instructions assigned the highest priority; Implement a Kafka custom partitioner based on user ID hash values to ensure that multiple instructions for the same user are routed to the same partition; The number of partitions is set to 2 to 4 times the number of consumer nodes, and load balancing between partitions is achieved through a polling algorithm; In the consumer end, a batch consumption mode is adopted, and the size of a single consumption batch is set to 100 to 500 instructions. The same user instructions are de-duplicated and merged.

[0011] Further, the method optimizes the computational complexity from the traditional O(nxm) to O(kxlogn), where n is the total number of users, m is the total number of strategy nodes, k is the average level depth, and k is in the range of 3 to 5.

[0012] A marketing management canvas batch decision system comprising the above method, comprising: A decision tree strategy node rule engine configured to parse the marketing management canvas and perform hierarchical decomposition through a breadth-first search algorithm; A batch calculation engine using Apache Spark Structured Streaming framework to realize distributed parallel computing; An instruction pushing optimization module, which realizes low-latency instruction distribution through Kafka message queue and custom partition strategy; A distributed storage system, including an HDFS cluster for storing user data and a Redis cluster for caching precompiled rules; An incremental calculation module based on Flink CDC technology to realize real-time monitoring and incremental processing of strategy changes.

[0013] The beneficial effects of the present application are: The present application uses a breadth-first search algorithm to decompose a complex marketing decision tree by level, and combines the parallel processing capability of a distributed stream calculation engine to optimize the traditional O(nxm) computational complexity to O(kxlogn), where k is the average level depth and is usually controlled within the range of 3 to 5 levels. This algorithm optimization brings significant performance improvement effect.

[0014] In a typical marketing scenario of one million users and one hundred strategies, the traditional solution needs to traverse all strategy nodes for each user, and the calculation time usually reaches minutes. The present application shortens the processing time of the same scenario to within 8 seconds through hierarchical parallel computing, with a calculation efficiency improvement of more than 900 times. The single node processing capacity is improved from 100,000 TPS in the traditional solution to 5,000,000 TPS, providing strong technical support for large-scale marketing activities.

[0015] The traditional marketing decision system needs to recalculate the full user path when the strategy node is updated, causing serious computational resource redundancy. Test data shows that a single strategy iteration usually consumes 85% of the CPU resources, causing great pressure on system operation.

[0016] The present application realizes the accurate influence range identification of strategy change through the incremental calculation model and Flink CDC technology. When the strategy node changes, the system only recompiles and calculates the affected downstream level, greatly reducing unnecessary computing overhead. In practical application, the CPU utilization rate is reduced from 85% to 22%, and the resource occupation is reduced by 73%. The memory usage is also significantly optimized, and the storage cost is reduced by 45% through the precompiled rule cache and sharding storage technology.

[0017] The existing marketing decision system generally adopts a serialized message pushing mechanism, and the transmission delay of the decision result is more than 500 milliseconds, which cannot meet the real-time requirements of high-concurrency marketing scenarios. Especially during the big promotion period of e-commerce platforms, the decision delay directly leads to a user loss rate of up to 12%.

[0018] The present application realizes the millisecond-level instruction pushing delay through the Kafka partition strategy and instruction priority queue design. The reach delay of VIP customers is stably controlled within 50 milliseconds, and the average pushing delay of ordinary users is also controlled within 200 milliseconds. The custom partitioner ensures that the instructions of the same user are routed to a fixed partition, which not only guarantees the consumption sequence, but also realizes load balancing. In the high-concurrency scenario, the system can stably support real-time decision processing of ten million users.

[0019] The strategy iteration of the traditional marketing system needs to be maintained, which affects the business continuity for an average of 2 hours each time, seriously restricting the flexible adjustment of marketing activities. The present application supports dynamic strategy hot updating, and the strategy iteration period is improved from hours to seconds, which greatly improves the business response speed.

[0020] Through the precompiled mechanism of the rule engine and the Redis dynamic rule pool, business personnel can adjust the marketing strategy configuration in real time, and the system can complete incremental calculation and take effect within 30 seconds. This flexibility enables enterprises to quickly respond to market changes and adjust marketing strategies in a timely manner, improving the accuracy and effectiveness of marketing activities.

[0021] The present application significantly reduces the IT operation cost of enterprises while ensuring a significant improvement in system performance. Compared with the traditional solution which needs to expand the server cluster size by more than 3 times, the present application optimizes the algorithm and improves the resource utilization rate, so that enterprises can handle the same amount of business without large-scale expansion.

[0022] In the actual application of financial institutions, the decision calculation time of the daily million-level user reach scenario is shortened from several hours to minutes, and the server resource demand is reduced by more than 70%. The significant reduction in operation and maintenance cost enables small and medium-sized enterprises to also bear the technical cost of large-scale precision marketing, promoting the popularization and application of technology in the entire industry.

[0023] Strictly regulated industries such as finance have high audit compliance requirements for marketing decisions, requiring complete decision traceability. The present invention provides complete decision traceability through hierarchical calculation result record and version management mechanism, meeting regulatory audit compliance requirements.

[0024] The calculation results of each level retain complete user filtering records and rule execution logs, forming complete link tracking from the root node to the final decision. This design not only meets the compliance requirements of the financial industry, but also provides data support for marketing effectiveness analysis and strategy optimization.

[0025] The hybrid computing architecture adopted by the present invention combines the flexibility of rule engine configuration with the batch processing capabilities of big data technology, demonstrating significant advancement in technical architecture. Through the micro-batch processing mode of Apache Spark Structured Streaming, the system not only guarantees real-time data processing capabilities, but also avoids the complexity and instability of pure streaming computation.

[0026] The distributed storage and computing design enables the system to have good horizontal expansion capability, allowing smooth expansion as the business scale grows. Fault-tolerant mechanisms and data backup strategies ensure high availability of the system, providing reliable technical support for enterprise-level applications.

[0027] In summary, the present invention achieves comprehensive optimization of marketing canvas batch decision processing through technical innovation, not only achieving significant breakthroughs in computing performance, resource utilization, and real-time response capabilities, but also creating substantial value for enterprises in terms of business flexibility, cost-effectiveness, and compliance guarantees, demonstrating important practical value and broad application prospects. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only the present invention, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0029] Fig. 1 The system architecture of the present invention is shown in the figure; Fig. 2 The decision tree strategy node rule engine architecture of the present invention is shown in the figure. DETAILED DESCRIPTION

[0030] The application will be described in detail below with reference to the drawings and specific embodiments. It should be noted that the following embodiments are the best, preferred embodiments, and other alternative embodiments can also be implemented by those skilled in the art for some known technologies; and the drawings are only used to more specifically describe the embodiments, and are not intended to specifically limit the application.

[0031] It should be noted that in the specification, "one embodiment", "embodiment", "exemplary embodiment", "some embodiments" and the like indicate that the described embodiment can include a specific feature, structure or property, but not necessarily every embodiment includes the specific feature, structure or property. In addition, when a specific feature, structure or property is described in combination with an embodiment, it should be within the knowledge of those skilled in the related art to implement such a feature, structure or property in combination with other embodiments (whether or not explicitly described).

[0032] Generally, the terms can be understood at least in part from the context of their usage. For example, depending at least in part upon the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular or can be used to describe combinations of features, structures or characteristics, in the plural. Still further, terms, such as "based on" can be understood as not necessarily of exclusive alternatives, as the expression allowing for the existence of additional factors not expressly described.

[0033] Reference Figs. 1-2 System overall architecture The marketing operation canvas batch decision method based on big data proposed by the application adopts a hierarchical architecture design, mainly including three core components of decision tree strategy node rule engine, batch calculation engine and instruction pushing optimization module.

[0034] The system architecture design follows the principle of separating calculation and storage, stores user attribute data through the HDFS distributed file system, pre-compiles the rule set through Redis cache, and processes instruction pushing through Kafka message queue, to realize high availability and scalability.

[0035] Implementation of decision tree strategy node rule engine The core of the decision tree strategy node rule engine is to convert the complex marketing operation canvas into a hierarchical structure that can be processed in parallel. The specific implementation steps are as follows: Step 1: Operation canvas analysis The system first analyzes the input operation canvas configuration, identifies the root operation plan node and each level of strategy node. Each node is assigned a unique nodeId identifier, and a parent-child relationship mapping table is established. The root operation plan node contains customer group label intersection and union rule configuration, and the child strategy node inherits and refines the filtering conditions.

[0036] Step two: Breadth-first search layering The policy nodes are decomposed into L1 to Ln rule layers using a breadth-first search algorithm. The algorithm starts from the root node and traverses the child nodes layer by layer, ensuring that nodes in the same layer can be processed in parallel and there is a dependency relationship between different layers. The layer depth k is usually controlled at 3 to 5 layers to balance the computational complexity and business flexibility.

[0037] Step three: Rule pre-compilation The rule set of each policy node is pre-compiled into a standard SQL statement or HiveSQL statement. The compilation process includes syntax parsing, condition expression optimization, and execution plan generation. The compiled rule statement is stored in the Redis dynamic rule pool, supporting hot updates and version management.

[0038] Batch computing engine implementation The batch computing engine is implemented based on Apache Spark Structured Streaming, using a micro-batch processing mode to handle large-scale user data.

[0039] Step one: Data preprocessing User attribute data is stored in HDFS shards according to user ID hash values, with each shard size controlled between 128MB and 256MB. The data format uses Parquet columnar storage, supporting predicate pushdown optimization to improve query efficiency.

[0040] Step two: Layered parallel computing The computing engine executes according to the pre-decomposed rule layer sequence. For the nth layer rule, the system reads the (n-1)th layer calculation results as input data set and performs intersection operation with the current layer rule. This design avoids the traditional scheme of traversing user dimensions one by one, reducing the computational complexity from O(nm) to O(klogn).

[0041] Step three: Incremental computation optimization When the policy node changes, the system captures the changes in real time through Flink CDC (Change Data Capture) and only recompiles and calculates the affected downstream layers. The change impact range is determined by the dependency graph, significantly reducing unnecessary computational overhead.

[0042] Step four: Result aggregation After each layer of calculation is completed, the system generates intermediate results in the form of {user ID, hit instruction set} key-value pairs. Through the MapReduce framework, the results are aggregated to eliminate duplicate instructions and generate the final user touch instruction set.

[0043] Instruction push optimization implementation The instruction push optimization module implements high concurrency and low latency instruction distribution through the Kafka message queue.

[0044] Step one: Priority queue design The system establishes an instruction priority queue based on user value levels. VIP customer instructions are assigned the highest priority, and ordinary customer instructions are classified according to business importance. The priority queue is implemented using a heap sorting algorithm to ensure that high-value user instructions are processed first.

[0045] Step two: Partition strategy optimization A custom Kafka partitioner is used to route instructions to fixed partitions based on user ID hash values. The number of partitions is set to an integer multiple of the number of consumer nodes to ensure load balancing. Multiple instructions for the same user are routed to the same partition to ensure consumption order.

[0046] Step three: Consumer optimization The consumer uses a batch consumption mode, with a single consumption batch size set to 100 to 500 instructions. The consumer maintains a local cache to de-duplicate and merge instructions for the same user, reducing the pressure on downstream systems.

[0047] Example one: Financial institution precision marketing scenario A large bank needs to conduct a credit card promotion campaign for 10 million active users. The traditional solution requires calculating the recommendation strategy for each user, taking more than 2 hours.

[0048] Technical implementation process: The system first decomposes the marketing strategy into three levels: L1 layer filters based on user asset level, L2 layer is based on consumption habits, and L3 layer combines risk rating to determine the final promotion strategy. Each layer of rules is pre-compiled into SQL statements and stored in Redis.

[0049] After starting the batch calculation engine, the L1 layer processes 10 million user data and filters out 5 million users that meet the asset level. The L2 layer further filters out 200 million target users based on the L1 results. The L3 layer finally determines 800,000 high-value promotion objects.

[0050] The entire calculation process is executed in parallel, with a total time of less than 8 seconds. The system generates 800,000 personalized promotion instructions and distributes them to various marketing channels through Kafka.

[0051] Effect verification: Compared with the traditional solution, the calculation time is reduced from 120 minutes to 8 seconds, with a 900-fold increase in calculation efficiency. The CPU utilization rate is reduced from 85% to 22%, and the memory occupancy is reduced by 60%.

[0052] Example Two: E-commerce platform big promotion activity scenario An e-commerce platform needs to push personalized coupons to 200 million users during Double Eleven. The traffic surges during the event, requiring the system to have high concurrent processing capability.

[0053] Technical implementation process: The system divides users into four value levels and designs corresponding coupon distribution strategies. High-value users enjoy a 50-millisecond instruction push delay, and ordinary users are controlled within 200 milliseconds.

[0054] The batch computing engine uses a sliding window mechanism to process a batch of new user data every 5 minutes. The instruction push module enables 16 Kafka partitions and configures 32 consumer nodes to ensure a push capacity of 100,000 TPS.

[0055] During the event, the system successfully handled the distribution of 200 million user coupons, with an average push delay of 120 milliseconds. The push delay for high-value users was stable at below 45 milliseconds.

[0056] Effect verification: The system remains stable in high-concurrency scenarios, with single-node processing capacity increasing from 100,000 TPS to 500,000 TPS. User churn rate decreased from 12% to 3%, and business conversion rate increased by 40%.

[0057] Example Three: Dynamic strategy adjustment scenario An internet financial company needs to adjust risk control strategies in real-time according to market changes, requiring the strategies to take effect quickly after updating.

[0058] Technical implementation process: The system monitors changes in the strategy configuration table through Flink CDC. When the risk control strategy is adjusted from "strict" to "lenient", the system automatically identifies the affected strategy nodes and recompiles the relevant rules.

[0059] The incremental computing engine only recalculates the downstream levels of the changed nodes, avoiding full data reprocessing. After strategy update, the system completes incremental calculation within 30 seconds to generate new user touch instructions.

[0060] Effect verification: The strategy iteration period is shortened from 2 hours to 30 seconds, meeting the business's rapid response needs. Under the incremental calculation mode, system resource occupancy is only 15% of that of full calculation, significantly reducing operating costs.

[0061] The application realizes efficient processing of marketing operation canvas batch decision through three technical innovations of layered decision optimization, hybrid computing architecture and low-delay transmission. The system significantly improves computing performance and resource utilization while ensuring business flexibility, providing a reliable technical solution for large-scale marketing scenarios.

[0062] In practical applications, the method has been verified in multiple industry scenarios, with single-node processing capacity improved by more than 50 times, computing resource occupancy reduced by 73%, and policy iteration period improved from hours to seconds, fully proving the advancement and practicality of the technical solution.

[0063] The present application covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present application. In order to make the public have a thorough understanding of the present application, specific details are described in the following preferred embodiments of the present application, and the present application can also be fully understood without the description of these details for those skilled in the art. In addition, in order to avoid unnecessary confusion to the essence of the present application, well-known methods, processes, procedures, elements and circuits, etc. are not described in detail.

[0064] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principle of the present application, a number of improvements and refinements can be made, which should also be considered as the protection scope of the present application.

Claims

1. A method for batch decision-making using a marketing management canvas based on big data, characterized in that, Includes the following steps: Receive a marketing operations canvas configuration that includes a root operations plan node and multiple strategy nodes, wherein the strategy nodes have a hierarchical structure and each node is configured with a customer group tag intersection and merge rule; The strategy nodes are decomposed into rule layers L1 to Ln according to the hierarchical relationship using a breadth-first search algorithm. Nodes at the same level can be processed in parallel, and there are data dependencies between different levels. The rule set of each policy node is pre-compiled into executable SQL statements and stored in a distributed caching system; The distributed stream computing engine executes rule calculations in a hierarchical order, where the nth layer rule uses the calculation results of the (n-1)th layer as the input dataset, and the user filtering results of the current layer are generated through intersection operations. The calculation results are converted into key-value pairs of user ID and hit instruction set, and then output to the message queue system after distributed aggregation processing. Command priorities are set based on user value levels, and commands are distributed to different message queue partitions through a custom partitioning strategy to achieve the push of batch decision commands.

2. The method according to claim 1, characterized in that, The breadth-first search algorithm specifically includes the following steps for decomposing strategy nodes hierarchically: Starting from the root management plan node, establish a mapping table between node IDs and parent-child relationships; By traversing all policy nodes in hierarchical order using a queue data structure, nodes at the same level are grouped into the same rule layer. Bitwise operations are used to mark processed nodes to avoid repeated traversal and ensure that each strategy node is assigned to a unique rule level.

3. The method according to claim 1, characterized in that, The rule set pre-compilation step specifically includes: Parse the customer group label intersection and union rules in the strategy node, and extract the conditional expressions and logical operators; Convert conditional expressions into standard SQL WHERE clause syntax and handle parenthesis precedence for compound conditions; Generate a complete SQL statement containing table joins, filter conditions, and projection fields, and optimize the execution plan; Pre-compiled SQL statements are stored in the Redis distributed cache system as key-value pairs, supporting dynamic rule updates and version management.

4. The method according to claim 1, characterized in that, The distributed stream computing engine uses the Apache SparkStructured Streaming framework, and the specific computing process includes: User attribute data is stored in HDFS shards based on user ID hash values, with each shard size controlled within the range of 128MB to 256MB. Read the pre-compiled SQL statements in the order of the rule hierarchy, and perform a correlation query between the current level rule and the calculation results of the previous level; The Catalyst optimizer optimizes the SQL execution plan by predicate pushdown and column pruning, reducing data transfer volume. The computation task is executed in a micro-batch processing mode with a batch interval of 5 to 30 seconds to balance computation latency and system throughput.

5. The method according to claim 1, characterized in that... It also includes incremental calculation optimization steps: Real-time monitoring of policy node configuration change events is achieved through Flink CDC technology; Construct a dependency graph between strategy nodes, and automatically identify the affected downstream hierarchy when a node change is detected; Only the affected policy nodes are re-executed for rule compilation and calculation, while the historical calculation results of unaffected nodes are retained; The incremental calculation results are merged and updated with the existing results to enable the strategy changes to take effect quickly.

6. The method according to claim 1, characterized in that, The custom partitioning strategy specifically includes: Instructions are divided into three priority queues—high, medium, and low—based on user value levels, with VIP user instructions assigned the highest priority. Implement a custom Kafka partitioner based on user ID hash values ​​to ensure that multiple commands from the same user are routed to the same partition; Set the number of partitions to 2 to 4 times the number of consumer nodes, and use a round-robin algorithm to achieve load balancing between partitions; On the consumer side, a batch consumption model is adopted, with the batch size of a single consumption set to 100 to 500 instructions, and duplicate instructions from the same user are deduplicated and merged.

7. The method according to any one of claims 1 to 6, characterized in that, The method optimizes the computational complexity from the traditional O(n×m) to O(k×logn), where n is the total number of users, m is the total number of policy nodes, k is the average level depth, and k ranges from 3 to 5.

8. A marketing management canvas batch decision-making system implementing the method of any one of claims 1 to 7, characterized in that, include: The decision tree strategy node rule engine is configured to parse the marketing management canvas and perform hierarchical decomposition using a breadth-first search algorithm. The batch computing engine uses the Apache Spark Structured Streaming framework to implement distributed parallel computing; The instruction push optimization module achieves low-latency instruction distribution through Kafka message queues and custom partitioning strategies. Distributed storage systems, including an HDFS cluster for storing user data and a Redis cluster for caching pre-compiled rules; The incremental calculation module, based on Flink CDC technology, enables real-time monitoring and incremental processing of policy changes.

Images