Flink-based rule configuration service generation system

Abstract

The invention provides a rule configuration business generation system based on Flink. The system comprises a front-end configuration module, a constraint module, a rule expression engine module, an Flink engine packaging module and a task scheduling module. The front-end configuration module provides a visual dragging configuration interface and is used for constructing a target service; the constraint module configures constraint conditions to ensure legality and effectiveness of rule configuration; the rule expression engine module analyzes and executes a rule expression; the Flink engine packaging module integrates the processed service configuration information; and the task scheduling module controls the life cycle of the service. All the modules are connected through input and output ends, so that efficient and flexible service generation and management are realized; according to the invention, efficient, flexible and automatic business construction can be realized, the accuracy of rule configuration is ensured, and real-time data processing and refined business management are supported.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a rule-configured service generation system based on Flink. Background Art

[0002] With the development of the economy, the banking industry has entered the era of existing customers. How to efficiently and quickly market the existing customers of banks and enhance customer stickiness has become the key to the survival of banks. Real-time marketing of existing customers through stream computing has become an inevitable choice. However, stream computing development requires developers to master various components of the big data ecosystem, and the requirements for developers are relatively high.

[0003] Currently, the online launch of marketing activities often requires a series of processes such as activity configuration, activity approval, activity release, activity update, task development, task testing, task release, activity stop, and task offline to be realized. For some complex business logics, business personnel need to tell developers, and developers need to understand them before development. Moreover, a large number of tests are required to verify the business correctness, and the development results are prone to deviation from the proposed business. All these lead to an increase in development costs, and the time from the proposal of a business activity to its online launch is often more than half a month. How to efficiently and quickly support the marketing activities of marketing business personnel has become a difficult point. Summary of the Invention

[0004] In view of this, the embodiments of this application provide a rule-configured service generation system based on Flink, which can achieve high efficiency, flexibility, and automation in service construction, ensure the accuracy of rule configuration, and support real-time data processing and refined service management.

[0005] The technical solution of the embodiments of this application is implemented as follows:

[0006] In a first aspect, the embodiments of this application provide a rule-configured service generation system based on Flink, and the system includes:

[0007] A front-end configuration module, a constraint module, a rule expression engine module, a Flink engine encapsulation module, and a task scheduling module;

[0008] The front-end configuration module is used to provide a configuration interface, and the target service is constructed based on the draggable pages in the configuration interface. Each draggable page represents a service function; the constraint module is used to configure constraint conditions, and the constraint conditions are used to detect the legality and effectiveness of rule configuration; the rule expression engine module is used to parse and execute the rule expressions configured in the front-end configuration module; the Flink engine encapsulation module is used to integrate the configuration information of the target service processed by the rule expression engine module and the configuration constraint module; the task scheduling module is used to control the life cycle of the target service;

[0009] The output end of the front-end configuration module is connected to the input end of the constraint module. The output ends of the constraint module and the rule expression engine module are connected to the input end of the Flink engine encapsulation module. The output end of the Flink engine encapsulation module is connected to the input end of the task scheduling module.

[0010] In a second aspect, an embodiment of the present application further provides a marketing activity generation system, and the marketing activity generation system generates marketing activities based on the Flink-based rule configuration business generation system according to any one of the first aspects.

[0011] The embodiments of the present application have the following beneficial effects:

[0012] By integrating multiple modules such as front-end configuration, constraint management, rule expression parsing and execution, Flink engine encapsulation, and task scheduling, the high flexibility and automation of business construction are achieved. The system not only simplifies the development process of complex businesses, reduces the need for professional programming skills, but also ensures the legality and effectiveness of rule configuration through strict constraint conditions, thereby improving the stability of business operation and the accuracy of data processing. In addition, with the powerful capabilities of the Flink engine, the system can efficiently process large-scale data streams, meet real-time business requirements, and at the same time, through the task scheduling module, achieve refined management of the business life cycle, providing strong support for business operation. Description of the Drawings

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0014] Figure 1 is a flowchart of the online launch of a marketing activity in the prior art;

[0015] Figure 2 is a schematic diagram of the linkage of system internal modules provided by an embodiment of the present application;

[0016] Figure 3 is an interaction diagram of associated systems provided by an embodiment of the present application. Detailed Embodiments

[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. It should be understood that the accompanying drawings in this application are only for the purposes of illustration and description, and are not used to limit the protection scope of this application. Additionally, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without a logical context relationship may be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of this application.

[0018] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0019] In addition, the described embodiments are only some embodiments of this application, rather than all embodiments. The components of the embodiments of this application usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of this application claimed, but merely represents the selected embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of this application.

[0020] In the following description, the terms "first / second / third" involved are only used to distinguish similar objects, and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when allowed, so that the embodiments of this application described here can be implemented in an order other than that illustrated or described here.

[0021] It should be noted that the term "including" will be used in the embodiments of this application to indicate the existence of the features stated thereafter, but does not exclude the addition of other features.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0023] When the applicant implemented the embodiments of this application, it was found that the prior art had the following problems:

[0024] At present, the online launch of marketing activities often requires a series of processes such as activity configuration, activity approval, activity release, activity update, task development, task testing, task release, activity stop, and task offline (as shown in Figure 1 ) to be realized. For some complex business logics, business personnel need to tell developers, who need to understand before development. And a large number of tests are required to verify the business correctness. Moreover, the development results are prone to deviation from the proposed business, all of which lead to an increase in development costs. The time from the proposal of business activities to their online launch often exceeds half a month. How to efficiently and quickly support the marketing activities of marketing business personnel has become a difficult point.

[0025] Since the original marketing timeliness cannot quickly respond to market changes and changes in marketing needs, it has become necessary for banks to be able to keenly grasp marketing opportunities and meet the dynamic needs of customers in real time to adapt to the new situation. In the absence of a rule expression engine module and a configuration rule constraint framework, for some complex business codes, only if-else statements can be continuously added to meet complex business scenarios. When there are too many if-else statements, the code becomes extremely difficult to read. Although it can be optimized through the strategy pattern, the problem of slow development and the need for online launch still cannot be solved. If the code is written in a fixed way and the logic changes, the code needs to be modified and launched again. In the case of frequent logic changes, this approach is obviously unacceptable.

[0026] Therefore, a configuration rule constraint framework is needed to use a rule engine to change the current situation and make these business rule changes in an efficient and reliable way through formula calculation. Benefits: Reduce development costs. Business personnel can independently configure business rules without developers having to understand. Improve the efficiency of rule changes and the online launch speed. Through the rule engine, we only need to convert the rules configured by business personnel into a string, and then save the string of this rule to the database. When using this rule, only the parameters required by this rule need to be passed, and the result can be directly calculated, which not only ensures the accuracy of business rules, but also eliminates the need for developers to write any code for these rules.

[0027] Please refer to Figure 2 , Figure 2 which is a schematic diagram of the internal module linkage of the system provided by the embodiment of the present application. The embodiment of the present application provides a rule-configured business generation system based on Flink. The system includes:

[0028] A front-end configuration module, a constraint module, a rule expression engine module, a Flink engine encapsulation module, and a task scheduling module;

[0029] The front-end configuration module is used to provide a configuration interface, and the target service is built based on the draggable pages in the configuration interface. Each draggable page represents a service function; the constraint module is used to configure constraint conditions, and the constraint conditions are used to detect the legality and effectiveness of rule configurations; the rule expression engine module is used to parse and execute the rule expressions configured in the front-end configuration module; the Flink engine encapsulation module is used to integrate the configuration information of the target service processed by the rule expression engine module and the configuration constraint module; the task scheduling module is used to control the life cycle of the target service;

[0030] The output end of the front-end configuration module is connected to the input end of the constraint module, the output ends of the constraint module and the rule expression engine module are connected to the input end of the Flink engine encapsulation module, and the output end of the Flink engine encapsulation module is connected to the input end of the task scheduling module.

[0031] In some embodiments, the front-end configuration module is used to configure at least one of the following functions:

[0032] Basic information, data source, activity status, field conversion, condition filtering, dimension table association, home customer group selection, aggregation configuration, compliance rules, data output, and default configuration;

[0033] The compliance rules are determined based on the service data required by the target service, and the data details of the service data are displayed in the draggable page;

[0034] The customer group selection includes a fixed customer group and a dynamic customer group. The fixed customer group is configured once and not updated. The customer group list in the dynamic customer group is in an updatable state. The customer group list is selected into the customer group by selecting the customer group list in the configuration interface, and specific customer groups are blocked or passed by selecting the black and white list identifier in the configuration interface;

[0035] The activity status is used to take the target service offline in advance. The activity status of the target service is adjusted by adjusting the opening and closing status of the switch. When the target service is taken offline through the activity status, the activity data of the target service is filtered and not calculated.

[0036] The marketing front-end module provides a visual and draggable page for marketers to configure, and can configure basic information, data source, activity status, field conversion, condition filtering, dimension table association, home customer group selection, aggregation configuration, compliance rules, data output, and default configuration, etc.

[0037] The threshold for users to meet the requirements is often determined by periodic transaction volume, transaction amount, number of transactions, adjustment limits and other activities. In order to facilitate marketing personnel to configure various condition combinations, financial transaction, adjustment limit and other data are detailed in real time.

[0038] Customer group selection is divided into fixed customer group and dynamic customer group selection. Fixed customer groups are imported once at the beginning of the activity, and the list will not be updated during the activity. Dynamic customer groups update the customer group list regularly. Marketing business personnel only need to select the customer group list and the black and white list logo on the front end to block or select a certain type of customer group.

[0039] The activity status configuration is used when an activity is abnormal and needs to be taken offline urgently. You can set an already effective activity to be unavailable. After the activity status is set to invalid, the activity task will not be stopped immediately, but the mainstream data will be filtered internally and no further calculation will be performed. If the activity needs to be resumed, you can also set the switch to the available state, and the activity will continue, but the data during the invalid period will be discarded and not calculated.

[0040] In some embodiments, the constraint module corresponds to a constraint framework, and the constraint framework includes at least one of an activity flow constraint, an activity business constraint, and an activity configuration constraint;

[0041] The activity flow constraints are used to check business approval and data approval;

[0042] The active service constraint is used to check the effective interval of the target service;

[0043] The activity configuration constraints include marketing activity configuration and other configurations. The marketing activity configuration is used to filter fields in a specific data source, and non-filtered data sources are prohibited from performing configuration.

[0044] The constraint module implements various restrictions on business configuration through a constraint framework to ensure the accuracy and compliance of the configuration. The following is a detailed explanation of each constraint type in the constraint framework:

[0045] Activity process constraints: In addition to the approval of related business, the process approval of marketing activities also adds data configuration approval during the configuration process. If there are problems or abnormalities in the configuration, the configuration can be rejected and reconfigured. This solves the problem that some marketing personnel are not familiar with configuration constraints, resulting in some activities not being able to start tasks normally or abnormal data processing at the beginning. Data configuration verification approval is a necessary condition to ensure activity configuration.

[0046] Activity business constraints: The effective period of an activity is basically from 0:00 on a certain day to 0:00 on a certain day. Activity configuration needs to be configured according to the approved activity needs, and cannot be changed or used for other purposes at will. Configuration needs to be configured according to the operation case description.

[0047] Activity configuration constraints: For the configuration of marketing activity fields and other configurations, processing can only be based on the fields in the filtered data source, and processing of non-filtered data sources is not supported. Marketers can achieve data processing through simple drag-and-drop configurations on the front-end page. Each drag-and-drop module corresponds to the encapsulation of a backend Flink operator, that is, the step-by-step processing of data. There is a sequence in data processing among the dragged modules, and the order cannot be reversed.

[0048] In some embodiments, the rule expression engine module performs dynamic evaluation based on the Aviator expression engine. The rule expression engine module implements custom functions by inheriting the AbstractFunction class. The custom functions at least include in, not_in, equal, not_equal, md5, date_diff, rlike, like.

[0049] In the embodiments of the present application, the rule expression engine module is built based on the Aviator expression engine and is used to perform dynamic evaluation on business rules. By inheriting the AbstractFunction class, this module has implemented multiple custom functions to expand the functions of the Aviator expression engine. The following is a detailed explanation of these custom functions:

[0050] Rule expression engine module: This module utilizes the flexibility and high performance of the Aviator expression engine to dynamically parse and execute business rules. Through custom functions, this module can handle more complex business logics and data operations.

[0051] By inheriting the AbstractFunction class, the rule expression engine module has implemented the following custom functions:

[0052] in function: Checks whether a value exists in a specified set.

[0053] Application scenario: For example, checking whether a customer ID exists in a specific set of customer IDs.

[0054] not_in function: Checks whether a value is not in a specified set.

[0055] Application scenario: For example, excluding certain specific product IDs.

[0056] equal function: Compares whether two values are equal.

[0057] Application scenario: For example, verifying whether the password entered by the user is the same as the stored password.

[0058] not_equal function: Compares whether two values are not equal.

[0059] Application scenarios: For example, filtering records that do not meet specific conditions.

[0060] md5 function: Calculates and returns the MD5 hash value of a given string.

[0061] Application scenarios: For example, encrypting passwords before storing them.

[0062] date_diff function: Calculates the difference between two dates (in days, hours, etc.).

[0063] Application scenarios: For example, calculating the number of days between the shipping date and the expected delivery date of an order.

[0064] rlike function: Matches strings using regular expressions.

[0065] Application scenarios: For example, filtering email addresses that match a specific pattern.

[0066] like function: Matches strings using wildcards (such as % and _).

[0067] Application scenarios: For example, searching for product names that contain a specific substring.

[0068] In the Aviator expression engine, custom functions are implemented by implementing the AbstractFunction class and overriding its call method. The call method receives an expression context and a list of parameters and returns the result of the function.

[0069] In some embodiments, the Flink engine encapsulation module is used for task global configuration and data encapsulation;

[0070] The task global configuration includes at least one of a publishing queue, checkpoint configuration, time semantics, and parallelism;

[0071] The data encapsulation includes data source encapsulation, data transformation encapsulation, and data output encapsulation;

[0072] The data source encapsulation corresponds to the data source configured for the target service. The data transformation encapsulation includes the encapsulation of specific operators, and the data output encapsulation is used to encapsulate different types of data output sources.

[0073] In some embodiments, the specific operators include at least one of the following operators:

[0074] Map operator, Filter operator, asynchronous operator, broadcast operator, aggregation operator, TopN operator;

[0075] The Map operator is used for field mapping transformation, the Filter operator is used for data filtering, the asynchronous operator is used for asynchronous query operations of dimension table association and customer group selection, the broadcast operator is used for broadcast association between mainstream data sources and rule data sources, the aggregation operator is used to support incremental aggregation and full-scale aggregation operations, and the TopN operator is used for sorting intermediate results during the calculation process to intercept the first N or last N records.

[0076] Specifically, the global task configuration includes, for example, the publishing queue, checkpoint configuration (type, interval time, timeout time, etc.), time semantics, parallelism, etc.

[0077] Data source encapsulation (Source source encapsulation): corresponding to the data source of the marketing activity configuration. The Source encapsulation is based on the native flinkapi encapsulation, inherits the RichParallelSourceFunction, and implements the ListCheckpointed interface. The encapsulation involves the kafka cluster id, topic, kafka authentication information of the default tenant corresponding to the cluster, consumption parallelism, name, watermark expression, allowed default delay time of the watermark, maximum allowed delay time of the data, field expression (whether the message body is parsed once when consuming the kafka topic configuration), etc.

[0078] Data Transformation Encapsulation (Transform Encapsulation): Encapsulate the logic transformation of some common operators. For example: (1) Map operator: Solve the problem of field mapping transformation in configuration rules, parse and transform cleaning expressions such as field renaming, field logic transformation, extended fields, and default value assignment for fields. Some transformation logics will perform rule transformation based on the rule expression engine avaitor. The Map encapsulation is based on the native flink api, inherits RichMapFunction, and implements the ListCheckpointed interface. The field cleaning expression is passed in the class constructor. (2) Filter operator: Filter the data flowing into this operator according to the filtering conditions in the rule configuration, similar to the where condition in sql. The Filter encapsulation is based on the native flink api, inherits RichFilterFunction, and implements the ListCheckpointed interface. The filtering expression is passed in the class constructor. (3) Asynchronous operator: This operator is mainly used for dimension table association, asynchronous query of customer groups (which can be understood as white lists), and asynchronous query of blacklists. The asynchronous operator encapsulation is based on the native flink api, inherits the RichAsyncFunction interface, and the association type, association table, and rowkey expression are passed in the class constructor to obtain the fields to be retrieved. This operator can be encapsulated differently based on HBase, Redis, Mysql, Oracle, etc. to meet the needs of asynchronous association queries for different stream storages. Query the associated table in the corresponding storage medium according to the association type parameter, and obtain the corresponding fields in the table by parsing the rowkey expression. For other information such as connection information for each data source, whether dimension data is cached, cache record count, cache data lifecycle, parallelism, operator operation name, operator uuid, etc., if not custom-set, the default values will be used. (4) Broadcast operator encapsulation: This operator is mainly used for the broadcast association between the mainstream data source and the rule data source to achieve the function of dynamic rule update and take effect. In addition, it is encapsulated in combination with the Filter operator. In the case where an activity needs to be urgently taken offline due to an exception, it can implement the blocking processing of consumption data, and the data during the blocking period will be discarded. (5) Aggregation operator: The encapsulation of ReduceFunction and AggregateFunciton realizes incremental aggregation, which is suitable for scenarios of incremental calculation; the encapsulation of ProcessWindowFunction can achieve full-scale aggregation through secondary aggregation to solve similar scenarios such as global topN; KeyedProcessFunction solves the problem of sorting windows according to custom fields, time, version, etc. (6) TopN encapsulation: This TopN encapsulation is mainly based on the encapsulation of the aggregation operator. The intermediate results during the calculation process are stored in Redis. After sorting by Redis Sort, the first N or last N are intercepted to meet the needs of reports or dashboards.In addition to encapsulating logical conversions, these operators also add some parameters such as operator UUID, name, parallelism, etc. These parameters take default values or can be customized.

[0079] As Figure 2 shown, the front-end configuration is from 1 to 9, and the back-end encapsulation is from 1* to 9*. The digital labels and the corresponding numbers * have a mapping relationship. For example, data source 1 corresponds to Source encapsulation 1*.

[0080] Data output encapsulation (Sink source encapsulation): mainly solves the problem of output after data processing. According to different output storage media, the connection information will be differentially encapsulated such as HBase, Kafka, Redis, ClickHouse, StarRocks, Oracle, etc. It is based on the native flinkapi class to inherit RichSinkFunction and implement the ListCheckpointed interface. In addition, for output to Redis and HBase, keyExp settings are involved, and for output to ClickHouse and StarRocks, there are also differences in cache batch size, trigger time interval, etc.

[0081] In some embodiments, the task scheduling module periodically schedules and obtains valid services during the active period every time period, invokes the unstarted services through the YARN API, and periodically schedules and obtains invalid services every time period, and stops the still-running services through the YARN API;

[0082] The task scheduling module is also used to compare the batch compliance list data with the real-time issued list and compensate and send the list.

[0083] Specifically, the task scheduling module is used to implement the following functions:

[0084] Task release: The scheduling system periodically schedules and obtains the valid activities still within the active period in the rule table every day, obtains the task activity list running on YARN through the yarn api, and after comparison, invokes the unstarted active tasks that are valid, and at the same time adds them to the task monitoring and warning list.

[0085] Task termination: The scheduling system periodically schedules tasks every day to obtain the expired or expired activities yesterday in the rule table, obtains the task activities still running on YARN through the yarn api, and after comparison, stops the active tasks that have expired but are still running, and at the same time takes offline the monitoring of the tasks.

[0086] Reconciliation compensation: Compare the batch compliance list data with the real-time issued list after landing, and compensate and send the list.

[0087] In some embodiments, the system further includes a data service platform, which is used for real-time computing to provide data support for the target business. The data service platform includes a Kafka cluster, a Yarn cluster, and an Hbase cluster.

[0088] Please refer to Figure 3 , Figure 3 FIG. is the interaction diagram of the associated system provided by the embodiments of the present application. The data service platform is a core component designed to meet the requirements of real-time data processing. It can quickly respond to data changes. By integrating multiple technology clusters, it realizes the real-time collection, processing, and storage of data, and then provides instant and accurate data services for the business layer.

[0089] As a message middleware, the Kafka cluster is mainly responsible for the real-time collection and transmission of data. It has the characteristics of high throughput and low latency, and can easily handle the real-time stream processing scenarios of large data volumes.

[0090] The Yarn (Yet Another Resource Negotiator) cluster is a resource manager in the Hadoop ecosystem, focusing on resource allocation and task scheduling. In the data service platform, the Yarn cluster is responsible for managing and scheduling real-time computing tasks.

[0091] Hbase is a distributed and scalable big data storage system, built on Hadoop HDFS and designed for real-time reading and writing. It provides query functions similar to relational databases and supports efficient random access at the same time.

[0092] In the data service platform, the three major clusters of Kafka, Yarn, and Hbase cooperate closely to jointly support the complete process of real-time data processing. Kafka is responsible for the real-time collection and transmission of data, Yarn is responsible for task scheduling and execution, and Hbase is responsible for data storage and query. This process ensures that data can be quickly captured, processed from the source, and finally stored in an efficient data storage system for the business layer to call at any time.

[0093] In some embodiments, the system further includes an operation system, which is used for auditing the target business and evaluating the results of the target business. The evaluation data is obtained based on the data service platform.

[0094] Please continue to refer to Figure 3, The operation system is the core control node in the entire business operation process. Relying on advanced data processing and analysis capabilities, it implements a strict review process for the target business to ensure compliance with established business rules, laws and regulations, as well as the company's operation strategies. At the same time, the operation system also uses the real-time and accurate data support of the data service platform to deeply evaluate the results of the target business, providing strong data support for business optimization and strategy adjustment.

[0095] The operation system comprehensively reviews all aspects, processes, and potential risks of the target business to ensure compliance with relevant laws and regulations, industry standards, and the company's internal regulations.

[0096] The operation system can seamlessly connect to the data service platform to obtain various types of data generated during the operation of the target business in real time, such as user behavior data, transaction data, market feedback, etc. Using data analysis technologies and algorithms, the operation system conducts multi-dimensional and in-depth evaluations of the results of the target business. This includes aspects such as the achievement of business goals, user satisfaction, and market response. Based on the evaluation results, the operation system can propose targeted strategy adjustment suggestions, such as optimizing business processes, adjusting market promotion strategies, etc., to continuously enhance the competitiveness and profitability of the business.

[0097] In the overall architecture of the system, the operation system is closely connected to the data service platform, jointly constituting the data-driven core of business operation. The data service platform provides rich and real-time data resources for the operation system, while the operation system uses these data resources to precisely manage and evaluate the target business.

[0098] The embodiment of the present application also provides a marketing activity generation system, and the marketing activity generation system generates marketing activities based on any one of the Flink-based rule-configured business generation systems provided by the embodiment of the present application.

[0099] In summary, the embodiment of the present application has the following beneficial effects:

[0100] (1) High flexibility and scalability: Through the front-end configuration module, the system provides a visual drag-and-drop page construction method, enabling users to easily configure and generate target businesses without writing complex codes. This configuration-based method greatly improves the flexibility and scalability of the system, enabling users to quickly respond to market changes and business requirements.

[0101] (2) Guarantee of the legality and effectiveness of rule configuration: The constraint module strictly detects the legality and effectiveness of rule configuration by configuring constraint conditions. This ensures that the rule expressions configured by users are logically correct, avoiding problems such as business operation failures or data errors caused by configuration errors.

[0102] (3) Strong rule parsing and execution capabilities: The rule expression engine module performs dynamic evaluation based on the Aviator expression engine, supports custom functions, and can parse and execute complex rule expressions. This enables the system to handle various complex business logics and meet diverse business requirements.

[0103] (4) Efficient Flink task encapsulation and scheduling: The Flink engine encapsulation module integrates the target business configuration information processed by the rule expression engine module and the constraint module, and performs global task configuration and data encapsulation. This ensures the efficient and stable operation of Flink tasks. At the same time, the task scheduling module can schedule and obtain valid and invalid services at regular intervals, and start and stop tasks through the YARN API, realizing the automated management of Flink tasks.

[0104] (5) Rich data processing operator support: The Flink engine encapsulation module provides a variety of data processing operators, including Map operators, Filter operators, asynchronous operators, broadcast operators, aggregation operators, and TopN operators, etc. These operators can meet users' needs for various operations such as data transformation, filtering, association, aggregation, and sorting, improving the efficiency and flexibility of data processing.

[0105] (6) Real-time data support and business evaluation: The data service platform performs real-time calculations through the Kafka cluster, Yarn cluster, and Hbase cluster to provide data support for the target business. This enables the system to process and analyze data in real time, providing timely and accurate data support for business decisions. At the same time, the operation system can review and evaluate the target business, and conduct quantitative analysis of business effects based on the data obtained from the data service platform, providing a scientific basis for business optimization and strategy adjustment.

[0106] (7) Reduce development and maintenance costs: This system greatly reduces the complexity of business development and maintenance costs through configuration and encapsulation. Users do not need to deeply understand the underlying technical details, and can generate and deploy services through simple configuration. This makes the system easier to use and maintain, improving development efficiency and business launch speed.

[0107] In summary, the rule configuration-based business generation system based on Flink has beneficial effects such as high flexibility, scalability, strong rule parsing and execution capabilities, efficient Flink task encapsulation and scheduling, rich data processing operator support, real-time data support and business evaluation, and reduced development and maintenance costs. These advantages enable the system to quickly respond to market changes and business requirements, improve the efficiency and accuracy of business processing, and create greater value for enterprises.

[0108] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the method embodiments, and will not be elaborated herein. In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings, direct couplings, or communication connections shown or discussed among each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or modules can be in electrical, mechanical, or other forms.

[0109] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0110] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0111] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A rule-based configuration business generation system based on Flink, characterized in that: The system comprises: Front-end configuration module, constraint module, rule expression engine module, Flink engine encapsulation module and task scheduling module; The front-end configuration module is used to provide a configuration interface, and the target business is constructed based on the draggable pages in the configuration interface, and each draggable page represents a business function; the constraint module is used to configure constraint conditions, and the constraint conditions are used to detect the legality and validity of the rule configuration; the rule expression engine module is used to parse and execute the rule expression configured in the front-end configuration module; the Flink engine encapsulation module is used to integrate the configuration information of the target business processed by the rule expression engine module and the configuration constraint module; the task scheduling module is used to control the life cycle of the target business; The output end of the front-end configuration module is connected to the input end of the constraint module, the output end of the constraint module and the output end of the rule expression engine module are connected to the input end of the Flink engine encapsulation module, and the output end of the Flink engine encapsulation module is connected to the input end of the task scheduling module.

2. The system according to claim 1, characterized in that The front-end configuration module is used to configure at least one of the following functions: Basic information, data source, activity status, field conversion, conditional filtering, dimension table association, customer group selection, aggregation configuration, compliance rules, data output and default configuration; The compliance rule is determined based on the business data required by the target business, and the draggable page displays data details of the business data; The customer group selection includes fixed customer groups and dynamic customer groups. The fixed customer groups are configured once and are not updated. The customer group list in the dynamic customer group is updateable. The customer group list is selected by selecting the customer group list in the configuration interface, and the blacklist and whitelist identifiers are selected in the configuration interface to block or pass specific customer groups. The activity status is used to take the target service offline in advance. The activity status of the target service is adjusted by adjusting the on / off status of the switch. When the target service is taken offline by the activity status, the activity data of the target service is filtered and not calculated.

3. The system according to claim 1, characterized in that The constraint module corresponds to a constraint framework, and the constraint framework includes at least one of an activity flow constraint, an activity business constraint, and an activity configuration constraint; The activity flow constraints are used to check business approval and data approval; The active service constraint is used to check the effective interval of the target service; The activity configuration constraints include marketing activity configuration and other configurations. The marketing activity configuration is used to filter fields in a specific data source, and non-filtered data sources are prohibited from performing configuration.

4. The system according to claim 1, characterized in that The rule expression engine module performs dynamic evaluation based on the Aviator expression engine. The rule expression engine module implements custom functions by inheriting the AbstractFunction class. The custom functions include at least in, not_in, equal, not_equal, md5, date_diff, rlike, and like.

5. The system according to claim 1, characterized in that The Flink engine encapsulation module is used for global task configuration and data encapsulation; The task global configuration includes at least one of a release queue, a checkpoint configuration, time semantics, and parallelism; The data encapsulation includes data source encapsulation, data conversion encapsulation and data output encapsulation; The data source encapsulation corresponds to the data source of the target business configuration, the data conversion encapsulation includes the encapsulation of a specific operator, and the data output encapsulation is used to encapsulate different types of data output sources.

6. The system according to claim 5, characterized in that The specific operator includes at least one of the following operators: Map operator, Filter operator, asynchronous operator, broadcast operator, aggregation operator, TopN operator; The Map operator is used for field mapping conversion, the Filter operator is used for data filtering, the asynchronous operator is used for asynchronous query operations such as dimension table association and customer group selection, the broadcast operator is used for broadcast association of mainstream data sources and regular data sources, the aggregation operator is used to support incremental aggregation and full aggregation operations, and the TopN operator is used to sort intermediate results during the calculation process and intercept the first N or last N records.

7. The system according to claim 1, characterized in that The task scheduling module regularly schedules to obtain valid services during the activity period in each time period, and calls up unstarted services through the YARN API, and regularly schedules to obtain invalid services in each time period, and stops the services that are still running through the YARN API; The task scheduling module is also used to compare the batch qualified list data with the real-time issued list, and to send the list for compensation.

8. The system according to claim 1, characterized in that The system also includes a data service platform, which is used to perform real-time computing to provide data support for the target business. The data service platform includes a Kafka cluster, a Yarn cluster and an Hbase cluster.

9. The system according to claim 8, characterized in that The system also includes an operation system, which is used to review the target business and evaluate the results of the target business, and the evaluation data is obtained based on the data service platform.

10. A marketing activity generation system, characterized in that: The marketing activity generation system generates a marketing activity based on the Flink-based rule-configured business generation system described in any one of claims 1-9.

Images