Customer tag query method, device, equipment and media based on rule transformation
By using a rule converter in the customer tag query system to convert tag rules into MPP database stored procedures, and leveraging the parallel processing capabilities of the MPP database for batch calculations, the problem of the Aviator rule engine not supporting aggregate functions is solved, tag calculation efficiency is improved and system coupling is reduced, achieving an efficient and decoupled tag query process.
Patent Information
- Application Number
- CN202611331788.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-31
- Publication Date
- 2026-11-17
AI Technical Summary
In the existing customer tag query system, the Aviator rule engine does not support aggregation functions, the Flink computing engine has limited computing resources, tag calculation takes a long time, and the tag rules are highly coupled with Flink jobs, making it difficult to meet the needs of rapid business iteration.
By using a rule converter independent of the data processing chain, user-configured tag rules are automatically converted into stored procedures executable by the MPP database. The parallel processing capabilities of the MPP database are used to complete batch calculations of tag data, and the Flink computing engine is responsible for data format conversion and writing to Elasticsearch.
It improves tag calculation efficiency, reduces system coupling, enhances system maintainability and user experience, and solves the performance bottleneck in the tag calculation process.
Smart Images

Figure CN123115220A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, device, and medium for querying customer tags based on rule transformation. Background Technology
[0002] Currently, with the digital transformation of retail banking, customer tagging systems have become a core infrastructure for precision marketing and risk control. Existing customer tagging platforms typically adopt a T+1 offline batch processing architecture: upstream data sources push basic customer data to the MPP database daily; subsequently, the data exchange platform converts the data into files and transmits them to the data warehouse server, then uploads them to HDFS (Hadoop Distributed File System); finally, the Flink computing engine, in conjunction with a tagging engine based on Aviator, completes tag rule matching, and the results are written to an Elasticsearch cluster for querying.
[0003] However, the above solution has the following technical drawbacks: First, the Aviator rule engine does not support aggregate functions. Aviator is a lightweight expression evaluator with limited syntax functionality. It does not support aggregate functions in the SQL (Structured Query Language) standard (such as SUM, AVG, MAX, MIN, COUNT, etc.), which prevents business users from developing aggregate tags such as "total consumption in the last 30 days is greater than 5,000 yuan", severely restricting the expansion of business scenarios.
[0004] Second, Flink's computing engine has limited computing resources, resulting in long tag computation times. In existing solutions, tag computation tasks are entirely handled by Flink. When processing tens of millions of customer data, the tag computation stage accounts for as much as 65% of the time, becoming a systemic bottleneck and severely impacting the timeliness of T+1 offline data processing.
[0005] Third, the tag rules are highly coupled with Flink jobs. Tag rules are hard-coded or bound to Flink jobs in the form of configuration files. Once the rules change, the code needs to be modified, repackaged and deployed, and the job needs to be restarted, which increases the complexity of operation and maintenance and makes it difficult to meet the needs of rapid business iteration.
[0006] Currently, there is a label processing method based on a rule engine and a real-time label calculation method based on Kafka message queues in the industry. This method uses the Aviator rule engine in combination with Kafka to achieve real-time label calculation, but the batch processing part still uses Flink to load rules and perform calculations.
[0007] As can be seen from the above, improving the efficiency of customer tag querying is an urgent problem to be solved. Summary of the Invention
[0008] In view of this, the purpose of this invention is to provide a customer tag query method, apparatus, device, and medium based on rule conversion. This method automatically converts user-configured tag rules into executable stored procedures for an MPP database through a rule converter independent of the data processing chain. This allows for batch calculation of tag data within the database, effectively improving tag calculation efficiency and reducing system coupling. The specific solution is as follows: Firstly, this application provides a customer tag query method based on rule transformation, applied to a bank customer tag determination system including a rule configuration module, a rule converter, an MPP database, a Flink computing engine, and an Elasticsearch cluster. The method includes: The rule configuration module receives tag configuration information and generates rule configuration parameters based on the tag configuration information; the tag configuration information includes tag type, data source table, field information, aggregation function, time window, filtering conditions, and threshold. The rule converter parses the rule configuration parameters to obtain the parsing result, and generates a corresponding MPP stored procedure based on the preset template and the parsing result, so as to store the MPP stored procedure in the rule storage table of the MPP database. The customer's basic data is tagged based on the MPP database and the MPP stored procedures in the rule storage table. The tag calculation results are then written into the tag result table. The tag result table includes tag number, customer number, tag value, and data processing time. The Flink computing engine is used to convert the data format of the tag calculation results in the tag result table to obtain tag data in Elasticsearch index format; The tag data is stored in the Elasticsearch cluster so that customer tag data can be queried.
[0009] Optionally, the step of receiving tag configuration information using the rule configuration module and generating rule configuration parameters based on the tag configuration information includes: Obtain the tag configuration information corresponding to the user, determine the tag type corresponding to the tag configuration information, and determine the corresponding data calculation method based on the tag type; The range of data to be calculated is determined based on the data source table and field information in the label configuration information, and corresponding calculation constraints are generated based on the aggregation function, time window, filtering conditions and threshold in the label configuration information. The rule configuration parameters are determined based on the data calculation method, the range of data to be calculated, and the calculation constraints.
[0010] Optionally, the step of parsing the rule configuration parameters through the rule converter to obtain the parsing result, and generating a corresponding MPP stored procedure based on a preset template and the parsing result, includes: Determine the tag calculation field, aggregation calculation method, calculation time range, and calculation conditions corresponding to the rule configuration parameters; The parsing result is determined based on the tag calculation field, the aggregation calculation method, the calculation time range, and the calculation conditions; The parsing result is written to the location corresponding to the preset template to generate an MPP stored procedure based on the writing result.
[0011] Optionally, storing the MPP stored procedure in the rule storage table of the MPP database includes: Determine the preset rule identifier corresponding to the MPP stored procedure, and establish a mapping relationship between the preset rule identifier and the storage address of the MPP stored procedure; The mapping relationship is written into the rule storage table, and the rule storage table is sent to the MPP database; the mapping relationship is used to call the MPP stored procedure corresponding to the preset rule identifier.
[0012] Optionally, the step of calculating tags for customer basic data based on the MPP database and the MPP stored procedures in the rule storage table, and then writing the tag calculation results into the tag result table, includes: Based on the mapping relationship, determine the MPP stored procedure corresponding to the rule storage table, and read the customer basic data sent to the MPP database from the preset upstream data source; The MPP stored procedure is invoked to perform aggregation calculations and conditional judgments on the customer basic data to obtain the tag calculation results; A tag result record is generated based on the tag number, customer number, tag value, and data processing time. The tag result record is then written into the tag result table, and the tag result table is sent to the Flink computing engine.
[0013] Optionally, the step of using the Flink computing engine to convert the data format of the tag calculation results in the tag result table to obtain tag data in Elasticsearch index format includes: Obtain the tag calculation results from the tag result table, and use the preset Elasticsearch index field structure to perform field mapping on the tag calculation results to obtain the field mapping results; The tag number, customer number, tag value, and data processing time in the tag result table are converted into corresponding index field information. Then, tag data in Elasticsearch index format is generated based on the index field information, and the tag data is sent to the Elasticsearch cluster.
[0014] Optionally, the step of using the Elasticsearch cluster to store the tag data, and then using the tag data to query customer tag data, includes: Obtain tag data in Elasticsearch index format issued by the Flink computing engine, and build a tag index based on the tag number and the customer number; The tag data is written to the Elasticsearch cluster, and then the customer tag data is queried based on the tag index and the customer tag query request to obtain the tag data of the corresponding customer.
[0015] Secondly, this application provides a customer tag query device based on rule transformation, applied to a bank customer tag determination system including a rule configuration module, a rule converter, an MPP database, a Flink computing engine, and an Elasticsearch cluster. The device includes: The rule configuration parameter generation module is used to receive tag configuration information using the rule configuration module and generate rule configuration parameters based on the tag configuration information; the tag configuration information includes tag type, data source table, field information, aggregation function, time window, filtering conditions, and threshold. The configuration parameter parsing module is used to parse the rule configuration parameters through the rule converter, obtain the parsing result, and generate a corresponding MPP stored procedure based on the preset template and the parsing result, so as to store the MPP stored procedure in the rule storage table of the MPP database. The tag calculation module is used to perform tag calculation on customer basic data based on the MPP database and the MPP stored procedures in the rule storage table, and then write the tag calculation results into the tag result table; the tag result table includes tag number, customer number, tag value and data processing time; The tag data generation module is used to convert the data format of the tag calculation results in the tag result table using the Flink computing engine to obtain tag data in Elasticsearch index format; The customer tag data query module is used to store the tag data in the Elasticsearch cluster, and to query customer tag data using the tag data.
[0016] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned rule-based customer tag query method.
[0017] Fourthly, this application provides a computer-readable medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned customer tag query method based on rule transformation.
[0018] As can be seen from the above, before performing customer tag queries based on rule transformation, this application needs to receive tag configuration information and generate rule configuration parameters using the rule configuration module. Then, a rule converter, independent of the data processing chain, parses the rule configuration parameters and generates corresponding MPP stored procedures, which are then stored in the rule storage table of the MPP database. During tag calculation, tags are calculated based on the customer basic data using the MPP stored procedures in the MPP database and rule storage table. The tag calculation results are then written to the tag result table, and after data format conversion using the Flink computing engine, they are stored in the Elasticsearch cluster for querying.
[0019] Therefore, this application first requires a rule converter to automatically transform user-configured tag rules (including complex logic such as aggregation functions and time windows) into executable stored procedures for the MPP database. Then, leveraging the parallel processing capabilities of the MPP database, batch computation of tag data is performed within the database, while the Flink computing engine is only responsible for writing the computation results to the Elasticsearch cluster. This approach overcomes the limitation of the Aviator rule engine's lack of support for aggregation functions during the customer tag determination process, significantly improving tag computation efficiency. Simultaneously, it decouples tag rules from the computing engine, thereby enhancing system maintainability and user experience. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0021] Figure 1 This application discloses a flowchart of a customer tag query method based on rule transformation. Figure 2 This is a schematic diagram of a specific system module for querying customer tags based on rule transformation disclosed in this application; Figure 3 This is a schematic diagram of a specific data processing link disclosed in this application; Figure 4 This is a schematic diagram of a customer tag query device based on rule transformation disclosed in this application; Figure 5 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation
[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] Existing customer tag query solutions, based on the Aviator rule engine and Flink computing engine, suffer from technical drawbacks such as limited rule representation capabilities, insufficient computational performance, and high system coupling. To address these shortcomings, this application provides a customer tag query method based on rule transformation. This method automatically converts user-configured Aviator rules into MPP stored procedures using a rule converter independent of the data processing chain. It then leverages the parallel processing capabilities of the MPP database to perform batch tag computation, and finally, Flink writes the data to Elasticsearch for querying.
[0024] See Figure 1 As shown in the figure, this invention discloses a customer tag query method based on rule transformation, applied to a bank customer tag determination system including a rule configuration module, a rule converter, an MPP database, a Flink computing engine, and an Elasticsearch cluster. The method includes: Step S11: Receive tag configuration information using the rule configuration module, and generate rule configuration parameters based on the tag configuration information; the tag configuration information includes tag type, data source table, field information, aggregation function, time window, filtering conditions, and threshold.
[0025] In this embodiment, Figure 2 This is a schematic diagram of the system modules for querying customer tags based on rule transformation. For example... Figure 2As shown, the bank customer tagging system proposed in this application includes the following five core modules: The rule configuration module provides a business-oriented configuration page, allowing users to configure tag rules through graphical interface components such as dropdown menus and input boxes. Tag configuration information includes tag type (basic tag / aggregate tag), data source table, field information, aggregation functions (SUM / AVG / MAX / MIN / COUNT), time window (e.g., the last 30 days), filtering conditions, thresholds, etc. The rule configuration module sends the user-configured parameters to the rule converter in JSON format.
[0026] Rule Converter: A service component independent of the data processing chain, responsible for automatically converting user-configured tag rules into MPP stored procedures. The rule converter parses the rule configuration parameters in JSON format, generates the corresponding MPP stored procedure code according to the preset template engine, and persists the generated stored procedure to the rule storage table of the MPP database via JDBC (Java Database Connectivity), while recording the mapping relationship between rule ID and stored procedure name.
[0027] MPP Database: Stores customer base data (source data) and transformed stored procedures, and acts as a computing engine to perform batch calculations with tags. The MPP database employs a Shared Nothing distributed architecture, leveraging features such as parallel querying, partitioning and pruning, and column-oriented storage to complete large-scale data aggregation calculations. MPP (Massively Parallel Processing) refers to breaking down a large computational task into multiple subtasks, distributing them to multiple nodes in a cluster for parallel execution, and finally aggregating the results and returning them to the user.
[0028] The Flink computing engine is responsible for reading completed tag data from the tag result table in the MPP database, converting it into Elasticsearch index format, and writing it to the Elasticsearch cluster via the Bulk API. In the technical solution of this application, the Flink computing engine no longer undertakes tag computation tasks, significantly reducing the computational load.
[0029] Elasticsearch cluster: Stores the final tag results, providing data services to applications and interfaces. Elasticsearch is a distributed search and analytics engine based on Lucene, providing a REST (Representational State Transfer) style web interface and supporting near real-time data indexing and querying. The Elasticsearch cluster builds a tag index based on tag number and customer number, supporting fast customer tag data querying.
[0030] In this embodiment, Figure 3 This is a diagram illustrating the data processing chain. (For example...) Figure 3 As shown, the data processing link between the above five modules is as follows: upstream source data (such as retail market customer behavior data, including consumption records, credit card records, account information, etc.) and the stored procedures pre-generated by the rule converter and stored in the MPP database enter the MPP database together.
[0031] Specifically, the upstream data source pushes customer basic data to the raw data table of the MPP database every morning. Simultaneously, the rule converter transforms user-configured tag rules into MPP stored procedures and stores them in the rule storage table of the MPP database. The MPP database executes the stored procedures based on the customer basic data, completing batch tag calculations and writing the results to the tag result table. The calculation results are then read from the tag result table of the MPP database by the Flink computing engine, undergo data format conversion, and written to the Elasticsearch cluster. After storing the tag data, the Elasticsearch cluster provides customer tag data query services for business applications and interfaces.
[0032] In this embodiment, the bank customer tag determination system first uses a rule configuration module to receive tag configuration information input by a user (such as a banking staff member). The rule configuration module provides a business-oriented configuration page, allowing users to configure tag rules through graphical interface components such as dropdown menus and input boxes, without writing any code. The tag configuration information specifically includes the following: Tag Type: This identifies whether the currently configured tag is a basic tag or an aggregated tag. Basic tags are tags that are directly mapped from a single field in the customer's basic data, such as "customer gender" or "customer age group". Aggregated tags are tags that require grouping and aggregating the customer's basic data to obtain, such as "total consumption in the last 30 days" or "average transaction amount in the last 90 days".
[0033] Data source table: Used to specify the customer base data table on which the tag calculation depends, such as the transaction history table, credit card table, account information table, etc. in the retail marketplace.
[0034] Field information: Used to specify the names of the fields that participate in the tag calculation, such as the consumption amount field, transaction time field, etc.
[0035] Aggregate functions: These specify what kind of aggregation calculations are performed on field information, including standard SQL aggregate functions such as SUM (summation), AVG (average), MAX (maximum), MIN (minimum), and COUNT (count).
[0036] Time window: Used to specify the time range considered in the label calculation, such as "last 30 days", "last 90 days", etc.
[0037] Filtering criteria: Used to specify the criteria for filtering customer basic data, such as "transaction type is consumption" or "channel is online".
[0038] Threshold: Used to specify the critical value for judging a label. For example, 5000 in "total consumption greater than 5000 yuan" is the threshold.
[0039] It is worth mentioning that after receiving the aforementioned tag configuration information, the rule configuration module generates rule configuration parameters based on the tag configuration information. Specifically, the rule configuration module first determines the tag type corresponding to the tag configuration information, and then determines the corresponding data calculation method based on the tag type: If the tag type is a basic tag, then the data calculation method is direct field mapping; If the tag type is an aggregate tag, the data calculation method is grouped aggregation calculation.
[0040] Then, the rule configuration module determines the range of data to be calculated based on the data source table and field information in the tag configuration information, and generates corresponding calculation constraints based on the aggregation function, time window, filtering conditions, and threshold in the tag configuration information. Finally, the rule configuration module determines the complete rule configuration parameters based on the data calculation method, the range of data to be calculated, and the calculation constraints, and sends the rule configuration parameters to the rule converter in JSON (JavaScript Object Notation) format.
[0041] Step S12: The rule configuration parameters are parsed by the rule converter to obtain the parsing result, and a corresponding MPP stored procedure is generated based on the preset template and the parsing result, so as to store the MPP stored procedure in the rule storage table of the MPP database.
[0042] In this embodiment, the rule converter is a service component independent of the data processing chain, responsible for automatically converting user-configured tag rules into executable stored procedures for the MPP database. After receiving the JSON-formatted rule configuration parameters sent by the rule configuration module, the rule converter first parses the rule configuration parameters.
[0043] Specifically, the rule converter determines the calculated label fields (i.e., the field names to be calculated), aggregation calculation methods (i.e., the aggregation functions used, such as SUM, AVG, etc.), calculation time range (i.e., the start and end dates corresponding to the time window), and calculation conditions (i.e., the SQL WHERE clause conditions corresponding to the filter conditions) corresponding to the rule configuration parameters. Then, the rule converter determines the complete parsing results based on the calculated label fields, aggregation calculation methods, calculation time range, and calculation conditions.
[0044] Furthermore, the rule converter generates a corresponding MPP stored procedure based on a preset template and the parsing results. The preset template is a predefined framework for stored procedure code, containing the stored procedure's name, parameters, variable declarations, main logic, and other structure. The rule converter writes each element from the parsing results (label calculation fields, aggregation calculation methods, calculation time ranges, calculation conditions, etc.) into the corresponding location in the preset template, thereby generating complete MPP stored procedure code.
[0045] In one specific implementation, for the aggregate tag "total consumption in the last 30 days is greater than 5,000 yuan", the MPP stored procedure code generated by the rule converter contains the following core SQL logic: filter the transaction records of the last 30 days from the consumption transaction table, group them by customer number, perform SUM summation on the consumption amount field, and then determine whether the summation result is greater than the threshold of 5,000. If it is greater, the tag value is 1 (indicating a hit); otherwise, the tag value is 0 (indicating a miss).
[0046] An MPP stored procedure is a set of SQL statements that performs a specific function, compiled and stored in the database. When executing a stored procedure, the user needs to specify the procedure name and provide parameters. In this application, the rule converter uses JDBC to persist the generated stored procedure to the MPP database using the `CREATE OR REPLACE PROCEDURE` statement.
[0047] When storing an MPP stored procedure into the rule storage table of the MPP database, the rule converter first determines the preset rule identifier (i.e., the unique number of the tag rule) corresponding to the MPP stored procedure and establishes a mapping relationship between the preset rule identifier and the storage address of the MPP stored procedure. Then, the rule converter writes the mapping relationship into the rule storage table and sends the rule storage table to the MPP database. This mapping relationship is used to call the MPP stored procedure corresponding to the preset rule identifier during subsequent tag calculations.
[0048] Step S13: Calculate tags for customer basic data based on the MPP database and the MPP stored procedures in the rule storage table, and then write the tag calculation results into the tag result table; the tag result table includes tag number, customer number, tag value, and data processing time.
[0049] In this embodiment, after rule transformation and storage are completed, the bank customer tag determination system enters the tag calculation stage. Each morning, the upstream data source (such as a retail marketplace) pushes basic customer data (including transaction records, credit card tables, account information tables, etc.) to the raw data table in the MPP database. The T+1 offline task trigger calls the scheduler, traversing the MPP stored procedures stored in the rule storage table and executing them sequentially according to the flow defined in the stored procedures.
[0050] Specifically, the system first needs to determine the MPP stored procedure corresponding to the rule storage table based on the mapping relationship, and then read the customer basic data sent to the MPP database from the preset upstream data source. Then, the system calls the MPP stored procedure to perform aggregation calculations and conditional judgments on the customer basic data to obtain the tag calculation results.
[0051] The MPP database engine utilizes its parallel query, partitioning, and column-oriented storage features to perform large-scale data aggregation calculations when executing stored procedures. The MPP database employs a shared-nothing architecture, where each node in the cluster has its own independent CPU (Central Processing Unit), memory, and disk storage. Query tasks are executed in a distributed parallel manner, decomposing a query task into parallel execution tasks on each node of the cluster.
[0052] For commonly used operators in aggregation calculations such as JOIN and GROUP BY, the MPP database employs a distributed implementation. Data is stored column-wise, with each data file consisting of several data packets, and data operations are performed on a packet-by-packet basis. Furthermore, the MPP database supports three levels of compression: instance-level, table-level, and field-level, and supports column-based data encoding and efficient compression techniques, effectively improving I / O performance. Through the parallel processing capabilities of the MPP database, the execution efficiency of stored procedures is significantly improved, and the tag calculation time is significantly reduced compared to existing technologies.
[0053] After tag calculation is complete, the system needs to generate tag result records based on the tag number, customer number, tag value, and data processing time. These records are then written to the tag result table, which is subsequently sent to the Flink computing engine. The tag result table is a dedicated table in the MPP database for storing tag calculation results. Its fields include: tag number (used to uniquely identify a tag rule), customer number (used to uniquely identify a customer), tag value (the result value of the tag calculation, such as 0 or 1, a specific numerical value, etc.), and data processing time (the timestamp of the tag calculation). This stage leverages the large-scale service cluster resources of the MPP database, significantly reducing data calculation time compared to existing solutions and effectively addressing the performance bottleneck of the Flink computing engine.
[0054] Step S14: Use the Flink computing engine to convert the data format of the tag calculation results in the tag result table to obtain tag data in Elasticsearch index format.
[0055] In this embodiment, after tag computation is completed, the system monitors data computation tasks through a batch job scheduler. Once MPP data processing is complete, the scheduler automatically launches the Flink computation engine. Flink (Apache Flink) is an open-source distributed stream and batch processing framework with features such as high throughput, low latency, and exactly-once semantics. In the technical solution of this application, the Flink computation engine no longer undertakes tag computation tasks but focuses on data format conversion and writing tasks.
[0056] The Flink computing engine reads the tag computation results from the tag result table in the MPP database, performs data format conversion on these results, and transforms them into tag data in the Elasticsearch index format. Specifically, the Flink computing engine retrieves the tag computation results from the tag result table and uses a predefined Elasticsearch index field structure to perform field mapping on the tag computation results, obtaining the field mapping result. The predefined Elasticsearch index field structure predefines the storage format of the tag data in Elasticsearch, including field names, field types, tokenizers, and other configuration information. The purpose of field mapping is to convert the relational table structure in the MPP database into a document structure in Elasticsearch.
[0057] Then, the Flink computing engine converts the tag ID, customer ID, tag value, and data processing time from the tag results table into corresponding index field information. Specifically: the tag ID (TAG_ID) is mapped to a keyword type field in Elasticsearch for exact match queries; the customer ID (CUST_ID) is mapped to a keyword type field in Elasticsearch for exact match queries; the tag value (TAG_VALUE) is mapped to an integer or float type field in Elasticsearch for range queries and aggregation statistics; and the data processing time (PROC_TIME) is mapped to a date type field in Elasticsearch for time range queries.
[0058] After completing the field mapping, the Flink computing engine generates tag data in Elasticsearch index format based on the index field information, that is, converting each tag result record into an Elasticsearch document. Each document contains fields such as tag number, customer number, tag value, and data processing time, and specifies the index name to which the document belongs. Finally, the Flink computing engine uses the Elasticsearch Bulk API to batch distribute the tag data to the Elasticsearch cluster.
[0059] It is worth mentioning that in this embodiment, the role of the Flink computing engine has undergone a fundamental change: from the core engine that originally undertook all tag computing tasks, it has been transformed into a lightweight component that is only responsible for data format conversion and writing. This effectively reduces the computational load of the Flink computing engine, allowing it to focus on data export tasks, thereby improving the stability and efficiency of the entire system.
[0060] Step S15: Store the tag data using the Elasticsearch cluster, and then use the tag data to query customer tag data.
[0061] In this embodiment, the Elasticsearch cluster receives tag data in Elasticsearch index format from the Flink computing engine. The Elasticsearch cluster first builds a tag index based on the tag number and customer number. The tag index is a logical namespace in Elasticsearch used to organize and retrieve documents, similar to a table in a relational database. In this application, a separate index can be created for each tag number, or all tag data can be stored in the same index and distinguished by the tag number field. When creating the index, Elasticsearch defines the field type and attributes according to the preset index mapping.
[0062] Then, the Elasticsearch cluster writes the tagged data to the cluster. Elasticsearch uses a distributed architecture, where data is sharded and stored on different nodes, and each shard can have multiple replicas for high availability. Write operations are performed in batches via the Bulk API to improve write efficiency.
[0063] When a business application or API initiates a customer tag query request (e.g., querying all tags for customer ID "CUST_12345"), the Elasticsearch cluster retrieves customer tag data based on the tag index and the customer tag query request. The Elasticsearch query engine supports various query types, including term queries (exact match), range queries, and boolean queries. For customer tag queries, a term query is typically used for exact matching by customer ID, quickly returning all tag data for that customer. The query results are returned to the business application or API in JSON format for use in scenarios such as targeted marketing, risk control, and personalized recommendations.
[0064] In this way, the entire customer tag data query chain is completed. From user configuration of tag rules, to automatic conversion of rules into MPP stored procedures, to batch calculation of tags in the MPP database, to data format conversion by Flink, and finally to storage and query services by Elasticsearch, the entire process is automated, efficient, and decoupled.
[0065] As can be seen from the above, this embodiment first uses a rule converter to automatically convert user-configured tag rules into executable stored procedures for the MPP database. Then, it utilizes the parallel processing capabilities of the MPP database to perform batch calculations of tag data within the database. The Flink computing engine is only responsible for writing the calculation results to the Elasticsearch cluster. This overcomes the limitation of the Aviator rule engine's lack of support for aggregation functions during the customer tag determination process, significantly improving tag calculation efficiency. Simultaneously, it decouples tag rules from the computing engine, thereby enhancing system maintainability and user experience.
[0066] Accordingly, see Figure 4 As shown, this application also provides a customer tag query device based on rule transformation, applied to a bank customer tag determination system including a rule configuration module, a rule converter, an MPP database, a Flink computing engine, and an Elasticsearch cluster. The device includes: The rule configuration parameter generation module 11 is used to receive tag configuration information using the rule configuration module and generate rule configuration parameters based on the tag configuration information; the tag configuration information includes tag type, data source table, field information, aggregation function, time window, filtering conditions, and threshold. The configuration parameter parsing module 12 is used to parse the rule configuration parameters through the rule converter, obtain the parsing result, and generate a corresponding MPP stored procedure based on the preset template and the parsing result, so as to store the MPP stored procedure in the rule storage table of the MPP database. Tag calculation module 13 is used to perform tag calculation on customer basic data based on the MPP database and the MPP stored procedures in the rule storage table, and then write the tag calculation results into the tag result table; the tag result table includes tag number, customer number, tag value and data processing time; The tag data generation module 14 is used to convert the data format of the tag calculation results in the tag result table using the Flink computing engine to obtain tag data in Elasticsearch index format; The customer tag data query module 15 is used to store the tag data using the Elasticsearch cluster, so as to perform customer tag data query using the tag data.
[0067] In one specific implementation, the rule configuration parameter generation module 11 specifically includes: The tag type determination unit is used to obtain the tag configuration information corresponding to the user, determine the tag type corresponding to the tag configuration information, and determine the corresponding data calculation method based on the tag type; The constraint generation unit is used to determine the range of data to be calculated based on the data source table and field information in the label configuration information, and to generate corresponding calculation constraints based on the aggregation function, time window, filtering conditions and threshold in the label configuration information. The rule configuration parameter determination unit is used to determine the rule configuration parameters based on the data calculation method, the range of data to be calculated, and the calculation constraints.
[0068] In one specific implementation, the configuration parameter parsing module 12 specifically includes: The calculation element determination unit is used to determine the tag calculation field, aggregation calculation method, calculation time range, and calculation conditions corresponding to the rule configuration parameters. The parsing result determination unit is used to determine the parsing result based on the tag calculation field, the aggregation calculation method, the calculation time range, and the calculation conditions; The stored procedure generation unit is used to write the parsing result to the location corresponding to the preset template, so as to generate an MPP stored procedure based on the writing result.
[0069] In one specific implementation, the configuration parameter parsing module 12 further includes: A mapping relationship establishment unit is used to determine the preset rule identifier corresponding to the MPP stored procedure and establish a mapping relationship between the preset rule identifier and the storage address of the MPP stored procedure; The mapping relationship storage unit is used to write the mapping relationship into the rule storage table and send the rule storage table to the MPP database; the mapping relationship is used to call the MPP stored procedure corresponding to the preset rule identifier.
[0070] In one specific implementation, the tag calculation module 13 specifically includes: The stored procedure reading unit is used to determine the MPP stored procedure corresponding to the rule storage table based on the mapping relationship, and to read the customer basic data sent to the MPP database from the preset upstream data source; The tag calculation execution unit is used to call the MPP stored procedure to perform aggregation calculation and condition judgment on the customer basic data to obtain the tag calculation result; The tag result writing unit is used to generate tag result records based on tag number, customer number, tag value and data processing time, then write the tag result records into the tag result table, and send the tag result table to the Flink computing engine.
[0071] In one specific implementation, the tag data generation module 14 specifically includes: The field mapping unit is used to obtain the tag calculation results in the tag result table, and to perform field mapping on the tag calculation results using a preset Elasticsearch index field structure to obtain the field mapping results; The index data generation unit is used to convert the tag number, customer number, tag value, and data processing time in the tag result table into corresponding index field information, then generate tag data in Elasticsearch index format based on the index field information, and send the tag data to the Elasticsearch cluster.
[0072] In one specific implementation, the customer tag data query module 15 specifically includes: The tag index building unit is used to obtain tag data in Elasticsearch index format issued by the Flink computing engine, and build a tag index based on the tag number and the customer number; The tag query execution unit is used to write the tag data into the Elasticsearch cluster, and then perform a customer tag data query based on the tag index and the customer tag query request to obtain the tag data of the corresponding customer.
[0073] Furthermore, embodiments of this application also disclose an electronic device, Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the rule-based customer tag query method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be a computer.
[0074] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0075] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.
[0076] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the rule-based customer tag query method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.
[0077] Furthermore, this application also discloses a computer-readable medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned customer tag query method based on rule transformation. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.
[0078] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.
[0079] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0080] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of media known in the art.
[0081] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0082] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method of customer tag query based on rule conversion, characterized in that, The method, applied to a bank customer tagging system that includes a rule configuration module, a rule converter, an MPP database, a Flink computing engine, and an Elasticsearch cluster, comprises: The rule configuration module receives tag configuration information and generates rule configuration parameters based on the tag configuration information; the tag configuration information includes tag type, data source table, field information, aggregation function, time window, filtering conditions, and threshold. The rule converter parses the rule configuration parameters to obtain the parsing result, and generates a corresponding MPP stored procedure based on the preset template and the parsing result, so as to store the MPP stored procedure in the rule storage table of the MPP database. The customer's basic data is tagged based on the MPP database and the MPP stored procedures in the rule storage table. The tag calculation results are then written into the tag result table. The tag result table includes tag number, customer number, tag value, and data processing time. The Flink computing engine is used to convert the data format of the tag calculation results in the tag result table to obtain tag data in Elasticsearch index format; The tag data is stored in the Elasticsearch cluster so that customer tag data can be queried.
2. The rule-based conversion-based customer tag query method of claim 1, wherein, The step of receiving tag configuration information using the rule configuration module and generating rule configuration parameters based on the tag configuration information includes: Obtain the tag configuration information corresponding to the user, determine the tag type corresponding to the tag configuration information, and determine the corresponding data calculation method based on the tag type; The range of data to be calculated is determined based on the data source table and field information in the label configuration information, and corresponding calculation constraints are generated based on the aggregation function, time window, filtering conditions and threshold in the label configuration information. The rule configuration parameters are determined based on the data calculation method, the range of data to be calculated, and the calculation constraints.
3. The rule-based conversion-based customer tag query method of claim 1, wherein, The step of parsing the rule configuration parameters through the rule converter to obtain the parsing result, and generating a corresponding MPP stored procedure based on the preset template and the parsing result, includes: Determine the tag calculation field, aggregation calculation method, calculation time range, and calculation conditions corresponding to the rule configuration parameters; The parsing result is determined based on the tag calculation field, the aggregation calculation method, the calculation time range, and the calculation conditions; The parsing result is written to the location corresponding to the preset template to generate an MPP stored procedure based on the writing result.
4. The method of claim 1, wherein, The step of storing the MPP stored procedure in the rule storage table of the MPP database includes: Determine the preset rule identifier corresponding to the MPP stored procedure, and establish a mapping relationship between the preset rule identifier and the storage address of the MPP stored procedure; The mapping relationship is written into the rule storage table, and the rule storage table is sent to the MPP database; the mapping relationship is used to call the MPP stored procedure corresponding to the preset rule identifier.
5. The rule-based conversion-based customer tag query method of claim 4, wherein, The step of performing tag calculations on customer basic data based on the MPP database and the MPP stored procedure in the rule storage table, and then writing the tag calculation results into the tag result table, includes: Based on the mapping relationship, determine the MPP stored procedure corresponding to the rule storage table, and read the customer basic data sent to the MPP database from the preset upstream data source; The MPP stored procedure is invoked to perform aggregation calculations and conditional judgments on the customer basic data to obtain the tag calculation results; A tag result record is generated based on the tag number, customer number, tag value, and data processing time. The tag result record is then written into the tag result table, and the tag result table is sent to the Flink computing engine.
6. The rule-based conversion-based customer tag query method of claim 5, wherein, The process of using the Flink computing engine to convert the data format of the tag calculation results in the tag result table to obtain tag data in Elasticsearch index format includes: Obtain the tag calculation results from the tag result table, and use the preset Elasticsearch index field structure to perform field mapping on the tag calculation results to obtain the field mapping results; The tag number, customer number, tag value, and data processing time in the tag result table are converted into corresponding index field information. Then, tag data in Elasticsearch index format is generated based on the index field information, and the tag data is sent to the Elasticsearch cluster.
7. The rule-based conversion-based customer tag query method of claim 6, wherein, The step of using the Elasticsearch cluster to store the tag data, and then using the tag data to query customer tag data, includes: Obtain tag data in Elasticsearch index format issued by the Flink computing engine, and build a tag index based on the tag number and the customer number; The tag data is written to the Elasticsearch cluster, and then the customer tag data is queried based on the tag index and the customer tag query request to obtain the tag data of the corresponding customer.
8. A rule-based conversion-based customer tag query apparatus, characterized by comprising: An apparatus for determining bank customer tags, applicable to a system including a rule configuration module, a rule converter, an MPP database, a Flink computing engine, and an Elasticsearch cluster, comprising: The rule configuration parameter generation module is used to receive tag configuration information using the rule configuration module and generate rule configuration parameters based on the tag configuration information; the tag configuration information includes tag type, data source table, field information, aggregation function, time window, filtering conditions, and threshold. The configuration parameter parsing module is used to parse the rule configuration parameters through the rule converter, obtain the parsing result, and generate a corresponding MPP stored procedure based on the preset template and the parsing result, so as to store the MPP stored procedure in the rule storage table of the MPP database. The tag calculation module is used to perform tag calculation on customer basic data based on the MPP database and the MPP stored procedures in the rule storage table, and then write the tag calculation results into the tag result table; the tag result table includes tag number, customer number, tag value and data processing time; The tag data generation module is used to convert the data format of the tag calculation results in the tag result table using the Flink computing engine to obtain tag data in Elasticsearch index format; The customer tag data query module is used to store the tag data in the Elasticsearch cluster, and to query customer tag data using the tag data.
9. An electronic device, comprising: include: Memory, used to store computer programs; A processor for executing the computer program to implement the rule-based customer tag query method as described in any one of claims 1 to 7.
10. A computer readable medium characterized by Used to store a computer program, wherein the computer program, when executed by a processor, implements the customer tag query method based on rule transformation as described in any one of claims 1 to 7.
