Dynamic user grouping and loan marketing system and method based on Dores real-time warehouse counting

Abstract

The invention discloses a dynamic user grouping and loan marketing system and method based on a Dores real-time data warehouse, and belongs to the technical field of finance, the dynamic user grouping and loan marketing system based on the Dores real-time data warehouse comprises a data acquisition layer, a data storage layer, a rule configuration layer, a real-time calculation layer and a marketing execution layer. According to the method, a full-link system of real-time data acquisition, dynamic rule configuration, real-time label calculation, intelligent grouping and precision marketing is constructed, and refined operation of loan users is realized based on efficient calculation and storage capability of a Dores real-time data bin and in combination with a dynamic rule engine. The core of the system is to break through the bottleneck of module coupling and timeliness of a traditional scheme and form a closed loop of user behavior triggering-rule real-time effectiveness-marketing immediate execution.

Description

Technical Field

[0001] This invention relates to the field of financial technology, specifically to a dynamic user segmentation and loan marketing system and method based on the Doris real-time data warehouse. Background Technology

[0002] In the consumer finance and credit sector, repeat purchases are a core indicator driving business growth, and "accurately identifying user needs and timely marketing outreach" is the key path to improving repeat purchase rates.

[0003] Current mainstream user segmentation and marketing triggering solutions in the industry have the following significant shortcomings: Poor data timeliness: Most systems rely on T+1 offline data warehouses (such as the data warehouse software Hive) for user tag calculation. For example, a user may become eligible for a new loan after repaying their debt on the same day, but the marketing system may not be able to recognize this change in status until the next day, resulting in the loss of the best marketing opportunity (such as the short-term demand window after a user repays their loan).

[0004] Rigid rules and slow response: Marketing rules are mostly hard-coded into business systems or rely on predefined fixed tags. When business personnel need to create subgroups based on new scenarios (such as "frequently browsing the 'credit limit increase' page in the past 3 days but not submitting an application"), technical personnel need to modify the code, test, and relaunch, a process that usually takes 1-3 days, making it impossible to quickly respond to market changes and user needs.

[0005] Query performance bottleneck: The user base for credit business often reaches hundreds of millions, and the daily increase in user behavior data (such as login, browsing, and application records) exceeds terabytes. When marketers need to combine dozens of conditions such as "available credit limit > 3,000 yuan + login behavior in the past 7 days + risk score > 650" to select users, traditional relational databases (such as MySQL) or the Hadoop ecosystem take several minutes to several hours to query, which cannot support interactive marketing strategy adjustments.

[0006] High system coupling: In existing solutions, data storage (such as HDFS—Hadoop Distributed FileSystem), tag computation (such as MapReduce tasks), rule engine (such as hard-coded logic), and outreach system (such as SMS interface) are tightly coupled. If it is necessary to add marketing channels (such as AI outbound calling) or adjust tag dimensions (such as adding the "repayment frequency" tag), multiple modules of code need to be modified in conjunction, resulting in poor scalability and high maintenance costs.

[0007] Currently, real-time data warehouse technologies such as Apache Doris are known in the industry. However, there is no publicly disclosed and effective complete solution in existing technologies for how to deeply integrate them with a highly dynamic and configurable rule management system to solve the complete closed-loop problem in loan marketing scenarios, which involves "real-time perception of user status changes, dynamic definition of complex rules, instantaneous completion of group calculations, and immediate triggering of marketing actions".

[0008] Based on this, the present invention designs a dynamic user segmentation and loan marketing system and method based on the Doris real-time data warehouse to solve the above problems. Summary of the Invention

[0009] In view of the above-mentioned shortcomings of the existing technology, the present invention provides a dynamic user segmentation and loan marketing system and method based on Doris real-time data warehouse.

[0010] To achieve the above objectives, the present invention provides the following technical solution: The dynamic user segmentation and loan marketing system based on the Doris real-time data warehouse includes a data acquisition layer, a data storage layer, a rule configuration layer, a real-time computing layer, and a marketing execution layer. The data acquisition layer deploys a Flink cluster, a Kafka cluster, and Flink-CDC tasks to monitor the MySQL business database in a master-slave architecture. The data storage layer constructs a user basic information table, a user behavior information table, an attribute configuration table, and a behavior attribute configuration table, deploys a Doris cluster, and performs bucketing processing on the user basic information table and the user behavior information table based on the USER_ID field. Rule configuration layer: The web management interface is developed based on the Spring Boot framework. Configuration data is uniformly stored in the MySQL configuration database. It supports the visual configuration of attribute definition, tag rules, grouping rules and marketing rules. Specifically, the visual configuration of tag rules is implemented based on the attribute configuration table and the behavior attribute configuration table. It supports both real-time update and offline update modes for rule activation and adopts an event-driven computing mechanism to automatically activate the rule engine when the user triggers a specified event. The real-time computing layer deploys a rule engine service based on the Drools framework and a tag computing service developed in Java. Both services cache frequently accessed tag rules through Redis. When a user behavior event or a system scheduled task is triggered, the system queries the latest user data from Doris, loads the matching rules, and completes tag calculation and updates. At the same time, it performs cluster calculation based on the clustering rules. The cluster calculation methods include real-time clustering and scheduled clustering, and the clustering results are written to the Doris clustering result table in real time. The marketing execution layer deploys marketing scheduling services, channel adaptation services, and performance analysis services.

[0011] Furthermore, the data acquisition layer captures changes in user basic information and behavioral data in the MySQL business database in real time through Flink-CDC, builds a data buffer layer through a Kafka cluster, and splits user attributes and user behavior into two topics according to data type; based on the Doris Routine Load mechanism, it subscribes to Kafka topics and writes the cleaned data into the user basic information table and user behavior information table of the Doris data warehouse in real time.

[0012] Furthermore, a user base information table is constructed through the user base model to store and manage multi-dimensional user attribute information, including the following dimensions: identity information, demographic characteristics, financial business attributes, risk assessment indicators, marketing tags, and timestamp management.

[0013] Furthermore, a user behavior information table is constructed using a user behavior model to record and analyze the user's entire behavioral trajectory, including the following dimensions: Behavioral Identification System: Establishes unique identifiers through behavioral IDs and user IDs; Behavioral classification dimensions: a three-level classification system of behavior type, behavior action, and behavior state; Time-based management: Records the initiation and end times of actions; Channel and scenario management: Records information about the channels and scenarios in which behaviors occur; Extension mechanism: Design three types of extended fields: character, numeric, and date. Data timeliness management: Store data by date in partitions.

[0014] Furthermore, an attribute configuration table is established through an attribute metadata model to manage the metadata information of user attributes, thereby achieving standardized definitions of attributes, including the following dimensions: Attribute identification management: a dual identification system of attribute name and attribute display name; Data type definition: Supports four data types: character, numeric, enumeration, and date; Update timeliness control: Distinguish between two timeliness types: real-time updates and offline updates.

[0015] Furthermore, a behavior attribute configuration table is established through the behavior attribute metadata model: this table manages the attribute metadata of user behavior and achieves standardized definitions of behavior attributes, including the following dimensions: Behavioral classification system: A three-level classification and identification system is established based on behavior type, behavior action, and behavior ID; Behavioral attribute definition: Identified by both the behavioral attribute name and the behavioral attribute display name; Data type standard: Supports four data types: character, numeric, enumeration, and date.

[0016] Furthermore, the marketing execution layer deploys marketing scheduling services, channel adaptation services, and performance analysis services, including: binding differentiated rules to each segment, including reach channels, marketing content, reach timing, and reach restrictions; integrating multiple push interfaces; recording marketing execution logs and user response data, calculating marketing conversion rates for each segment; and optimizing rules.

[0017] To better achieve the objectives of this invention, this invention also provides a dynamic user segmentation and loan marketing method based on the Doris real-time data warehouse, comprising the following steps: Step 1: Data Synchronization 1.1: User data in the MySQL business database has been changed; 1.2: The Flink-CDC task captures changed data and performs format cleaning; 1.3: The cleaned data is sent to the corresponding Kafka topic according to its type; 1.4: Doris subscribes to Kafka topics through Routine Load tasks, consumes data in real time, and writes it to the user basic information table or user behavior information table. At the same time, it automatically encrypts and stores mobile phone numbers and ID card numbers using the SHA256 encryption algorithm. 1.5: After the data is written, Doris returns a write confirmation, and Flink-CDC records the synchronization log; Step 2: Tag Rule Configuration and Calculation 2.1: Through the Web management interface, select the "Label Configuration" function to load configurable attributes from the attribute configuration table and the behavior attribute configuration table; 2.2: Configure tag rules; 2.3: After the tag rules are configured, the system stores the rule logic in a MySQL rule table and synchronizes it to the Redis cache; 2.4: When a user triggers an event, the rules engine receives the event notification and loads the relevant tag rules for that user from Redis; 2.5: The rules engine calls the Doris query interface to obtain the latest tag rule matching data for the user; 2.6: The rule engine performs conditional judgments. If the rule is met, the corresponding tag is assigned to the user, and the tag information is updated to the Doris user tag table. 2.7: After the tag update is completed, the system triggers a cluster calculation notification; Step 3: User Segmentation 3.1: Configure grouping rules for business personnel; 3.2: Clustering rules are stored in the MySQL clustering table. Clustering calculation methods include real-time clustering and timed clustering. 3.3: Write the clustering results into the Doris clustering results table; Step 4. Marketing Execution.

[0018] Furthermore, in step 3.2, real-time grouping: after user tags are updated, the grouping service automatically determines whether the user meets the grouping rules, and if so, adds the user to the group; scheduled grouping: according to a preset period, the grouping service queries all user tags in Doris in batches and calculates the group affiliation of each user.

[0019] Furthermore, step 4 specifically involves: 4.1: Configure marketing rules for different user groups; 4.2: When a user joins a target segment, the marketing dispatch service receives a segment change notification; 4.3: The scheduling service checks user reach restrictions; if the conditions are met, it calls the push interface. 4.4: Execute push notifications, record the results, and synchronize them to the Doris marketing log table; 4.5: User behavior data is written to Doris through a data synchronization process, and the performance analysis service calculates the marketing conversion rate of each segment; 4.6: Optimize marketing rules based on conversion rate data.

[0020] Compared with the prior art, the beneficial effects of this invention are as follows: (1) Second-level tag computing capability: Relying on Doris' MPP architecture and RoutineLoad real-time import feature, the computing time for hundreds of millions of user tags is controlled within 1-3 seconds, which is more than 90% lower than the traditional Hadoop ecosystem (MapReduce tasks take minutes).

[0021] (2) Improved rule iteration efficiency: By decoupling rules from code through metadata tables, business personnel can independently configure tags and grouping rules, shortening the iteration cycle from the traditional 1-3 days to 5-10 minutes, greatly improving business response speed.

[0022] (3) Optimization of grouping timeliness: Based on user behavior events to drive grouping calculation, rather than traditional timed batch tasks, it ensures that the grouping is updated within 10 seconds after the user's status changes (such as paying off debts), so as not to miss the best marketing opportunity.

[0023] (4) Strong targeting of loan scenarios: The tag system is designed around the core needs of the loan business, covering four dimensions: credit limit (available credit limit), risk (A / B card score), repayment (outstanding principal and repayment records), and behavior (login / application frequency). Compared with the general user tag system (such as "purchase frequency" in the e-commerce field), the marketing accuracy is improved by more than 40%. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0025] Figure 1 The system architecture of this invention Figure 1 ; Figure 2 The system architecture of this invention Figure 2 ; Figure 3 This is a flowchart of the label calculation process of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0027] Example 1: Please refer to the accompanying drawings in the instruction manual. Figures 1-2 A dynamic user segmentation and loan marketing system based on the Doris real-time data warehouse, comprising a data acquisition layer, a data storage layer, a rule configuration layer, a real-time computing layer, and a marketing execution layer; I. Data Acquisition Layer: Deploy a Flink cluster containing 3 JobManagers and 10 TaskManagers, and a Kafka cluster containing 3 message broker nodes. Deploy Flink-CDC (Change Data Capture) tasks to monitor the MySQL business database in a master-slave architecture to avoid affecting the production database. Configure the number of replicas of the Kafka topic to 3 to ensure data reliability.

[0028] The data acquisition layer performs the following operations: 1.1 Data Capture: Flink-CDC (Change Data Capture) technology is used to capture the addition, modification and deletion operations of user basic information (such as identity and credit limit) and behavioral data (such as LOGIN and APPLY) in the MySQL business database in real time, avoiding the batch synchronization delay of traditional ETL tools.

[0029] 1.2 Data Buffering and Distribution: A data buffering layer is built through a Kafka cluster, and topics are split according to data type: user attribute topics (topic_user_base) and user behavior topics (topic_user_action), to decouple data production and consumption and support peak traffic shaping.

[0030] 1.3 Real-time writing to Doris: Based on Doris's Routine Load mechanism, subscribe to the corresponding Kafka topic and write the cleaned data to the user basic information table and behavior information table of the Doris data warehouse in real time, with the import latency controlled within seconds.

[0031] II. Data Storage Layer: Deploy a Doris cluster consisting of 1 FE master node (front-end master node), 2 FE slave nodes (front-end slave nodes), and 10 BE nodes (back-end nodes). Use Colocate Join to optimize table join queries. The user basic information table and behavior information table are bucketed according to the USER_ID field to improve query efficiency.

[0032] The data storage layer performs the following operations: A user base information table (dm_user_base_info) is constructed based on the user base model to store and manage multi-dimensional user attribute information, including: Identity verification information includes basic information such as user ID (USER_ID), registered name, mobile phone number, email address, and ID card number. Sensitive information (mobile phone number (C_MOBILE_SHA256) and ID card number (C_IDNO_SHA256)) is encrypted and stored using the SHA256 encryption algorithm, which supports efficient privacy computing while ensuring data security and compliance. Demographic characteristics: Multiple tags such as gender, date of birth, age, occupation, industry, education level, and marital status provide rich dimensions for user profile analysis; Financial business attributes: Current available credit limit (USER_AVAILABLE_AMT), outstanding principal (USER_REMAIN_CORPUS), cash loan identifier, scenario loan industry code, and other business-related indicators; Risk assessment indicators include risk control dimensions such as the USER_CARD_A_SCORE score and the B-card score. Marketing Tags: User marketing tags and their activation status, supporting precision marketing strategies; User marketing tags include: login within one hour without credit approval, exposure of the withdrawal page within one hour without withdrawal, more than 2 successful SMS messages sent in the past 7 days, temporary increase in cash loan credit limit, permanent increase in cash loan credit limit, cash loan credit approved for 30 days without withdrawal, cash loan credit approved but not withdrawn, etc. Timestamp Management: Records data by adding timestamps, supporting data version management and historical traceability.

[0033] A user behavior information table (dm_user_action_info) is constructed using a user behavior model. By combining behavioral metadata with multiple types of extended fields, it enables real-time capture and efficient storage of dynamic user behavior across all channels and scenarios. This data is used to record and analyze the user's entire behavioral trajectory, including: Behavior identification system: Establish unique identifiers through behavior ID and user ID to support behavior tracking and correlation analysis; The behavior ID is predefined, for example: Cash loan credit granting behavior ID-1, scenario loan borrowing behavior ID-2, cash loan borrowing behavior ID-3, scenario loan repayment behavior ID-4, cash loan repayment behavior ID-5, exposure behavior ID-6, click behavior ID-7, cash loan credit limit increase behavior ID-8, reach behavior ID-9, review behavior ID-10. Behavior classification dimensions: a three-level classification system of behavior type (such as LOGIN, APPLY), behavior action (such as CLICK, SUBMIT), and behavior status, supporting fine-grained behavior classification; The behavioral status revolves around the entire lifecycle of a user submitting a loan application and receiving a loan. The actions triggered by filling in and submitting relevant information in the loan application are in the saved state. The actions after the application is successfully submitted are in the review state. After the review is approved, it is in the disbursement state. After the loan is successfully disbursed, it is in the disbursement success state. The actions triggered by the repayment page after the loan is successfully disbursed are in the post-loan state. Time-based management: Records the initiation and end times of actions, supporting action duration calculation and time-series analysis; Channel and Context Management: Records information on the channels and contexts in which behaviors occur, and supports multi-channel attribution analysis; For example: Channel Scenarios In-client exposure / in-client clicks Outreach Marketing Outreach Repayment scenarios: Loan repayment / Cash loan repayment Central Bank Re-examination Loan scenario loan / cash loan Credit line cash loan credit line Increase cash loan credit limit; Flexible expansion mechanism: Three types of extended fields are designed: character, numeric, and date. Each type contains multiple reserved fields (4 for character, 3 for numeric, and 3 for date), supporting rapid business iteration and adaptation to new scenarios; Data timeliness management: Data is stored in partitions by date, supporting incremental data processing and historical data query.

[0034] An attribute configuration table (CDP_ATTR_DATA) is established through the attribute metadata model: This table manages the metadata information of user attributes, enabling standardized definitions of attributes, including: Attribute identification management: a dual identification system of attribute name (system identifier, such as user available credit USER_AVAILABLE_AMT) and attribute display name (business identifier, such as "available credit"); Data type definition: Supports four data types: character, numeric, enumeration, and date, to meet the needs of different business scenarios; Update timeliness control: Differentiates between real-time updates and offline updates, and supports different data refresh strategies; Logical deletion mechanism: Uses logical deletion flags, retains historical data, and supports data recovery; Audit traceability function: Records the creator, creation time, updater, and update time to achieve a complete data change audit chain; Index optimization design: Create indexes on attribute names, attribute display names, and data type fields to improve query efficiency.

[0035] A behavior attribute configuration table (CDP_BEHAVIOR_ATTR_DATA) is established through the behavior attribute metadata model: This table is used to manage the attribute metadata of user behavior and to standardize the definition of behavior attributes, including: Behavioral classification system: A three-level classification and identification system is established based on behavior type (such as login), behavior action, and behavior ID; Behavioral attribute definition: dual identification of behavioral attribute name and behavioral attribute display name, supporting decoupling between technology and business; The display name for behavior attributes is for convenience. When configuring business rules, business personnel can easily and intuitively view the Chinese names of attribute table fields. The same field may display different names in different behavior types. For example: Behavior type, behavior action, behavior attribute, attribute display name Loan scenario: Loan user ID (USER_ID) Loan Scenario: Loan BEHAVIOR_STATUS - Order Status Expose the EXT_CHAR1 event name within the event tracking client. The START_TIME event time of exposure within the event tracking client. Expose EXT_CHAR2 event encoding within the event tracking client Repayment Scenario: Repayment START_TIME (Event Repayment Time) Repayment Scenarios Repayment Scenarios Type Credit line for cash loans: START_TIME (credit line initiation time) Credit line cash loan credit line SCENE Credit line scenario types Data type standard: Supports four data types: character, numeric, enumeration, and date; Lifecycle management: logical deletion mechanism and complete audit traceability function; Efficient query support: Create composite indexes on behavior type, behavior action, behavior ID, behavior attribute display name, and data type fields.

[0036] III. Rule Configuration Layer: A web management interface is developed based on the Spring Boot framework. This interface is deployed on two application servers using a load balancing strategy. It supports visual configuration of attribute definitions, tag rules, grouping rules, and marketing rules. All configuration data is stored uniformly in a MySQL configuration database. Specifically, it includes: Visual rule configuration: Based on the attribute configuration table (CDP_ATTR_DATA) and the behavior attribute configuration table (CDP_BEHAVIOR_ATTR_DATA), the label rules can be configured visually, supporting multiple attribute types such as character, numeric, enumeration, and date.

[0037] Dual-mode rule activation: Supports both real-time update and offline update rule activation modes.

[0038] Event-driven computation: When a user triggers a specified event (such as logging into the APP or submitting a repayment), the rule engine is automatically activated, eliminating the need for scheduled batch scheduling and ensuring the real-time performance of tag calculation.

[0039] IV. Real-time Computing Layer: Deploys a rule engine service based on the Drools framework and a tag computing service developed in Java; both services adopt a 3-node cluster deployment mode and use Redis to cache frequently accessed tag rules, effectively reducing the query pressure on the Doris cluster. Specifically, this includes: Tag calculation triggers: Triggering conditions include two categories: one is user behavior events (such as BEHAVIOR_TYPE=LOGIN, where the behavior is logging in), and the other is system scheduled tasks (such as daily calculation of "monthly repayment target users"). After triggering, the system queries the latest data of the user from Doris, loads the matching rules, and completes tag calculation and updates (such as the "high potential user" tag becoming effective / ineffective).

[0040] Dynamic grouping logic: Supports defining grouping rules based on multi-dimensional tag combinations, such as "high-potential users (tag 1) + active in the last 7 days (tag 2) + have not used AI outbound calling channels (tag 3)". Grouping calculation methods include: real-time grouping (automatically determines whether to join a group when user tags are updated) and scheduled grouping (e.g., batch calculation of all user group affiliation at 9:00 AM daily).

[0041] Segmentation Results Management: Segmentation results are written to the Doris segmentation results table in real time, supporting queries for the number of users in each segment, detailed lists, and historical segment changes (such as the increase or decrease of users in the "high-potential segment" yesterday vs. today), facilitating the tracking of marketing strategies.

[0042] V. Marketing Execution Layer: Deploys marketing scheduling services (integrating scheduled tasks and event monitoring capabilities), channel adaptation services (interfacing with various channel APIs), and performance analysis services (analyzing marketing data based on Doris). All services support horizontal scaling. Specifically, this includes: Marketing rule configuration: Bind differentiated rules to each segment, including: reach channel (SMS / AI outbound call / APP push), marketing content (such as "70% discount on loan interest rate" or "credit limit increased to 100,000"), reach timing (such as 5 minutes after user login or immediately after successful repayment), and reach restrictions (such as reaching the same user a maximum of 2 times within 3 days to avoid harassment).

[0043] Multi-channel integration: Integrate SMS gateways (such as Alibaba Cloud SMS), AI outbound calling platforms (such as iFlytek), and APP push interfaces, and call various channels through a unified API to automate marketing actions.

[0044] Results closed-loop tracking: Record marketing execution logs (such as sending time, channel, content) and user response data (such as clicking links, initiating loan applications, and successful loan applications), calculate the marketing conversion rate (application rate / success rate) for each segment, and provide data support for rule optimization (such as "the conversion rate of AI outbound calling channel is higher than that of SMS, so the proportion of this channel can be increased").

[0045] Example 2: A dynamic user segmentation and loan marketing system and method based on the Doris real-time data warehouse, including the following steps: Step 1: Data Synchronization Process (Real-time: Seconds) 1.1: User data in the MySQL business database has changed (e.g., the USER_REMAIN_CORPUS field is reduced after a user repays a loan). 1.2: The Flink-CDC task captures changed data and performs format cleaning (e.g., unifying the date format to YYYY-MM-DD, and removing duplicates from numeric fields). 1.3: The cleaned data is sent to the corresponding Kafka topic according to its type (user attribute change → topic_user_base, behavior change → topic_user_action); 1.4: Doris subscribes to Kafka topics through Routine Load tasks, consumes data in real time, and writes it to the user basic information table (dm_user_base_info) or user behavior information table (dm_user_action_info). At the same time, it automatically encrypts and stores mobile phone numbers and ID card numbers using the SHA256 encryption algorithm. 1.5: After the data is written, Doris returns a write confirmation, and Flink-CDC records the synchronization log for easy troubleshooting.

[0046] Step 2: Tag rule configuration and calculation process (real-time: 1-3 seconds) 2.1: Business personnel can select the "Tag Configuration" function through the Web management interface to load configurable attributes (such as "Available Credit" and "Number of Logins in the Last 7 Days") from the attribute configuration table (CDP_ATTR_DATA) and the behavior attribute configuration table (CDP_BEHAVIOR_ATTR_DATA). 2.2: Configure tag rules, such as the "High Potential User" tag: Select "Available Credit (USER_AVAILABLE_AMT) > 5000" and "Risk A Card Score (USER_CARD_A_SCORE) > 700", and set the rule activation mode to "Real-time"; 2.3: After the rules are configured, the system stores the rule logic (such as SQL condition fragments) in a MySQL rule table and synchronizes it to the Redis cache; 2.4: When a user triggers an event (such as logging into the app), the rules engine receives the event notification and loads the relevant tag rules for that user from Redis; 2.5: The rule engine calls the Doris query interface to obtain the latest USER_AVAILABLE_AMT and USER_CARD_A_SCORE data for the user; 2.6: The rule engine performs conditional judgments. If the rule is met, the user is tagged as a "high-potential user" and updated to the Doris user tag table (user_tag_result). 2.7: After the tag update is completed, the system triggers a cluster calculation notification.

[0047] Step 3: User segmentation process (real-time: 5-10 seconds) 3.1: Business personnel configure grouping rules, such as "high-potential and recently active users": select "high-potential users (tag 1)" and "have logged in in the last 7 days (tag 2)"; 3.2: Clustering rules are stored in a MySQL clustering table, and the system supports two calculation methods: Real-time grouping: After the user tags are updated (as in step 2.6), the grouping service automatically determines whether the user meets the grouping rules, and if so, adds the user to the group. Scheduled grouping: At a preset period (e.g., 9 AM daily), the grouping service queries all user tags in Doris in batches and calculates the group affiliation of each user. 3.3: The segmentation results are written to the Doris segmentation results table (user_segment_result), which supports real-time querying (e.g., the current number of "high potential and recently active users" is 125,000, as viewed through the management interface).

[0048] Step 4. Marketing Execution Process (Real-time: 10-20 seconds) 4.1: Configure marketing rules for the "high-potential and recently active users" segment: outreach channel (AI outbound calling), content ("Your loan limit has been increased to 100,000, click to claim"), outreach timing (5 minutes after user login), and restrictions (maximum of 1 outreach within 3 days). 4.2: When a user joins a target segment, the marketing dispatch service receives a segment change notification; 4.3: The dispatch service checks user reach restrictions (such as whether they have been reached in the last 3 days). If the conditions are met, the AI ​​outbound call channel interface is invoked. 4.4: The AI ​​outbound calling platform executes calls, records call results (such as connected / not connected / user interested), and synchronizes them to the Doris marketing log table (marketing_log); 4.5: If a user clicks on a marketing link to initiate a loan application, the behavioral data is written into Doris through the data synchronization process, and the performance analysis service calculates the marketing conversion rate for that segment (e.g., application rate of 25%, loan success rate of 18%). 4.6: Business personnel optimize marketing rules based on conversion rate data (e.g., if the AI ​​outbound call conversion rate is lower than the set value, change "AI outbound call" to "APP push" to improve the reach).

[0049] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic user segmentation and loan marketing system based on the Doris real-time data warehouse, characterized by: It includes a data acquisition layer, a data storage layer, a rule configuration layer, a real-time computing layer, and a marketing execution layer; The data acquisition layer deploys a Flink cluster, a Kafka cluster, and Flink-CDC tasks to monitor the MySQL business database in a master-slave architecture. The data storage layer constructs a user basic information table, a user behavior information table, an attribute configuration table, and a behavior attribute configuration table, deploys a Doris cluster, and performs bucketing processing on the user basic information table and the user behavior information table based on the USER_ID field. Rule configuration layer: The web management interface is developed based on the Spring Boot framework. Configuration data is uniformly stored in the MySQL configuration database. It supports the visual configuration of attribute definition, tag rules, grouping rules and marketing rules. Specifically, the visual configuration of tag rules is implemented based on the attribute configuration table and the behavior attribute configuration table. It supports both real-time update and offline update modes for rule activation and adopts an event-driven computing mechanism to automatically activate the rule engine when the user triggers a specified event. The real-time computing layer deploys a rule engine service based on the Drools framework and a tag computing service developed in Java. Both services cache frequently accessed tag rules through Redis. When a user behavior event or a system scheduled task is triggered, the system queries the latest user data from Doris, loads the matching rules, and completes tag calculation and updates. At the same time, it performs cluster calculation based on the clustering rules. The cluster calculation methods include real-time clustering and scheduled clustering, and the clustering results are written to the Doris clustering result table in real time. The marketing execution layer deploys marketing scheduling services, channel adaptation services, and performance analysis services.

2. The dynamic user segmentation and loan marketing system based on the Doris real-time data warehouse according to claim 1, characterized in that, The data acquisition layer uses Flink-CDC to capture changes in user basic information and behavioral data in the MySQL business database in real time. It builds a data buffer layer through a Kafka cluster and splits the data into two categories: user attributes and user behavior. Based on the Doris Routine Load mechanism, it subscribes to the Kafka topics and writes the cleaned data into the user basic information table and user behavior information table in the Doris data warehouse in real time.

3. The dynamic user segmentation and loan marketing system based on the Doris real-time data warehouse according to claim 2, characterized in that, A user base information table is built using a user base model to store and manage multi-dimensional user attribute information, including the following dimensions: identity information, demographic characteristics, financial business attributes, risk assessment indicators, marketing tags, and timestamp management.

4. The dynamic user segmentation and loan marketing system based on the Doris real-time data warehouse according to claim 3, characterized in that, A user behavior information table is constructed using a user behavior model to record and analyze the user's entire behavior trajectory, including the following dimensions: Behavioral Identification System: Establishes unique identifiers through behavioral IDs and user IDs; Behavioral classification dimensions: a three-level classification system of behavior type, behavior action, and behavior state; Time-based management: Records the initiation and end times of actions; Channel and scenario management: Records information about the channels and scenarios in which behaviors occur; Extension mechanism: Design three types of extended fields: character, numeric, and date. Data timeliness management: Store data by date in partitions.

5. The dynamic user segmentation and loan marketing system based on the Doris real-time data warehouse according to claim 4, characterized in that, An attribute configuration table is established through an attribute metadata model to manage the metadata information of user attributes and to achieve standardized definition of attributes, including the following dimensions: Attribute identification management: a dual identification system of attribute name and attribute display name; Data type definition: Supports four data types: character, numeric, enumeration, and date; Update timeliness control: Distinguish between two timeliness types: real-time updates and offline updates.

6. The dynamic user segmentation and loan marketing system based on the Doris real-time data warehouse according to claim 5, characterized in that, A behavior attribute configuration table is established through a behavior attribute metadata model: this table manages the attribute metadata of user behavior and standardizes the definition of behavior attributes, including the following dimensions: Behavioral classification system: A three-level classification and identification system is established based on behavior type, behavior action, and behavior ID; Behavioral attribute definition: Identified by both the behavioral attribute name and the behavioral attribute display name; Data type standard: Supports four data types: character, numeric, enumeration, and date.

7. The dynamic user segmentation and loan marketing system based on the Doris real-time data warehouse according to claim 6, characterized in that, The marketing execution layer deploys marketing scheduling services, channel adaptation services, and performance analysis services, including: binding differentiated rules to each segment, including reach channels, marketing content, reach timing, and reach restrictions; integrating multiple push interfaces; recording marketing execution logs and user response data, calculating marketing conversion rates for each segment; and optimizing rules.

8. A dynamic user segmentation and loan marketing method based on a Doris real-time data warehouse, utilizing the system described in any one of claims 1 to 7, characterized in that, Includes the following steps: Step 1: Data Synchronization 1.1: User data in the MySQL business database has been changed; 1.2: The Flink-CDC task captures changed data and performs format cleaning; 1.3: The cleaned data is sent to the corresponding Kafka topic according to its type; 1.4: Doris subscribes to Kafka topics through Routine Load tasks, consumes data in real time, and writes it to the user basic information table or user behavior information table. At the same time, it automatically encrypts and stores mobile phone numbers and ID card numbers using the SHA256 encryption algorithm. 1.5: After the data is written, Doris returns a write confirmation, and Flink-CDC records the synchronization log; Step 2: Tag Rule Configuration and Calculation 2.1: Through the Web management interface, select the "Label Configuration" function to load configurable attributes from the attribute configuration table and the behavior attribute configuration table; 2.2: Configure tag rules; 2.3: After the tag rules are configured, the system stores the rule logic in a MySQL rule table and synchronizes it to the Redis cache; 2.4: When a user triggers an event, the rules engine receives the event notification and loads the relevant tag rules for that user from Redis; 2.5: The rules engine calls the Doris query interface to obtain the latest tag rule matching data for the user; 2.6: The rule engine performs conditional judgments. If the rule is met, the corresponding tag is assigned to the user, and the tag information is updated to the Doris user tag table. 2.7: After the tag update is completed, the system triggers a cluster calculation notification; Step 3: User Segmentation 3.1: Configure grouping rules for business personnel; 3.2: Clustering rules are stored in the MySQL clustering table. Clustering calculation methods include real-time clustering and timed clustering. 3.3: Write the clustering results into the Doris clustering results table; Step 4. Marketing Execution.

9. The dynamic user segmentation and loan marketing method based on the Doris real-time data warehouse according to claim 8, characterized in that, In step 3.2, real-time grouping: after user tags are updated, the grouping service automatically determines whether the user meets the grouping rules, and if so, adds the user to the group; scheduled grouping: according to a preset period, the grouping service queries all user tags in Doris in batches and calculates the group affiliation of each user.

10. The dynamic user segmentation and loan marketing method based on the Doris real-time data warehouse according to claim 9, characterized in that, Step 4 is as follows: 4.1: Configure marketing rules for different user groups; 4.2: When a user joins a target segment, the marketing dispatch service receives a segment change notification; 4.3: The scheduling service checks user reach restrictions; if the conditions are met, it calls the push interface. 4.4: Execute push notifications, record the results, and synchronize them to the Doris marketing log table; 4.5: User behavior data is written to Doris through a data synchronization process, and the performance analysis service calculates the marketing conversion rate of each segment; 4.6: Optimize marketing rules based on conversion rate data.

Images