A method for implementing a dynamic rule engine for abnormal ride-hailing pick-up and drop-off risk control based on flink CEP

Abstract

The present application relates to the technical field of online car-hailing risk control, and particularly relates to a method for implementing an online car-hailing pick-up and drop-off abnormality risk control dynamic rule engine based on flink CEP. In the online car-hailing service, after a passenger issues an order on a mobile phone, the driver is affected by external environmental factors, vehicle condition factors, other human factors and complex environments during the process of picking up and dropping off the passenger, which may result in a variety of abnormal situations such as pick-up timeout, driver driving in the opposite direction, driver not moving and the like. A related risk control system is established in each service system. The present application uses mysql to dynamically load rules to ensure the dynamic scalability of the rules, and uses flink CEP to calculate the rules to ensure the timeliness and accuracy of the calculation results. The overall scheme supports the timeliness and dynamic scalability requirements of abnormal indicators in the risk control system under the premise of not stopping service when the rules are changed.

Description

Technical Field

[0001] This invention relates to the field of ride-hailing risk management technology, specifically a method for implementing a dynamic rule engine for ride-hailing pick-up and drop-off driver anomaly risk control based on Flink CEP. Background Technology

[0002] One of the core aspects of a risk control system is the calculation of abnormal indicators. The real-time nature and dynamic scalability of abnormal indicator calculation are the technical challenges in the overall solution.

[0003] The current common approach is to use microservices to load rule data into ZooKeeper or other configuration centers. After the microservices consume event data from the upstream message queue, they read other auxiliary data (such as theoretical values) from cache media such as Redis based on the event classification in the data, perform rule comparison calculations, and then obtain the final abnormal indicator results.

[0004] Current technologies generally rely on ZooKeeper or similar configuration centers to dynamically load rules, which increases the complexity and maintenance cost of the overall solution.

[0005] The calculation of specific abnormal indicators requires the use of other third-party caching media such as Redis. The overall calculation process increases the dependence on third parties, lengthens the interaction chain, and increases uncertainty and complexity.

[0006] To address this, a method is provided for implementing a dynamic rule engine for ride-hailing driver anomaly risk control based on Flink CEP. Summary of the Invention

[0007] In view of this, the present invention aims to provide a method for implementing a dynamic rule engine for ride-hailing pick-up and drop-off driver anomaly risk control based on Flink CEP, so as to solve or alleviate the technical problems existing in the prior art and provide at least a beneficial option for the above-mentioned technical problems.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A method for implementing a dynamic rule engine for ride-hailing driver anomaly risk control based on FlinkCEP, the method includes the following steps:

[0010] S1. When a driver responds to an order, starts a trip, or uploads their location at a 5-second timer, the business system sends the corresponding event data through the RocketMQ message queue.

[0011] S2. Use Flink to consume initial events and location upload events in the MQ. Filter events based on event type and order information. Then, merge the event data within the process. Join the streams by order ID and order status to obtain information such as order ID, initial estimated duration, initial estimated mileage, current estimated duration, and current estimated mileage from the two types of events. After merging, rewrite the data into the RocketMQ message queue.

[0012] S3, the anomaly indicator calculation task uses Flink, which consumes event data from S2's MQ by pulling data, and uses the event time in the event data and subtracts the allowed delay time to calculate the watermark in the task calculation;

[0013] S4 and Flink tasks load computation rules stored in MySQL in the form of real-time dimension tables and set the rule update TTL;

[0014] S5. Use FlinkCEP to define the indicator calculation logic, set the partition field, sorting field, calculation logic, output field, and timeout time in CEP. FlinkCEP splits the data according to the partition and sorting fields and then enters the concurrent calculation. It produces abnormal behavior results through logical operations or timeout events in the calculation logic. According to the set output fields, it produces the result details of abnormal indicators and outputs them to RocketMQ for use by downstream risk control systems.

[0015] S6. The calculation of abnormal indicators in the pick-up and drop-off process is completed.

[0016] As a further aspect of the present invention: in step S2, event filtering is performed based on event type and order information.

[0017] This means filtering out location upload events that are not part of the pick-up / drop-off process.

[0018] As a further aspect of the present invention: the process in S2 includes a pick-up process or a drop-off process.

[0019] As a further aspect of the present invention, the allowed delay time of S3 is set to 5 seconds.

[0020] As a further embodiment of the present invention: the MySQL storage in S4 can be replaced by HBase, Hologres, or ODPS.

[0021] As a further aspect of the present invention: the rule update TTL in S5 is set according to the shortest time expected for the rule to take effect in actual business.

[0022] As a further aspect of the present invention, the method further includes the following steps:

[0023] Step 1: Define a rule template file or rule base;

[0024] Step 2: Collect points of demand change, analyze the granularity of demand change, and implement corresponding dynamic data cleaning and improvement as attribute features for dynamic configuration; the cleaning script dynamically adds, modifies, deletes, publishes, and includes rollback logic;

[0025] Step 3: Execute the rule engine to realize dynamic configuration of attributes, including dynamic display, addition, modification, deletion, publication and rollback logic of logical and physical models, and dynamically and in real time to control risk points in advance.

[0026] As a further aspect of the present invention: the rule template file or rule base in step one has highly readable rule logic and is maintained by the user.

[0027] As a further aspect of the present invention: Step three is based on dynamic configuration of the rule engine. According to the configuration data cleaning and the filtering conditions contained in the rules, it is determined whether the rules can match the real-time conditions at the runtime to execute the engine. This process is divided into four stages:

[0028] A. First, configure and publish the rule factors;

[0029] B. Then, after the transaction source data comes in, it is cleaned and transformed into a rule data object and sent to the queue manager;

[0030] C. Then the rule engine matches the cleaned rule data in the queue manager according to the published rules;

[0031] D. Finally, risk management is carried out based on the matched rules.

[0032] The embodiments of the present invention have the following advantages due to the adoption of the above technical solutions:

[0033] (1) This solution uses a MySQL database as a real-time calculation dimension table instead of ZooKeeper as a dynamic rule storage center, uses Flink to complete the real-time merging and deduplication of event data, and uses FlinkCEP to complete the calculation and output of specific data indicators.

[0034] (2) This solution relies on a relational database as the rule configuration center. The operation is simple and controllable, and the maintenance cost is extremely low. Combined with the real-time loading TTL configuration of Flink dimension tables, the update and effective time of dynamic rules can be controlled according to business needs, without the need to take the computing service offline and restart it.

[0035] (3) All calculations in this solution rely on the CPU and memory of the Flink computing cluster. The result data will only be produced after the data calculation is completed. This will not put any computing performance pressure on the database instance that stores the final calculation results, and at the same time control the cost increase of related storage.

[0036] (4) All calculations in this solution rely on the computing power of the Flink cluster. When the amount of data increases, the data processing performance can be improved quickly by expanding the computing power of the cluster. The operation is simple and risk-free.

[0037] (5) Flexibility and maintainability: The solution has high flexibility in rule definition and attribute configuration, and can dynamically adjust rule templates and attribute features according to actual business needs. At the same time, users can maintain the rule base and logical model themselves to ensure the maintainability of the system.

[0038] (6) Real-time performance and accuracy: By collecting demand change points and dynamically cleaning the data, combined with real-time matching and risk control by the rule engine, the solution can realize real-time monitoring and handling of abnormal situations in the delivery of passengers, thereby improving the real-time performance and accuracy of risk control.

[0039] (7) Dynamic configuration and management: The solution can flexibly respond to changes in business needs and improve the system’s adaptability and scalability by dynamically configuring the rule engine and rule factors, as well as managing the dynamic cleaning scripts.

[0040] (8) Comprehensiveness and completeness: The solution covers multiple aspects such as rule definition, dynamic data cleaning, dynamic configuration of the rule engine, and real-time risk management, which can comprehensively and completely realize the monitoring and control of abnormal situations in the delivery of passengers, and improve the comprehensiveness and completeness of the system.

[0041] (9) Prevention and responsiveness: The solution has a certain degree of prevention and responsiveness by controlling risk points in advance and managing risks according to the matched rules, which can effectively reduce potential risks and respond quickly to abnormal situations.

[0042] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart of a method for implementing a dynamic rule engine for ride-hailing driver anomaly risk control based on FlinkCEP, as proposed in this invention.

[0045] Figure 2 This is a flowchart of an embodiment of a dynamic rule engine method for ride-hailing pick-up and drop-off driver anomaly risk control based on FlinkCEP proposed in this invention. Detailed Implementation

[0046] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0047] It is important to note that terms such as "first," "second," "symmetric," and "array" are used only to distinguish between descriptive and positional descriptions and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, features specified with terms such as "first" or "symmetric" may explicitly or implicitly include one or more of that feature; similarly, when the quantity of certain features is not limited by words such as "two" or "three," it should be noted that such features also explicitly or implicitly include one or more features.

[0048] like Figure 1 As shown, this invention discloses a method for implementing a dynamic rule engine for ride-hailing driver and passenger transport anomaly risk control based on FlinkCEP. The method includes the following steps:

[0049] In the ride-hailing pick-up and drop-off process, the calculation of abnormal indicators in abnormal risk control scenarios mainly relies on two types of events: initial events and real-time location events.

[0050] Initial event: The first event at the start of the process. The initial event for the pick-up process is the order acceptance event, which is when the driver accepts the order and starts driving to the passenger pick-up point, including the initial time and initial distance from the current location to the destination.

[0051] The initial event in the passenger drop-off process is the start of the journey event, which is the start of the journey to the passenger's destination after the passenger boards the vehicle.

[0052] Real-time location events include the initial time and initial mileage from the current location to the destination: During pick-up and drop-off, the driver uploads a real-time location from the terminal every 5 seconds, including the estimated time and estimated mileage from the current location to the destination; the specific calculation steps for abnormal indicators are as follows:

[0053] S1. When a driver responds to an order, starts a trip, or uploads their location at a 5-second timer, the business system sends the corresponding event data through the RocketMQ message queue.

[0054] RocketMQ Message Queue: This method uses RocketMQ as a message queue to collect and send event data of ride-hailing pick-up and drop-off. RocketMQ is a highly available, high-throughput, distributed message queue system that can achieve reliable asynchronous communication and ensure reliable transmission of event data.

[0055] S2. Use Flink to consume initial events and location upload events in the MQ. Filter events based on event type and order information to remove location upload events outside the pick-up and drop-off process. Then, merge the event data of the pick-up and drop-off processes within the process by joining the streams using order ID and order status to obtain information such as order ID, initial estimated duration, initial estimated mileage, current estimated duration, and current estimated mileage from both types of events. After merging, rewrite the data into the RocketMQ message queue.

[0056] Flink consumes RocketMQ: Through Flink's RocketMQ connector, event data is consumed from RocketMQ. This connector enables data interaction between Flink and RocketMQ, allowing Flink to directly consume data from RocketMQ and perform subsequent processing and computation.

[0057] Stream merging and stream join: When processing initial events and real-time location events, it is necessary to perform stream merging and stream join operations on the two types of event data. Stream merging combines the event data in the pick-up process and the drop-off process to form a unified data stream; while stream join associates the two types of event data based on the order ID and order status to obtain order-related information.

[0058] S3, the anomaly indicator calculation task uses Flink, which consumes event data from S2's MQ by pulling data. The event time in the event data is used and the allowed delay time is subtracted to set it to 5 seconds. This can be adjusted according to the actual business situation and used as a watermark in the task calculation.

[0059] Event-time-based window computation: In the anomaly metric calculation task, Flink's event-time concept is required for window computation. By setting the event time and the allowed delay time, the time window of the event can be determined, and the anomaly metric can be calculated within the window.

[0060] S4. Flink tasks load computation rules from MySQL storage (other storage media that support dimension table mode reading can be used instead, such as HBase, Hologres, and ODPS) in the form of real-time dimension tables, and set the rule update TTL according to the shortest time expected for the rules to take effect in actual business.

[0061] Real-time dimension table loading: To load computation rules, this method uses real-time dimension tables, storing the rules in MySQL and loading them in real time via Flink. Real-time dimension tables ensure timely rule updates and provide efficient rule querying and matching.

[0062] S5. Use FlinkCEP to define the indicator calculation logic, set the partition field, sorting field, calculation logic, output field, and timeout time in CEP. FlinkCEP splits the data according to the partition and sorting fields and then enters the concurrent calculation. It produces abnormal behavior results through logical operations or timeout events in the calculation logic. According to the set output fields, it produces the result details of abnormal indicators and outputs them to RocketMQ for use by downstream risk control systems.

[0063] Using FlinkCEP: This method utilizes the FlinkCEP (ComplexEventProcessing) library to define the calculation logic for anomaly metrics. By setting partition fields, sorting fields, calculation logic, output fields, and timeouts, FlinkCEP can perform complex pattern matching and event sequence analysis on the input stream, thereby generating outputs of anomalous behavior results and anomaly metrics.

[0064] S6. The calculation of abnormal indicators in the pick-up and drop-off process is completed.

[0065] As a further solution of the present invention: in S2, event filtering is performed based on event type and order information, that is, location upload events in non-pickup and drop-off processes are filtered out.

[0066] like Figure 2 As shown, in one embodiment, the method further includes the following steps:

[0067] Step 1: Define a rule template file or rule library. The rule logic in the rule template file or rule library is highly readable and is maintained by the user.

[0068] Rule template file or rule base definition: A rule template file or rule base needs to be defined. The rule logic should be easy to read and easy for users to maintain and manage.

[0069] Step 2: Collect points of demand change, analyze the granularity of demand change, and implement corresponding dynamic data cleaning and improvement as attribute features for dynamic configuration; the cleaning script dynamically adds, modifies, deletes, publishes, and includes rollback logic;

[0070] Dynamic data cleaning and attribute feature extraction: By collecting the points of change in requirements and analyzing the granularity of the changes, corresponding dynamic data cleaning and improvement are achieved, and the data is used as dynamically configured attribute features. The cleaning script needs to support dynamic addition, modification, deletion and publication, and have rollback logic.

[0071] Step 3: Execute the rule engine to realize dynamic configuration of attributes, including dynamic display, addition, modification, deletion, publication and rollback logic of logical and physical models, and dynamically and in real time to control risk points in advance.

[0072] Dynamic configuration of the rule engine: In step three, the rule engine implements dynamic configuration of attributes. This includes the dynamic display, addition, modification, deletion, and publication of logical and physical models, as well as corresponding rollback logic. This enables the rule engine to achieve flexibility and scalability, facilitating dynamic adjustments based on business needs.

[0073] Based on a rule engine and dynamically configured, the system determines whether the rules match the real-time conditions at runtime by cleaning the configuration data and filtering conditions within the rules. The process is divided into four stages:

[0074] A. First, configure and publish the rule factors;

[0075] Rule factor configuration and deployment: The first step is to configure and deploy rule factors. Rule factors are the basic units used for judgment and calculation in the rule engine, including various attribute features, conditions and operators. By configuring and deploying rule factors, the necessary calculation foundation can be provided for the rule engine.

[0076] B. Then, after the transaction source data comes in, it is cleaned and transformed into a rule data object and sent to the queue manager;

[0077] Data cleaning and transformation: The transaction source data will undergo a cleaning process and be transformed into a rule data object before being sent to the queue manager. This process involves technologies such as data cleaning, transformation and transmission to ensure the accuracy and consistency of the data.

[0078] C. Then the rule engine matches the cleaned rule data in the queue manager according to the published rules;

[0079] Data cleaning and transformation: The transaction source data will undergo a cleaning process and be transformed into a rule data object before being sent to the queue manager. This process involves technologies such as data cleaning, transformation and transmission to ensure the accuracy and consistency of the data.

[0080] D. Finally, risk management is carried out based on the matched rules.

[0081] Real-time risk point prevention and control: The rule engine will determine whether it matches the real-time conditions at the time of runtime based on the configuration data cleaning and the filtering conditions in the rules, and execute the engine logic defined in the rules. This can realize real-time risk point prevention and control, and timely detection and response to abnormal situations.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0083] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. The above are merely embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for implementing a dynamic rule engine for ride-hailing driver anomaly risk control based on FlinkCEP, characterized in that, Initial event: The first event at the start of the process. The initial event for the pick-up process is the order acceptance event, which is when the driver accepts the order and starts driving to the passenger pick-up point, including the initial time and initial distance from the current location to the destination. The initial event of the driver drop-off process is the trip start event, which is the start of the trip from the passenger's boarding to the passenger's destination. The real-time location event includes the initial time and initial mileage from the current location to the destination. During the pick-up and drop-off process, the driver uploads a real-time location from the terminal every 5 seconds, including the estimated time and estimated mileage from the current location to the destination. This method includes the following steps: S1. When a driver responds to an order, starts a trip, or uploads their location at a 5-second timer, the business system sends the corresponding event data through the RocketMQ message queue. S2. Use Flink to consume the initial events and location upload events in the MQ. Filter the events according to the event type and order information. Then, merge the event data within the process. Join the streams by order ID and order status to obtain the order ID, initial estimated duration, initial estimated mileage, current estimated duration, and current estimated mileage information from the two types of events. After merging, rewrite the data into the RocketMQ message queue. In S2, event filtering is performed based on event type and order information, specifically filtering out location upload events that are not part of the pick-up and drop-off process. Stream merging and stream join: When processing initial events and real-time location events, it is necessary to perform stream merging and stream join operations on the two types of event data. Stream merging combines the event data in the pick-up process and the drop-off process to form a unified data stream; while stream join associates the two types of event data based on the order ID and order status to obtain order-related information. S3, the anomaly indicator calculation task uses Flink, which consumes event data from S2's MQ by pulling data, and uses the event time in the event data and subtracts the allowed delay time to calculate the watermark in the task calculation; S4 and Flink tasks load computation rules stored in MySQL in the form of real-time dimension tables and set the rule update TTL; S5. Use FlinkCEP to define the indicator calculation logic, set the partition field, sorting field, calculation logic, output field, and timeout time in CEP. FlinkCEP splits the data according to the partition and sorting fields and then enters the concurrent calculation. It produces abnormal behavior results through logical operations or timeout events in the calculation logic. According to the set output fields, it produces the result details of abnormal indicators and outputs them to RocketMQ for use by downstream risk control systems. S6. The calculation of abnormal indicators in the pick-up and drop-off process is completed; The method also includes the following steps: Step 1: Define a rule template file or rule base; Step 2: Collect demand change points, analyze the granularity of demand changes, and implement corresponding dynamic data cleaning and improvement as attribute features for dynamic configuration; the cleaning script dynamically adds, modifies, deletes, publishes, and rolls back logic; Step 3: Execute the rule engine to realize dynamic attribute configuration, including dynamic display, addition, modification, deletion, publication, and rollback logic of logical and physical models, and dynamically manage risk points in advance in real time.

2. The method for implementing a dynamic rule engine for ride-hailing vehicle pick-up and drop-off anomaly risk control based on Flink CEP according to claim 1, characterized in that: The process in S2 includes either a pick-up process or a drop-off process.

3. The method for implementing a dynamic rule engine for ride-hailing vehicle pick-up and drop-off anomaly risk control based on Flink CEP according to claim 2, characterized in that: The S3 allows a delay time of 5 seconds.

4. The method for implementing a dynamic rule engine for ride-hailing vehicle pick-up and drop-off anomaly risk control based on Flink CEP according to claim 3, characterized in that: In S4, MySQL storage can be replaced by HBase, Hologres, or ODPS.

5. The method for implementing a dynamic rule engine for ride-hailing vehicle pick-up and drop-off anomaly risk control based on Flink CEP according to claim 4, characterized in that: The rule update TTL in S5 is set according to the shortest time expected for the rule to take effect in actual business.

6. The method for implementing a dynamic rule engine for ride-hailing vehicle pick-up and drop-off anomaly risk control based on Flink CEP according to claim 5, characterized in that: The rule template file or rule base in step one has highly readable rule logic and is maintained by the user.

7. A method for implementing a dynamic rule engine for ride-hailing vehicle pick-up and drop-off anomaly risk control based on Flink CEP according to claim 6, characterized in that, Step three is based on dynamic configuration of the rule engine. It determines whether the configuration data cleaning and filtering conditions within the rules match the real-time conditions at runtime to execute the engine within the rules. This process is divided into four stages: A. First, configure and publish the rule factors; B. Then, after the transaction source data comes in, it is cleaned and converted into a rule data object and sent to the queue manager. C. Then the rule engine matches the cleaned rule data in the queue manager according to the published rules; D. Finally, risk management is carried out based on the matched rules.

Images