A dynamic rule message interaction method and system based on multi-level processing

Abstract

The present invention relates to a dynamic rule message interaction method and system based on multi-level processing, which uses the method of adding queue tags and consumption tags to realize the simultaneous processing of multiple different requests and multi-level processing of source data under different data requirements and corresponding verification. Through layered streaming calculation in series, lightweight or horizontal and vertical application expansion can be achieved, which is different from the original simple consumer producer model. The flexible and configurable design greatly improves the degree of productization, enables seamless access to various application scenarios to improve development efficiency and save costs, and can quickly realize data analysis and statistics to present data value faster.

Description

Technical Field

[0001] The present invention relates to the technical field of information distributed processing and big data processing, and in particular to a dynamic rule message interaction method and system based on multi-level processing. Background Art

[0002] Existing enterprise architectures primarily utilize message brokers as a mechanism for asynchronous data processing, encompassing a single producer-consumer application scenario, from application scenarios to message brokers, and ultimately to data consumption. This approach aims to address single scenarios through tool-based processing, failing to extend to productization, centralization, and platformization.

[0003] Typically, asynchronous messaging scenarios between enterprise systems are complex and numerous. Building only middleware mechanisms between production and consumption ends fails to effectively decouple the logic between message and application processing, and also leads to duplication of development and high costs. Statistics and analysis of data in messaging scenarios can only be performed through final results or through complex data mining to derive value from the data.

[0004] Production business applications generate a large amount of message-based data during actual operation. For example, some scenarios require pushing information to customers via SMS, WeChat messages, and emails. New customer registration requires aggregated customer data and subsequent processing of new customer business. After a customer completes payment, messages are used to connect logistics, customer service platforms, and other post-processing logic. This massive amount of message data is massive, characterized by randomness and a complex set of rules. This requires a high degree of flexibility and scalability in the system's computation and processing, resulting in a high level of overall complexity. With the widespread adoption of distributed microservices architectures in enterprise-level system development, online service capabilities are becoming increasingly powerful, and inter-service dependencies and collaboration are becoming increasingly close. The concept of integrating middle-office capabilities, such as service and data middle-offices, is also maturing. However, the constant challenge is the homogenization of various application technologies, which is the massive data processing required.

[0005] The diversity of service scenarios and numerous channels have made the interactions between systems and data formats chaotic, posing great challenges to process connection, data synchronization, and the consistency and timeliness of information access. Summary of the Invention

[0006] In order to address the shortcomings of the existing technology, the present invention proposes a dynamic rule message interaction method and system based on multi-level processing. Through layered streaming computing in series, lightweight or horizontal and vertical application expansion can be achieved. Different from the original simple consumer producer model, the flexible and configurable design greatly improves the degree of productization, so that various application scenarios can be seamlessly connected to improve development efficiency and save costs, and can quickly realize data analysis and statistics to present data value faster.

[0007] To achieve the above objectives, the technical solutions adopted by the present invention include:

[0008] A dynamic rule message interaction method based on multi-level processing, characterized by comprising:

[0009] S1. Obtain source data, add a queue tag and a consumption tag to the source data to form first processed data, wherein the queue tag is added with a default value, and the consumption tag includes a processing level value and a consumption status;

[0010] S2. Obtain data requests and assign processing queues to them, analyze the data requests to extract data sources and data processing level requirements;

[0011] S3. Use the data source to match the source data, select the corresponding first processing data, and modify the queue label of the first processing data to the processing queue;

[0012] S4. Determine whether the processing level value matches the data processing level requirement value. When it is determined that the processing level value matches the data processing level requirement value, use the first processed data as the processing result data.

[0013] S5. When it is determined that the processing level value does not match the data processing level requirement value, perform a data processing operation on the first processed data according to a preset rule and modify the processing level value accordingly, until it is determined that the processing level value matches the data processing level requirement value, and output the processing result data;

[0014] S6. Consume the processing result data and change the consumption status of the consumed processing result data accordingly.

[0015] Furthermore, executing the data processing operation according to the preset rules and modifying the processing level value accordingly includes:

[0016] a1. Configure a unique execution level value for each preset rule;

[0017] a2. Select any preset rule and determine whether to execute the preset rule;

[0018] a3. When it is determined that the preset rule is to be executed, performing corresponding processing on the first processed data according to the preset rule;

[0019] a4. Modify the processing level value of the first processed data using the execution level value corresponding to the preset rule used for the execution process;

[0020] a5. Repeat steps a2 to a4 until all preset rules are traversed and the modified processing level value of the first processed data matches the data processing level requirement value.

[0021] Furthermore, the step S1 further includes:

[0022] Set the processing level value to 1.

[0023] Furthermore, the execution level value includes a positive odd number other than 1;

[0024] The modifying the processing level value of the first processed data using the execution level value corresponding to the preset rule used for executing the processing includes adding the execution level value to the existing processing level value to obtain a modified processing level value.

[0025] Furthermore, the step S2 further includes:

[0026] Get multiple data requests and assign different processing queues to the multiple data requests.

[0027] Furthermore, the method further comprises:

[0028] Process multiple data requests simultaneously without interfering with each other based on different processing queues.

[0029] The present invention also relates to a dynamic rule message interaction system based on multi-level processing, characterized in that it includes:

[0030] A source data processing module is used to obtain source data and add a queue tag and a consumption tag to the source data to form first processed data;

[0031] The data request processing module is used to obtain data requests and assign processing queues to data requests, analyze data requests to extract data sources and data processing level requirements;

[0032] a first matching module, configured to match source data using a data source and select corresponding first processed data;

[0033] A second matching module is used to determine whether the processing level value matches the data processing level requirement value;

[0034] a hierarchical processing module, configured to perform a data processing operation on the first processed data according to a preset rule and modify the processing hierarchical value accordingly;

[0035] The consumption module is used to consume the processing result data.

[0036] The present invention also relates to a computer-readable storage medium, characterized in that a computer program is stored on the storage medium, and the computer program implements the above method when executed by a processor.

[0037] The present invention also relates to an electronic device, characterized in that it comprises a processor and a memory;

[0038] The memory is used to store the first processing data and the data processing level requirement value;

[0039] The processor is configured to execute the above method by calling the first processing data and the data processing level requirement value.

[0040] The present invention also relates to a computer program product, comprising a computer program and / or instructions, characterized in that the computer program and / or instructions implement the steps of the above method when executed by a processor.

[0041] The beneficial effects of the present invention are:

[0042] The dynamic rule message interaction method and system based on multi-level processing described in the present invention can achieve lightweight or horizontal and vertical application expansion through layered streaming computing in series, which is different from the original simple consumer producer model. It supports horizontal and vertical expansion and flexible pluggable assembly product design. The flexible and configurable design greatly improves the degree of productization, allowing each application scenario to be seamlessly connected to improve development efficiency and save costs, and can quickly realize data analysis and statistics to present data value faster, solving the message processing capabilities between diversified systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The figure is a flow chart of the dynamic rule message interaction method based on multi-level processing of the present invention.

[0044] Figure 2 This is a structural diagram of the dynamic rule message interaction system based on multi-level processing of the present invention. DETAILED DESCRIPTION

[0045] In order to more clearly understand the content of the present invention, it will be described in detail with reference to the accompanying drawings and embodiments.

[0046] The first aspect of the present invention relates to a process of steps as follows Figure 1 The dynamic rule message interaction method based on multi-level processing shown includes:

[0047] S1. Acquire source data, add a queue tag and a consumption tag to the source data to form first processed data, wherein the queue tag is added with a default value, and the consumption tag includes a processing level value and a consumption status.

[0048] Preferably, the processing level value can be preset to 1 to facilitate the subsequent calculation and modification process and the matching process of the data processing level. Of course, for different data processing requirements and matching habits, the processing level value can also be preset to 0 or any other appropriate value. The preset value itself does not have any corresponding meaning. The purpose of the preset value is to represent the starting value of the calculation of the data processing level and mark the current state of the first processed data. For example, in the case where no data processing has been performed, the preset processing level value actually represents that the existing first processed data is consistent with the unprocessed source data.

[0049] In practice, it's best to use Kafka message brokers to build a data collection cluster, centrally collecting message data from all production terminals and forming a standardized and unique data pair. For example, in an application domain with multiple business scenarios, each business processing scenario can apply for a corresponding tenant and session pair, and then create and submit configuration files in a standardized data format.

[0050] S2. Obtain data requests and assign processing queues to them, analyze the data requests to extract data sources and data processing level requirements.

[0051] Preferably, by using a set queue tag, multiple different data requests can be obtained at the same time and the corresponding processing queues can be assigned to them respectively. When the source data (first processing data) is selectively called, its queue tag can be modified accordingly, so that the same source data (first processing data) can be called by multiple different data requests at the same time without interfering with each other. The subsequent data processing of the source data (first processing data) based on preset rules also corresponds to distinguishing its target data request with a queue tag, so that multiple data requests can be executed simultaneously without interfering with each other, greatly improving processing efficiency.

[0052] S3. Use the data source to match the source data, select the corresponding first processed data, and modify the queue label of the first processed data to the processing queue.

[0053] Specifically, Flink can be used in conjunction with a Kafka data collection cluster for sequential consumption and topology calculation. Leveraging the Flink streaming computing framework, multi-layer data filtering processors can be built to decouple and control combined business logic scenarios. Stateful computing can be used to implement data rollback and multi-system consistency support scenarios. Furthermore, a configuration center allows for parameterized business logic calculations. Business personnel or system administrators can use the configuration center to uniformly set and modify system logic processing parameters, increasing data calculation flexibility.

[0054] The mapping between message queues and data processing stacks is configured. Different message queue sessions can correspond to multiple data processing stacks. These configurations take effect after being submitted to the Flink configuration center. The data structure of each message body must be defined according to standards. If multiple tenants rely on it, changes must be coordinated when they occur. When computing the message body in a message queue, the corresponding data processing stack is loaded by configuring the mapping information for processing.

[0055] S4. Determine whether the processing level value matches the data processing level requirement value. When it is determined that the processing level value matches the data processing level requirement value, use the first processed data as the processing result data.

[0056] At this time, the data request corresponds to the unprocessed source data (first processed data), which can be directly fed back.

[0057] During the judgment process, application developers can write rule-based message processing logic for each message and submit it to the Flink configuration center. After the Flink message processing unit completes data calculations, it outputs the results according to the configured logic. For example, if a customer needs to be notified of important information via SMS, multiple reminders can be configured and assigned to an account manager if the calculation logic determines that the user is a low-value customer. Similarly, multi-tenant, multi-application IDs are used to isolate the message processing unit logic in the configuration center.

[0058] S5. When it is determined that the processing level value does not match the data processing level requirement value, a data processing operation is performed on the first processed data according to a preset rule and the processing level value is modified accordingly, until it is determined that the processing level value matches the data processing level requirement value, and the processing result data is output.

[0059] Specifically, executing data processing operations according to preset rules and correspondingly modifying the processing level value include: a1, configuring a unique corresponding execution level value for each preset rule. When the processing level value is preset to 1, the execution level value is preferably selected as a positive odd number other than 1; a2, selecting any preset rule and determining whether to execute the preset rule; a3, when determining to execute the preset rule, performing corresponding processing on the first processed data according to the preset rule; a4, using the execution level value corresponding to the preset rule used for execution processing to modify the processing level value of the first processed data. For example, you can choose to add the execution level value of the selected preset rule to obtain a new processing level value; a5, repeatedly executing a2 to a4 until all preset rules are traversed, until the modified processing level value of the first processed data matches the data processing level requirement value.

[0060] In complex business computing and processing scenarios, data rule matching can be completed by calling a unified rule engine, such as the number of times a reminder event in SMS reminder message data needs to be touched and whether an SMS receipt is required, and other business rule determinations. Similarly, business personnel can customize the rule data loaded in the rule engine through the background. Both the configuration center and the rule engine configuration end provide users with efficient definition and production through low-code templates. After filtering and matching through the rule engine, a flow set of successfully matched rules, a flow set of unmatched rules, and a flow set of matched rules but inconsistent elements will be generated. One flow can successfully match multiple rules. Finally, the successfully matched result will be transferred to the next calculation element for downstream consumption.

[0061] When processing a single message, Flink can encapsulate multi-level processing within Task Slots, allowing the rules engine to match static rules. For example, these rules can be used to determine whether a customer's order is from a premium merchant or whether the amount range in the order information meets the merchant's return rate rules. These rules can be centrally managed within the rules engine, allowing application business personnel or system administrators to build them within the engine platform. The following is the overall process for rule engine design.

[0062] During specific execution, the result data after calculation and processing by the Flink cluster is pushed to the new Kafka cluster. The Kafka cluster here serves as the result data collection end, forming a standardized and unified message data pair. Downstream business applications can access and consume the required message data on demand to complete the post-processing of the business scenario.

[0063] S6. Consume the processing result data and change the consumption status of the consumed processing result data accordingly.

[0064] By applying this method, the result data from real-time message processing can be persisted and pushed to the messaging engine's data BI application, enabling data visualization and value presentation. In addition to unified business data calculation, it also integrates backend management data from each cluster, providing a centralized operational data visualization center and a platform-based cockpit.

[0065] When applied to multi-service scenarios, data flows and data definitions can be further evolved, enabling basic API encapsulation. Architecturally, a single set of code can handle both stream storage and batch storage. The overall message engine platform is the beginning of a real-time data warehouse. Through configuration center capabilities and seamless application access, it can be expanded in multiple dimensions to achieve the integration of multiple business scenarios. For example, message reminders can be implemented in multiple rounds and multiple modes, using SMS, WeChat, system messages, and other means to reach customers across multiple terminals. Customer registration enables pseudo-real-time customer data measurement and customer service processing. Customer order completion enables pseudo-real-time large-screen statistics of various operational indicators, as well as calculation and analysis of order product and merchant tags.

[0066] The above method can also be optimally implemented using a distributed microservices architecture. Service capabilities are published in a unified registration center for each service application, enabling service registration and discovery. The API gateway serves as the access point for each service, providing dynamic load balancing. Zookeeper provides distributed coordination, maintains configuration information, and performs heartbeat monitoring for service applications. Each product backend application and the configuration center backend application are microserviced according to business capability boundaries for better decoupling and facilitating iterative upgrades of fundamental capabilities. Foundational technical frameworks such as Flink and Kafka naturally support distributed architecture design and deployment.

[0067] Another aspect of the present invention relates to a dynamic rule message interaction system based on multi-level processing, the structure of which is as follows: Figure 2 Shown, including:

[0068] A source data processing module is used to obtain source data and add a queue tag and a consumption tag to the source data to form first processed data;

[0069] The data request processing module is used to obtain data requests and assign processing queues to data requests, analyze data requests to extract data sources and data processing level requirements;

[0070] a first matching module, configured to match source data using a data source and select corresponding first processed data;

[0071] A second matching module is used to determine whether the processing level value matches the data processing level requirement value;

[0072] a hierarchical processing module, configured to perform a data processing operation on the first processed data according to a preset rule and modify the processing hierarchical value accordingly;

[0073] The consumption module is used to consume the processing result data.

[0074] By using this system, the above-mentioned calculation and processing method can be executed and the corresponding technical effects can be achieved.

[0075] An embodiment of the present invention also provides a computer-readable storage medium capable of implementing all steps of the method in the above embodiment. The computer-readable storage medium stores a computer program, which implements all steps of the method in the above embodiment when executed by a processor.

[0076] An embodiment of the present invention also provides an electronic device for executing the above-mentioned method. As an implementation device of the method, the electronic device has at least a processor and a memory, and in particular, the memory stores the data and related computer programs required for executing the method, such as first processing data and data processing level requirement values, etc., and the processor calls the data and programs in the memory to execute all the steps of the implementation method and obtains the corresponding technical effects.

[0077] Preferably, the electronic device may include a bus architecture, which may include any number of interconnected buses and bridges, and the bus will include various circuits linked together by one or more processors and memories. The bus may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and, therefore, will not be described further herein. The bus interface provides an interface between the bus and the receiver and transmitter. The receiver and transmitter can be the same component, namely a transceiver, which provides a unit for communicating with various other systems over a transmission medium. The processor is responsible for managing the bus and general processing, while the memory can be used to store data used by the processor when performing operations.

[0078] Additionally, the electronic device may further include components such as a communication module, an input unit, an audio processor, a display, and a power supply. The processor (or controller, operating control) employed may include a microprocessor or other processor device and / or logic device, which receives input and controls the operation of various components of the electronic device. The memory may be one or more of a cache, flash memory, a hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices, and may store the aforementioned data and information. It may also store programs for executing the relevant information, and the processor may execute the programs stored in the memory to implement information storage or processing. The input unit is used to provide input to the processor, and may, for example, be a keypad or touch input device. The power supply is used to provide power to the electronic device. The display is used to display objects such as images and text, and may, for example, be an LCD display. The communication module is a transmitter / receiver that sends and receives signals via an antenna. The communication module (transmitter / receiver) is coupled to the processor to provide input signals and receive output signals, similar to the case of a conventional mobile communication terminal. Based on different communication technologies, multiple communication modules can be provided in the same electronic device, such as a cellular network module, a Bluetooth module, and / or a wireless local area network module. The communication module (transmitter / receiver) is also coupled to a speaker and a microphone via an audio processor to provide audio output via the speaker and receive audio input from the microphone, thereby implementing common telecommunications functions. The audio processor may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor is also coupled to a central processing unit, enabling local recording via the microphone and playback of stored audio via the speaker.

[0079] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or boxes.

[0081] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction system that is implemented in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0082] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The present invention is described in detail below. ...

[0083] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention are intended to be covered by the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A dynamic rule message interaction method based on multi-level processing, characterized in that: include: S1. Obtain source data, add a queue tag and a consumption tag to the source data to form first processed data, wherein the queue tag is added with a default value, and the consumption tag includes a processing level value and a consumption status; S2. Obtain data requests and assign processing queues to them, analyze the data requests to extract data sources and data processing level requirements; S3. Use the data source to match the source data, select the corresponding first processing data, and change the queue label of the first processing data to the processing queue; S4. Determine whether the processing level value matches the data processing level requirement value. When it is determined that the processing level value matches the data processing level requirement value, use the first processed data as the processing result data. S5. When it is determined that the processing level value does not match the data processing level requirement value, perform a data processing operation on the first processed data according to a preset rule and modify the processing level value accordingly, until it is determined that the processing level value matches the data processing level requirement value, and output the processing result data; S6. Consume the processing result data and change the consumption status of the consumed processing result data accordingly; The performing of data processing operations according to preset rules and correspondingly modifying the processing level value includes: a1. Configure a unique execution level value for each preset rule; a2. Select any preset rule and determine whether to execute the preset rule; a3. When it is determined that the preset rule is to be executed, performing corresponding processing on the first processed data according to the preset rule; a4. Modify the processing level value of the first processed data using the execution level value corresponding to the preset rule used for the execution process; a5. Repeat steps a2 to a4 until all preset rules are traversed and the modified processing level value of the first processed data matches the data processing level requirement value.

2. The method according to claim 1, wherein The step S1 further includes: Set the processing level value to 1.

3. The method according to claim 2, wherein The execution level value includes a positive odd number other than 1; The modifying the processing level value of the first processed data using the execution level value corresponding to the preset rule used for executing the processing includes adding the execution level value to the existing processing level value to obtain a modified processing level value.

4. The method according to claim 1, wherein The step S2 further includes: Get multiple data requests and assign different processing queues to the multiple data requests.

5. The method according to claim 4, wherein The method further comprises: Process multiple data requests simultaneously without interfering with each other based on different processing queues.

6. A dynamic rule message interaction system based on multi-level processing, characterized in that: include: A source data processing module is used to obtain source data and add a queue tag and a consumption tag to the source data to form first processed data; The data request processing module is used to obtain data requests and assign processing queues to data requests, analyze data requests to extract data sources and data processing level requirements; a first matching module, configured to match source data using a data source and select corresponding first processed data; A second matching module is used to determine whether the processing level value matches the data processing level requirement value; a hierarchical processing module, configured to perform a data processing operation on the first processed data according to a preset rule and modify the processing hierarchical value accordingly; The performing of data processing operations according to preset rules and correspondingly modifying the processing level value includes: a1. Configure a unique execution level value for each preset rule; a2. Select any preset rule and determine whether to execute the preset rule; a3. When it is determined that the preset rule is to be executed, performing corresponding processing on the first processed data according to the preset rule; a4. Modify the processing level value of the first processed data using the execution level value corresponding to the preset rule used for the execution process; a5. Repeat steps a2 to a4 until all preset rules are traversed and the modified processing level value of the first processed data matches the data processing level requirement value; The consumption module is used to consume the processing result data.

7. A computer-readable storage medium, characterized in that The storage medium stores a computer program, which implements the method according to any one of claims 1 to 5 when executed by a processor.

8. An electronic device, characterized in that: including processor and memory; The memory is used to store the first processing data and the data processing level requirement value; The processor is configured to execute the method according to any one of claims 1 to 5 by calling the first processing data and the data processing level requirement value.

9. A computer program product comprising a computer program and / or instructions, characterized in that When the computer program and / or instructions are executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images