Configuration parameter and service logic real-time updating method, system, device and medium

Abstract

This invention discloses a method, system, apparatus, and medium for real-time updating of configuration parameters and business logic. The method includes: sending original business data, configuration parameter update data, and business logic update data to a data processing system through a business system; verifying the original business data, configuration parameter update data, and business logic update data through the data processing system, and formatting the original business data and configuration parameter update data; updating the configuration parameters of the cluster nodes corresponding to the original business data according to the configuration parameter update data; and updating the business logic of the business processing flow corresponding to the original business data according to the business logic update data. This invention achieves real-time updating and adjustment of configuration parameters and business logic without adding external interfaces or message middleware and without interrupting business operations, improving the efficiency of updating configuration parameters and business logic, reducing costs, and can be widely applied in the field of data processing technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a configuration parameter and business logic real-time updating method, system, device and medium. BACKGROUND

[0002] At present, the number of mobile users of the three major operators, especially the number of 5G users, has reached an unprecedented huge number and is still growing. The competition between operators is becoming more and more fierce. In order to face such fierce competition, the services provided by major operators are becoming more and more refined. Compared with the previous business opening or business change, which takes at least the next month to take effect, many businesses such as traffic business can now achieve immediate effect. In the process of business operation and business operation of operators, customer abnormal use behaviors such as customer over-limit use, illegal use of a certain business need to be discovered, recorded, warned and intervened in real time. At this time, the real-time processing capability and business dynamic adaptation capability of the related application system of the operator are facing unprecedented challenges. The data processing system must have the real-time processing capability of several million or even several hundred million times of business data per second. Due to the change of customer business data and the change of business processing logic, the data processing system needs to update these changes in real time without interrupting the business to ensure the continuity and reliability of real-time business.

[0003] In the data processing system of the operator, especially in the real-time data processing business, the real-time risk monitoring and early warning and protection capability of the user is becoming more and more important, and the Ministry of Industry and Information Technology is constantly improving and improving the requirements of risk early warning. Business characteristics are real-time access to customer use of voice business, short message business or real-time bill data generated by traffic, and whether the customer is a risk customer is judged according to the business characteristics combined with the latest customer data. At present, in the data processing system, the configuration data of each item of program running is loaded when the program is started, and the business logic of program processing is fixed in the program or system when the program or system is started. If the demand changes, the business logic needs to be adjusted. Under normal circumstances, when the business demand changes involves the modification of business processing flow, it needs to go through three steps of stopping the online use of program function, modifying the program and redeploying to complete. Similarly, if the configuration parameters such as customer data of the data processing system change, the data processing program also needs to be restarted to load the latest configuration. These existing processing schemes have the defects of long processing period, low efficiency, business interruption, etc.

[0004] Of course, the software system can adopt the configuration parameters of the periodic task synchronization external data source. However, because of the period of data update and the protection problem of reading and writing, if the polling time is 5 minutes, there will be a maximum of 2 times the polling time, that is, 10 minutes of user configuration parameter synchronization. The software system can also develop and provide a real-time configuration interface for external programs to push updates in real time. However, this method requires the development and maintenance of additional interfaces, increasing the development and operation and maintenance costs and security risks of the system. Therefore, a more perfect solution is needed, which can update the configuration parameters and business logic in real time without interrupting the business, without the need to increase new data interfaces or message queues, and can realize the real-time update of the parameters and business logic. SUMMARY

[0005] The present application aims to at least partially solve one of the problems of the prior art.

[0006] To this end, one object of the embodiments of the present application is to provide a configuration parameter and business logic real-time updating method.

[0007] Another object of the embodiments of the present application is to provide a low-cost and high-efficiency configuration parameter and business logic real-time updating system.

[0008] In order to achieve the above technical purpose, the technical solutions adopted by the embodiments of the present application include:

[0009] In a first aspect, the embodiments of the present application provide a configuration parameter and business logic real-time updating method, including the following steps:

[0010] Sending the original business data, configuration parameter update data and business logic update data to the data processing system through the business system;

[0011] Verifying the original business data, configuration parameter update data and business logic update data by the data processing system, and formatting the original business data and the configuration parameter update data;

[0012] Sending the formatted original business data and configuration parameter update data to the configuration parameter management system by the data processing system, so that the configuration parameter management system updates the configuration parameters of the cluster nodes corresponding to the original business data according to the configuration parameter update data;

[0013] The data processing system sends the service logic update data and the formatted original service data to a service logic management system, so that the service logic management system updates the service processing flow corresponding to the original service data according to the service logic update data.

[0014] Further, in an embodiment of the present application, the step of sending the original service data, the configuration parameter update data and the service logic update data to the data processing system by the service system specifically comprises:

[0015] The service system collects log data of network access of the client in real time, and generates original service data according to the log data;

[0016] The service system determines the configuration parameter update data and the service logic update data, and the first field values of the configuration parameter update data, the service logic update data and the original service data are different from each other;

[0017] The service system sends the original service data, the configuration parameter update data and the service logic update data to the data processing system.

[0018] Further, in an embodiment of the present application, the step of checking the original service data, the configuration parameter update data and the service logic update data by the data processing system specifically comprises:

[0019] The data processing system judges the data type of the accepted data according to the first field value, and obtains the original service data, the configuration parameter update data and the service logic update data;

[0020] The data processing system respectively performs field number checking, key field value checking and key field type checking on the original service data, the configuration parameter update data and the service logic update data.

[0021] Further, in an embodiment of the present application, the step of formatting the original service data and the configuration parameter update data specifically comprises:

[0022] The useless fields in the original service data are removed, the remaining fields are numbered and sorted, the date fields are format-converted, and the field type descriptions are added to the fields, so as to obtain the formatted original service data;

[0023] The date field in the configuration parameter update data is converted to obtain a configuration update time, and a configuration parameter ID is generated according to a customer ID and a configuration type in the configuration parameter update data, so as to obtain the configuration parameter update data after format conversion.

[0024] Further, in an embodiment of the present application, the step of updating the configuration parameter of the cluster node corresponding to the original service data according to the configuration parameter update data specifically comprises:

[0025] determining the cluster node corresponding to the original service data according to the original service data, and updating the configuration parameter of the cluster node according to the configuration parameter update data;

[0026] When the update is successful, the update state in the configuration parameter is set to a successful state, a configuration update completion time is recorded, and then configuration update success information and the configuration update completion time are returned to the service system;

[0027] When the update fails, the update state in the configuration parameter is set to a failure state, a configuration update failure reason and a configuration update failure time are recorded, and then the configuration update failure reason and the configuration update failure time are returned to the service system.

[0028] Further, in an embodiment of the present application, the step of updating the service logic of the service processing flow corresponding to the original service data according to the service logic update data specifically comprises:

[0029] generating a service logic chain table according to the service logic update data, and determining the service processing flow corresponding to the original service data according to the original service data;

[0030] updating the service logic of the service processing flow according to the service logic chain table;

[0031] When the update is successful, the update state in the service logic is set to a successful state, a service update completion time is recorded, and then service update success information and the service update completion time are returned to the service system;

[0032] When the update fails, the update state in the service logic is set to a failure state, a service update failure reason and a service update failure time are recorded, and then the service update failure reason and the service update failure time are returned to the service system.

[0033] Further, in an embodiment of the present application, the service logic chain table comprises a service logic type, an operation type, a service logic ID and a processing step ID, and the step of updating the service logic of the service processing flow according to the service logic chain table specifically comprises:

[0034] when the operation type is addition, obtaining a first business logic corresponding to the business logic type, and inserting the first business logic into a step after the step corresponding to the processing step ID in the business processing flow;

[0035] when the operation type is modification, obtaining a first business logic corresponding to the business logic type, and replacing the step corresponding to the processing step ID in the business processing flow with the first business logic;

[0036] when the operation type is deletion, deleting the step corresponding to the business logic ID in the business processing flow.

[0037] In a second aspect, an embodiment of the present application provides a configuration parameter and business logic real-time updating system, comprising:

[0038] a data sending module configured to send original business data, configuration parameter updating data and business logic updating data to a data processing system through a business system;

[0039] a data checking module configured to check the original business data, the configuration parameter updating data and the business logic updating data through the data processing system, and perform format processing on the original business data and the configuration parameter updating data;

[0040] a configuration parameter updating module configured to send the original business data and the configuration parameter updating data after format processing to a configuration parameter management system through the data processing system, so that the configuration parameter management system performs configuration parameter updating on a cluster node corresponding to the original business data according to the configuration parameter updating data;

[0041] a business logic updating module configured to send the business logic updating data and the original business data after format processing to a business logic management system through the data processing system, so that the business logic management system performs business logic updating on a business processing flow corresponding to the original business data according to the business logic updating data.

[0042] In a third aspect, an embodiment of the present application provides a configuration parameter and business logic real-time updating device, comprising:

[0043] at least one processor;

[0044] at least one memory configured to store at least one program;

[0045] when the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned configuration parameter and business logic real-time updating method.

[0046] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, which stores a processor executable program, and the processor executable program is used for executing the configuration parameter and service logic real-time updating method when executed by a processor.

[0047] The advantages and beneficial effects of the present application will be partially given in the following description, partially will become obvious from the following description, or will be understood by the practice of the present application:

[0048] The configuration parameter and service logic of the embodiments of the present application can be configured in real time, the data interface for configuration is consistent with the original service data interface, no additional message middleware or interface needs to be added, the abnormal processing of updating the configuration parameter or service logic process is supported, the problems such as the need to restart the system or program, the increase of external data interface, the increase of storage resources and message middleware, and the need to poll the external interface to update the configuration parameter in real time can be solved, the real-time updating and adjustment of the configuration parameter and service logic are realized without increasing external interface or message middleware and without interrupting the service, the updating efficiency of the configuration parameter and service logic is improved, the cost is reduced, and the embodiments of the present application are particularly suitable for the scene of real-time processing of massive data. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following introduces the drawings needed to be used in the embodiments of the present application, and it should be understood that the drawings in the following introduction are only for the convenience of clearly describing some embodiments in the technical solutions of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the premise of the drawings.

[0050] Figure 1 A step flow chart of a configuration parameter and service logic real-time updating method provided by the embodiments of the present application;

[0051] Figure 2 A structure block diagram of a configuration parameter and service logic real-time updating system provided by the embodiments of the present application;

[0052] Figure 3 A structure block diagram of a configuration parameter and service logic real-time updating device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0053] Embodiments of the present application are described in detail below with reference to the accompanying drawings, wherein the same or similar components have the same or similar designations and functions throughout the various figures and like reference numerals denote like elements or components having the same or similar functions. The embodiments described below are exemplary only and are not intended to be limiting of the present application. For ease of description, step numbers in the following embodiments are set only for facilitating the explanation of the description, and no limitation is made on the order between the steps, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0054] In the description of the present application, the meaning of multiple is two or more, and if there is a description of first, second, it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or implicitly indicating the sequence of indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art.

[0055] The related solutions of the prior art are first introduced below.

[0056] Solution one: the patent application file with the application number "CN201110460075.4" and the invention name "a software system and its configuration updating method" discloses a software system and its configuration updating method. The method includes: setting a configuration updating maintenance component in the software system, the configuration updating maintenance component periodically judges whether the configuration parameters of the software system are updated, if there is an update, the configuration updating maintenance component acquires the configuration parameters that have been updated, and notifies the software components in the software system that use these configuration parameters that have been updated, and the software components that receive the notification acquire the configuration parameters and refresh their own configurations. The technical solution of the present application makes the updated configuration of the software real-time refreshed, without the need to restart the software system.

[0057] Solution one does not need to restart the system or program to realize real-time updating of the configuration parameters. However, solution one has the following technical defects: 1. the configuration updating component needs to periodically judge whether the configuration parameters of the software system are updated, i.e. it needs to periodically poll the relevant configuration parameters, and there is a situation of configuration asynchronization between two polling; 2. the solution is not suitable for real-time configuration updating of distributed systems or cluster systems; 3. the external system cannot acquire whether the configuration is updated successfully, and if the configuration updating fails, it will affect the normal operation of the business; 4. the solution only involves configuration parameter change processing, and cannot solve the real-time change of the business process caused by the change of the business demand.

[0058] Scheme two: the patent application file with the application number "CN202010957928.4" and the invention name "synchronization method of configuration parameters, business data system, computer system and medium" provides a synchronization method of configuration parameters, which is applied to a business data system. The business data system includes: an upstream system, a configuration management system, a message middleware and a downstream system. The synchronization method of configuration parameters includes: the configuration management system generates a configuration update message for the configuration item change in the case of configuration item data change in the upstream system; the configuration management system sends the configuration update message to the message middleware, so that the message middleware stores the configuration update message into a distributed message queue; and the downstream system obtains the configuration update message from the distributed message queue, and updates the configuration item data based on the configuration update message. The present disclosure also provides a business data system, a computer system and a medium. The synchronization method of configuration parameters and the business data system provided by the present disclosure can be applied to the timeliness and consistency update scene of the upstream and downstream systems in the financial field and other related fields.

[0059] The above-mentioned scheme two is to solve the configuration update by deploying a message middleware system, and the configuration update message needs to be stored into a distributed message queue by the message middleware. For the data processing system without a message middleware, a corresponding message middleware needs to be additionally deployed to complete the configuration message forwarding. At the same time, the above-mentioned scheme two can only be applied to the processing of configuration change, and cannot solve the real-time update problem of business logic change.

[0060] In summary, the configuration parameter or business logic update of the current software system has corresponding defects or deficiencies, mainly including the following shortcomings:

[0061] 1) The software system needs to poll the configuration parameters of the external data source regularly, and needs to synchronize the configuration parameters in full at the first time. There is a situation that the configurations are not synchronized during the two polling intervals. More importantly, the process of full synchronization of configuration parameters needs to take a long time, such as the time of the operator to synchronize the CRM data of the whole country may be several hours or several days, and during the process of full synchronization of configuration, the change data of configuration parameters will appear out of synchronization or backlog phenomenon, and during this period, the business processing will also appear errors due to the out of synchronization of configuration.

[0062] 2) The software system needs to provide a configuration parameter update interface or a message middleware. When the required configuration parameters need to be changed, the external service program or system sends the updated configuration parameters through the interface or the message middleware, and other modules of the software system synchronize the corresponding configuration from the interface or the message middleware. For the system without the required configuration parameter interface or message middleware, the architecture, configuration and security scheme of the software system need to be adjusted to supplement the corresponding interface or message middleware.

[0063] 3) Changes in the business requirements of the software system require modifications to the data processing logic and processes, and existing solutions for adjusting business logic usually lead to business interruption or partial business interruption.

[0064] To address the aforementioned shortcomings, this invention employs a data configuration and business logic update method based on existing data interfaces when processing massive amounts of real-time data. This method enables real-time updates of configuration parameters and business logic during data processing. Based on this invention, real-time changes to configuration parameters and business logic do not require restarting the system or program, adding external storage resources, storage interfaces, or message middleware, or synchronizing full or incremental configuration parameters through periodic polling of external interfaces. This approach can be applied to typical and important real-time processing scenarios, including but not limited to real-time risk monitoring for telecom operators, real-time financial anti-fraud scenarios, and real-time user profiling and tagging scenarios.

[0065] The relevant terms involved in the embodiments of the present invention will be explained below.

[0066] A business system is one or more software systems, categorized into upstream and downstream business systems based on the data flow from production to processing. The upstream business system generates raw business data and outputs this data, along with changed configuration parameters, to the data processing system. The downstream business system is the software system that receives the processed data from the data processing system.

[0067] The data processing system is responsible for receiving raw business data and configuration parameters, processing the raw business data through corresponding business processing flows, and finally outputting the processed data to downstream business systems. The data processing system can be a non-real-time data processing system or a real-time data processing system.

[0068] Configuration parameters are important data or rule-based data required for logical processing by programs or systems, such as user information table data and service activation data, which serve as the basis for judgments in some processing flows. For example, when determining whether a customer has exceeded the limit for using a certain service, it is necessary to first determine whether the customer has activated that service. Business logic update data are configuration parameters for processing links or steps in the data processing flow, such as adding a step to filter invalid data or adding a specified data comparison step to the data processing flow.

[0069] Users refer to organizations such as operators, enterprises, and government departments that deploy business systems and data processing systems.

[0070] A customer is defined as a natural person or device that uses the user's related business and generates related raw business data.

[0071] The original business data is the business data generated by customers using customer services in the user system. The original business data needs to be processed by the data processing system. Each piece of original business data has a unique ID field that can distinguish it from other original business data, such as customer number or customer ID identifier.

[0072] This invention provides a method for real-time updating of configuration parameters and business logic in a real-time or non-real-time processing system. An upstream business system sends raw business data, configuration parameter update data, and business logic update data to the data processing system through an existing raw business data interface. After receiving the data from the raw business data interface, the data processing system distinguishes between the raw business data, configuration parameter update data, and business logic update data using relevant flags. For configuration parameter update data and business logic update data, the data processing system synchronously updates the relevant configuration parameters and business processing flow, and feeds back the updated status data to the relevant upstream business system. The raw business data access interface of the data processing system can be any real-time or non-real-time data interface, including but not limited to FTP file interfaces, Kafka message interfaces, TCP / UDP SOCKET interfaces, and WEB Service interfaces.

[0073] Reference Figure 1 This invention provides a method for real-time updating of configuration parameters and business logic, specifically including the following steps:

[0074] S101. The original business data, configuration parameter update data, and business logic update data are sent to the data processing system through the business system.

[0075] Specifically, the data processing system can be a real-time data processing system or a non-real-time data processing system. If the data processing system is a real-time data processing system, the raw business data needs to be sent to the data processing system through a real-time data interface, such as a TCP / UDP socket or a Kafka message queue. If the data processing system is a non-real-time data processing system, the raw business data can be sent to the data processing system through a file interface, such as using file transfer protocols and interfaces like FTP. Step S101 specifically includes the following steps:

[0076] S1011. Collect log data of network access by clients in real time through the business system, and generate raw business data based on the log data;

[0077] S1012. The configuration parameter update data and business logic update data are determined through the business system. The first field values ​​of the configuration parameter update data, business logic update data and original business data are different from each other.

[0078] S1013. The original business data, configuration parameter update data, and business logic update data are sent to the data processing system through the business system.

[0079] Specifically, the upstream business system, such as the user's DPI data acquisition system, collects log data of customer network access in real time. This log data is the raw business data. The frequency, period, and concurrent data volume of the raw business data sent depend on the characteristics of the business system. If the concurrent raw business data volume is large, the data processing system needs to adopt a cluster deployment architecture. The distributed concurrent processing software architecture is needed to meet the real-time processing performance requirements of massive data.

[0080] The business system uses the same protocol and port to send configuration parameter update data or business logic update data as it does to send original business data. However, the interface directory, port, or TOPIC configuration can be the same or different as needed. For example, if using the Kafka protocol, the TOPIC can be different; if using the FTP protocol, the directory can be different. Configuration parameter update data refers to the parameter data required by the data processing system during processing, such as customer CRM data, customer service activation data, and province / city code dimension table data. Business logic update data represents the data processing system's processing flow. The data processing system can add, modify, or delete a processing step in the business processing flow based on the business logic update data, such as logical configuration for formatting a field in a real-time original business message or logical configuration for logical judgment and comparison of a field.

[0081] The configuration parameter update data or business logic update data can be in JSON format, "|" separated text data, or other formats. The first field value of the configuration parameter update data and the business logic update data are different preset values ​​and are different from all the original business data values. The data processing system can distinguish the type of data by the first field of the received data.

[0082] In some optional embodiments, if the data format is JOIN format, the first field value of the configuration parameter can be "Type":"FFFFFFFF", and the first field value of the business logic update data can be "Type":"EEEEEEEE". Of course, the values ​​of the first field of the configuration parameter and the business logic update data can also be defined according to the requirements of the data processing system.

[0083] In some optional embodiments, if the data format is a "|"-separated text format, the first field value of the configuration parameter can be "FFFFFFFF", and the first field value of the business logic update data can be "EEEEEEEE". Of course, the values ​​of the first field of the configuration parameter and the business logic update data can also be defined according to the requirements of the data processing system.

[0084] In addition, a fixed field in both configuration parameter update data and business logic update data is the message ID field. For example, if the message ID field is the second field, and the data format is JSON, the ID field and its value can be "MsgID":"12345678"; if the data format is "|"-separated text, then the value of the second field is "12345678". "12345678" is the specific message ID value, and the ID value will be different for each configuration parameter update data and business logic update data.

[0085] S102. The data processing system verifies the original business data, configuration parameter update data, and business logic update data, and formats the original business data and configuration parameter update data.

[0086] Specifically, if the amount of data being received is large, the data processing system can adopt a distributed software architecture and cluster processing approach to receive all the data. The data processing system records the corresponding location information or offset for each successfully received data. If the data access module of the data processing system experiences an interruption, it can continue receiving new data from the point where the previously successfully received data was received, without having to start from scratch to process redundant data or to avoid missing some data due to the interruption.

[0087] As an optional implementation, the step of verifying the original business data, configuration parameter update data, and business logic update data through a data processing system specifically includes:

[0088] S1021. The data processing system determines the data type of the received data based on the value of the first field, and obtains the original business data, configuration parameter update data and business logic update data.

[0089] S1022. The data processing system performs field quantity verification, key field value verification, and key field type verification on the original business data, configuration parameter update data, and business logic update data, respectively.

[0090] Specifically, if the original business data, configuration parameter update data, and business logic update data use the same protocol port, the same TOPIC, or the same directory, the data processing system needs to determine the data type based on the first field of the received message. For example, if the value is FFFFFFFF, the data is classified as configuration parameters; if the value is EEEEEEEE, the data is classified as business logic update data; otherwise, it is original business data. If the original business data, configuration parameters, and business logic access use different ports, different TOPICs, or different directories, the data processing system can determine the data type of the received data based on the port, TOPIC, or directory.

[0091] After determining the data type from the received real-time data, the data processing system validates the data according to the data type and discards invalid data. Invalid data includes, but is not limited to: mismatched number of fields, empty values ​​for key fields, key field values ​​exceeding the valid range, and abnormal data types for key field values.

[0092] As an optional implementation, the step of formatting the original business data and configuration parameter update data specifically includes:

[0093] S1023. Remove useless fields from the original business data, sort the remaining fields by number, convert the format of the date field, and add field type descriptions to each field to obtain the formatted original business data.

[0094] S1024. Convert the format of the date field in the configuration parameter update data to obtain the configuration update time, and generate the configuration parameter ID based on the customer ID and configuration type in the configuration parameter update data to obtain the formatted configuration parameter update data.

[0095] Specifically, the data processing system performs data formatting operations on the parsed, legitimate raw business data and configuration parameter update data. Business logic update data follows a format agreed upon by the business system and the data processing system, and therefore does not require formatting. The formatted raw business data and configuration parameter update data facilitate rapid processing by the data processing system.

[0096] Formatting operations for raw business data include, but are not limited to: numbering and sorting fields, adding field type descriptions, converting the format of date fields, and removing unnecessary fields. After completing the formatting of the raw business data, the data processing system will generate the formatted raw data structure in memory. The data structure format is shown in Table 1 below:

[0097] Number Type Value 1 long 1234565 2 string Beijing 3 double 123.13 4 double 456.89 5 long (date) 1441026486000 6 long (date) 1324324444235

[0098] Table 1

[0099] The formatting operations for configuration parameter update data include, but are not limited to: generating configuration parameter IDs based on customer IDs, configuration types, and other data in the configuration parameter update data; converting the format of date fields to generate configuration parameter timestamps. After completing the formatting of the original configuration parameters, the data processing system will generate formatted configuration parameter records in memory. The record format is shown in Table 2 below.

[0100] Configuration update time (long) 1324324444235 (converted date data) Configuration parameter operation type Add / Delete / Modify Configuration type ID $Configuration type ID Configuration parameter ID $Configuration ID value Configuration name 1 $Configuration item value Configuration name 2 $Configuration item value Configuration name 3 $Configuration item value Configuration name 1n $Configuration item value

[0101] Table 2

[0102] S103. The formatted original business data and configuration parameter update data are sent to the configuration parameter management system through the data processing system, so that the configuration parameter management system updates the configuration parameters of the cluster nodes corresponding to the original business data according to the configuration parameter update data.

[0103] Specifically, the data processing system sends the formatted configuration parameter update data to the configuration parameter management system for processing. If the data volume exceeds the processing capacity of a single server, a cluster approach is required. Different data need to be processed on different nodes in the cluster according to certain rules. Data can be grouped based on a modulo operation of customer or user accounts in the original business data. The original business data from different groups is processed on the corresponding cluster nodes, and the configuration parameters are also forwarded to the corresponding data processing nodes.

[0104] As an optional implementation, the step of updating the configuration parameters of the cluster nodes corresponding to the original business data based on the configuration parameter update data specifically includes:

[0105] A1. Determine the corresponding cluster nodes based on the original business data, and update the configuration parameters of the cluster nodes according to the configuration parameter update data;

[0106] A2. When the update is successful, set the update status in the configuration parameters to the successful status, record the configuration update completion time, and then return the configuration update success information and configuration update completion time to the business system.

[0107] A3. When the update fails, set the update status in the configuration parameters to the failed status, record the reason for the configuration update failure and the time of the configuration update failure, and then return the reason for the configuration update failure and the time of the configuration update failure to the business system.

[0108] Specifically, the data processing system updates internal configuration parameters and their status. The configuration parameter management system selects the cluster node corresponding to the original business data to update the configuration parameters; the relevant configuration parameters are stored in the memory of the processing node where the customer resides. The data storage can be shared memory or distributed memory, and the data structure can use a HashMap indexed by the customer, with the key being the customer ID and the value being the customer's configuration parameter information; after the data processing system successfully updates the configuration parameters, the update status in the updated configuration parameters is updated to a success status and the update completion time; if the data processing system fails to update the configuration parameters, such as if there is no cluster node processing the corresponding customer, the update status is set to a failure status, and the reason for the failure and the update failure time are recorded.

[0109] In addition, the data processing system sends updated configuration information to the business system, and the updated configuration parameters are the updated configuration parameters. After receiving the updated configuration information, the business system identifies which configuration parameter it is based on the configuration parameter ID, and determines whether the update was successful based on the update status of the configuration parameter. In the case of update failure, the business system can resend the updated configuration parameter data or perform other processing based on the reason for failure.

[0110] S104. The data processing system sends the updated business logic data and the formatted original business data to the business logic management system, so that the business logic management system updates the business logic of the business processing flow corresponding to the original business data according to the updated business logic data.

[0111] Specifically, the data processing system sends the business logic update data to the business logic management system for processing. The data processing system sends the formatted business logic update data and the formatted configuration parameter update data in step S103 simultaneously, and it will format and forward the data as soon as it receives the corresponding data, without any sequential dependency.

[0112] As a further optional implementation, the step of updating the business logic of the business processing flow corresponding to the original business data based on the updated business logic data specifically includes:

[0113] B1. Generate a business logic linked list by updating data according to business logic, and determine the corresponding business processing flow based on the original business data;

[0114] B2. Update the business logic of the business processing flow based on the business logic linked list;

[0115] B3. When the update is successful, set the update status in the business logic to a successful status, record the business update completion time, and then return the business update success information and business update completion time to the business system.

[0116] B4. When an update fails, set the update status in the business logic to a failed state, record the reason for the business update failure and the time of the business update failure, and then return the reason for the business update failure and the time of the business update failure to the business system.

[0117] As an optional implementation, the business logic linked list includes a business logic type, an operation type, a business logic ID, and a processing step ID. Step B2, which updates the business logic of the business processing flow based on the business logic linked list, specifically includes:

[0118] B21. When the operation type is "add", obtain the corresponding first business logic according to the business logic type, and insert the first business logic after the step in the business process corresponding to the processing step ID.

[0119] B22. When the operation type is modification, obtain the corresponding first business logic according to the business logic type, and replace the steps of the business process corresponding to the processing step ID with the first business logic.

[0120] B23. When the operation type is deletion, delete the step in the business process that corresponds to the business logic ID.

[0121] Specifically, the data processing system combines business logic update data into a business logic linked list, with the following format: business logic type, operation type, business logic ID, processing step ID, original business data field number, and logical operation formula. The operation type includes add, modify, and delete; the original business data field number can be multiple; and the processing step ID is an integer value indicating which step in the business processing flow it represents.

[0122] If the business logic type is "add", the business logic will be inserted after the step corresponding to the processing step ID; if the business logic type is "modify", the business logic will be inserted after the step with the business logic ID, and the existing business logic update data with the same business logic ID will be deleted; if the business logic type is "delete", the data processing system will delete the step with the specified business logic ID in the business processing logic and readjust the processing step IDs of all business logic update data in the linked list.

[0123] In addition, the data processing system processes data according to the business logic linked list. If all the original business data matches the business logic, it is considered to be the original data that has been matched. The system will then trim useless fields, enhance the required data fields, and finally forward the data to the required business system.

[0124] After the data processing system successfully updates the business logic, the update status in the updated business logic is set to successful and the update completion time is recorded. If the data processing system fails to update the business logic, the update status is set to failed and the reason for failure and the update failure time are recorded.

[0125] The data processing system sends updated business logic information to the business system. After receiving the updated business logic data, the business system identifies which business logic update data it is based on the business logic update data ID, and determines whether the update was successful based on the update status of the business logic. In the case of update failure, the business system can resend the business logic update data or perform other processing based on the reason for failure.

[0126] The method steps of the embodiments of the present invention have been described above. It can be understood that, through the examples of the present invention, software systems can update configuration parameters or business logic in real time during data processing to synchronize constantly changing configuration parameters and meet ever-changing business needs. An important feature of the embodiments of the present invention is that configuration parameters and business logic can be configured in real time, the data interface for configuration is consistent with the original business data interface, no additional message middleware or interface is required, and exception handling is supported for the process of updating configuration parameters or business logic.

[0127] Currently, there are three main methods for real-time configuration updates in industry data processing systems: software system or program restart; software system or program periodically polling and synchronizing configuration parameters from external data sources; and software system or program providing a real-time configuration interface, where external service programs or systems proactively write updated configuration information to the real-time configuration interface when configuration data changes. These three methods suffer from issues such as business interruption, asynchronous configuration parameters, and the need for additional data interface systems to connect to.

[0128] This invention can access raw data, configuration parameters, and business logic update data using a unified existing data access method, without requiring additional data interfaces or middleware systems, thus reducing coupling between systems. The data processing system can update configuration parameters in real time, including business configuration updates and business processing logic (i.e., business processing flow updates), without interrupting business processing. Simultaneously, the data processing system does not need to add external data source polling or real-time configuration update interfaces, thereby reducing the system's dependence on external interfaces, reducing the cost of external data source storage and interface construction, avoiding resource waste caused by storing configuration parameters in multiple systems simultaneously, and reducing the pressure on the data processing system. Furthermore, when business processing requirements change, the data processing system can update the business processing logic or business processing flow through this process scheme, thereby achieving a synchronous processing of functions and requirements. This invention is applicable to real-time data processing and non-real-time data processing, and is particularly suitable for real-time streaming data processing scenarios with massive amounts of data.

[0129] Reference Figure 2 This invention provides a system for real-time updating of configuration parameters and business logic, including:

[0130] The data sending module is used to send raw business data, configuration parameter update data, and business logic update data to the data processing system through the business system;

[0131] The data verification module is used to verify the original business data, configuration parameter update data, and business logic update data through the data processing system, and to format the original business data and configuration parameter update data.

[0132] The configuration parameter update module is used to send the formatted original business data and configuration parameter update data to the configuration parameter management system through the data processing system, so that the configuration parameter management system can update the configuration parameters of the cluster nodes corresponding to the original business data according to the configuration parameter update data.

[0133] The business logic update module is used to send business logic update data and formatted original business data to the business logic management system through the data processing system, so that the business logic management system can update the business logic of the business processing flow corresponding to the original business data according to the business logic update data.

[0134] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0135] Reference Figure 3 This invention provides a device for real-time updating of configuration parameters and business logic, comprising:

[0136] At least one processor;

[0137] At least one memory for storing at least one program;

[0138] When the above-mentioned at least one program is executed by the above-mentioned at least one processor, the above-mentioned at least one processor implements the above-mentioned method for real-time updating of configuration parameters and business logic.

[0139] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0140] This invention also provides a computer-readable storage medium storing a processor-executable program that, when executed by a processor, performs the aforementioned method for real-time updating of configuration parameters and business logic.

[0141] A computer-readable storage medium according to an embodiment of the present invention can execute a method for real-time updating of configuration parameters and business logic provided in an embodiment of the present invention, and can execute any combination of implementation steps of the method embodiment, possessing the corresponding functions and beneficial effects of the method.

[0142] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform... Figure 1 The method shown.

[0143] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the aforementioned blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0144] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the aforementioned functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0145] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0146] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0147] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the aforementioned program can be printed, because the aforementioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or, if necessary, processing in other suitable ways, and then stored in computer memory.

[0148] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0149] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0150] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0151] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for real-time updating of configuration parameters and business logic, characterized in that, Includes the following steps: The business system sends the original business data, configuration parameter update data, and business logic update data to the data processing system. The data processing system verifies the original business data, the configuration parameter update data, and the business logic update data, and formats the original business data and the configuration parameter update data. The data processing system sends the formatted original business data and the configuration parameter update data to the configuration parameter management system, so that the configuration parameter management system updates the configuration parameters of the cluster node corresponding to the original business data according to the configuration parameter update data. The data processing system sends the business logic update data and the formatted original business data to the business logic management system, so that the business logic management system updates the business logic of the business processing flow corresponding to the original business data according to the business logic update data. The interface through which the business system sends the configuration parameter update data and the business logic update data is the same as the interface through which the original business data is sent.

2. The method for real-time updating of configuration parameters and business logic according to claim 1, characterized in that, The step of sending the original business data, configuration parameter update data, and business logic update data from the business system to the data processing system specifically includes: The business system collects log data of network access by clients in real time and generates raw business data based on the log data. The configuration parameter update data and the business logic update data are determined by the business system, and the first field values ​​of the configuration parameter update data, the business logic update data, and the original business data are all different. The original business data, the configuration parameter update data, and the business logic update data are sent to the data processing system through the business system.

3. The method for real-time updating of configuration parameters and business logic according to claim 2, characterized in that, The step of verifying the original business data, the configuration parameter update data, and the business logic update data through the data processing system specifically includes: The data processing system determines the data type of the received data based on the value of the first field, and obtains the original business data, the configuration parameter update data, and the business logic update data. The data processing system performs field quantity verification, key field value verification, and key field type verification on the original business data, the configuration parameter update data, and the business logic update data, respectively.

4. The method for real-time updating of configuration parameters and business logic according to claim 1, characterized in that, The step of formatting the original business data and the configuration parameter update data specifically includes: Remove useless fields from the original business data, sort the remaining fields by number, convert the format of the date field, and add field type descriptions to each field to obtain the formatted original business data. The date field in the configuration parameter update data is formatted to obtain the configuration update time, and a configuration parameter ID is generated based on the customer ID and configuration type in the configuration parameter update data to obtain the formatted configuration parameter update data.

5. The method for real-time updating of configuration parameters and business logic according to claim 1, characterized in that, The step of updating the configuration parameters of the cluster node corresponding to the original business data according to the configuration parameter update data specifically includes: The corresponding cluster node is determined based on the original business data, and the configuration parameters of the cluster node are updated based on the configuration parameter update data. When the update is successful, the update status in the configuration parameters is set to a successful status, and the configuration update completion time is recorded. Then, the configuration update success information and the configuration update completion time are returned to the business system. When an update fails, the update status in the configuration parameters is set to a failed status, and the reason for the configuration update failure and the time of the configuration update failure are recorded. Then, the reason for the configuration update failure and the time of the configuration update failure are returned to the business system.

6. A method for real-time updating of configuration parameters and business logic according to any one of claims 1 to 5, characterized in that, The step of updating the business logic of the business processing flow corresponding to the original business data based on the updated business logic data specifically includes: A business logic linked list is generated based on the updated data according to the business logic, and the corresponding business processing flow is determined based on the original business data; The business logic is updated based on the business logic linked list; When the update is successful, the update status in the business logic is set to a successful status, and the business update completion time is recorded. Then, the business update success information and the business update completion time are returned to the business system. When an update fails, the update status in the business logic is set to a failed status, and the reason for the business update failure and the business update failure time are recorded. Then, the reason for the business update failure and the business update failure time are returned to the business system.

7. The method for real-time updating of configuration parameters and business logic according to claim 6, characterized in that, The business logic linked list includes business logic type, operation type, business logic ID, and processing step ID. The step of updating the business logic of the business processing flow according to the business logic linked list specifically includes: When the operation type is "add", the corresponding first business logic is obtained according to the business logic type, and the first business logic is inserted after the step of the business processing flow corresponding to the processing step ID; When the operation type is modification, the corresponding first business logic is obtained according to the business logic type, and the first business logic is used to replace the step of the business process corresponding to the processing step ID. When the operation type is deletion, delete the step in the business processing flow that corresponds to the business logic ID.

8. A system for real-time updating of configuration parameters and business logic, characterized in that, include: The data sending module is used to send raw business data, configuration parameter update data, and business logic update data to the data processing system through the business system; The data verification module is used to verify the original business data, the configuration parameter update data, and the business logic update data through the data processing system, and to format the original business data and the configuration parameter update data. The configuration parameter update module is used to send the formatted original business data and the configuration parameter update data to the configuration parameter management system through the data processing system, so that the configuration parameter management system updates the configuration parameters of the cluster node corresponding to the original business data according to the configuration parameter update data. The business logic update module is used to send the business logic update data and the formatted original business data to the business logic management system through the data processing system, so that the business logic management system updates the business logic of the business processing flow corresponding to the original business data according to the business logic update data. The interface through which the business system sends the configuration parameter update data and the business logic update data is the same as the interface through which the original business data is sent.

9. A device for real-time updating of configuration parameters and business logic, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a method for real-time updating of configuration parameters and business logic as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform a real-time update method for configuration parameters and business logic as described in any one of claims 1 to 7.

Images