Financial data processing method and device

Abstract

The embodiment of the present application provides a financial data processing method and device, which can be used in the field of big data technology. The method includes: judging whether the target real-time event corresponding to the current financial business request involves non-real-time data based on a preset rule engine, and if so, retrieving the target historical transaction data corresponding to the business parameters of the target real-time event, and storing the target historical transaction data in the REDIS cluster in the batch processing system; applying the stream processing system to obtain the target real-time data corresponding to the target real-time event, and obtaining the target historical transaction data corresponding to the business parameters from the REDIS cluster; performing data association on the target real-time data and the target historical transaction data, and obtaining the processing result data corresponding to the target real-time event based on the associated target real-time data and the target historical transaction data. The present application can effectively realize the batch-stream fusion between the batch processing system and the stream processing system under the large architecture of financial data processing.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, in particular to the field of big data technology, and specifically to a financial data processing method and device. Background Art

[0002] Commercial banks usually use streaming data processing frameworks for real-time online transaction data processing. The streaming data framework currently consists of two frameworks: batch processing and stream processing, which are used independently. Batch processing mainly uses the SPARK framework to process non-real-time data jobs, and stream processing uses FLINK to process quasi-real-time data jobs with high timeliness.

[0003] At present, since the two frameworks of batch processing and stream processing are independent of each other, two different cluster environments need to be maintained during the development and operation process. In addition, different frameworks have different development specifications and receive different types of data, resulting in low efficiency in data cleaning and conversion. At the same time, the existing independently running frameworks cannot obtain each other's data in a timely manner, which makes it impossible to meet the needs of batch and stream fusion, such as the need to refer to previous non-real-time data when performing real-time data analysis in stream processing, and the need to add real-time data when performing non-quasi-real-time data analysis in batch processing.

[0004] In other words, the existing financial data processing methods have problems such as low financial data processing efficiency, poor reliability, and inability to meet scenarios where batch-stream fusion is needed. Summary of the invention

[0005] In response to the problems in the prior art, the present application provides a financial data processing method and device, which can effectively realize the batch-stream fusion between the batch processing system and the stream processing system under the large-scale financial data processing architecture, thereby effectively improving the processing efficiency and reliability of the financial data processing process where there is a need for batch-stream fusion, and effectively meeting the financial business processing scenarios where there is a need for batch-stream fusion.

[0006] In order to solve the above technical problems, this application provides the following technical solutions:

[0007] In a first aspect, the present application provides a financial data processing method, comprising:

[0008] Based on the preset rule engine, determine whether the target real-time event corresponding to the current financial business request involves non-real-time data. If so, retrieve the target historical transaction data corresponding to the business parameters of the target real-time event, and store the target historical transaction data in the REDIS cluster in the batch processing system;

[0009] The application stream processing system obtains the target real-time data corresponding to the target real-time event, and obtains the target historical transaction data corresponding to the business parameter from the REDIS cluster;

[0010] The target real-time data and the target historical transaction data are data-associated, and processing result data corresponding to the target real-time event is obtained based on the associated target real-time data and target historical transaction data.

[0011] Furthermore, before the preset rule engine determines whether the target real-time event corresponding to the current financial service request involves non-real-time data, it also includes:

[0012] Receive financial service requests;

[0013] Determine the target processing type of the transaction code corresponding to the financial service request according to a preset transaction code configuration rule;

[0014] If the target processing type is the preset first or second category, the to-be-processed event corresponding to the financial service request is confirmed as a target real-time event, wherein the first category is a real-time processing type and the second category is a combination of a real-time processing type and a non-real-time processing type.

[0015] Furthermore, it also includes:

[0016] If the target processing type is the preset third type, confirming the pending event corresponding to the financial service request as a target non-real-time event, wherein the third type is a non-real-time processing type;

[0017] The business parameters of the target non-real-time event are sent to a batch processing system, so that the batch processing system retrieves data corresponding to the business parameters of the target non-real-time event within a preset processing time range, and obtains processing result data corresponding to the target non-real-time event.

[0018] Furthermore, it also includes:

[0019] If it is determined that the target real-time event corresponding to the current financial business request does not involve non-real-time data, the business parameters of the target real-time event are sent to the stream processing system so that the stream processing system can retrieve the data corresponding to the business parameters of the target real-time event in real time and obtain the processing result data corresponding to the target real-time event.

[0020] Further, the step of retrieving target historical transaction data corresponding to the business parameter of the target real-time event and storing the target historical transaction data in a REDIS cluster in a batch processing system includes:

[0021] Sending the business parameters of the target real-time event to a preset batch data lake;

[0022] Retrieving corresponding target historical transaction data from a preset distributed file system according to the business parameters in the batch data lake;

[0023] The target historical transaction data is stored in the form of key-value pairs in the REDIS cluster in the batch processing system.

[0024] Furthermore, before sending the business parameters of the target real-time event to the preset batch processing data lake, it also includes:

[0025] Sending the financial service request to a real-time rule engine corresponding to a stream processing system and a non-real-time batch rule engine corresponding to a batch processing system respectively;

[0026] Correspondingly, the data association of the target real-time data and the target historical transaction data includes:

[0027] According to the rules corresponding to the non-real-time batch rule engine and the rules corresponding to the real-time rule engine, data association is performed between the target historical transaction data corresponding to the non-real-time batch rule engine and the target real-time data corresponding to the real-time rule engine.

[0028] Furthermore, the application stream processing system obtains target real-time data corresponding to the target real-time event, including:

[0029] The preset KAFKA distributed publish-subscribe system is applied to write the target real-time data corresponding to the target real-time event into the stream processing system.

[0030] In a second aspect, the present application provides a financial data processing device, comprising:

[0031] The data judgment module is used to judge whether the target real-time event corresponding to the current financial business request involves non-real-time data based on a preset rule engine. If so, the target historical transaction data corresponding to the business parameters of the target real-time event is retrieved and stored in the REDIS cluster in the batch processing system.

[0032] A data acquisition module, used for applying a stream processing system to acquire target real-time data corresponding to the target real-time event, and acquiring target historical transaction data corresponding to the business parameters from the REDIS cluster;

[0033] The data association module is used to perform data association on the target real-time data and the target historical transaction data, and obtain the processing result data corresponding to the target real-time event based on the associated target real-time data and the target historical transaction data.

[0034] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the financial data processing method when executing the program.

[0035] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the financial data processing method when executed by a processor.

[0036] It can be seen from the above technical scheme that the present application provides a financial data processing method and device, the method comprising: judging whether the target real-time event corresponding to the current financial business request involves non-real-time data based on a preset rule engine, and if so, retrieving the target historical transaction data corresponding to the business parameters of the target real-time event, and storing the target historical transaction data in the REDIS cluster in the batch processing system; applying the stream processing system to obtain the target real-time data corresponding to the target real-time event, and obtaining the target historical transaction data corresponding to the business parameters from the REDIS cluster; performing data association on the target real-time data and the target historical transaction data, and obtaining the processing result data corresponding to the target real-time event based on the associated target real-time data and the target historical transaction data, by storing the target historical transaction data in the REDIS cluster in the batch processing system, applying the stream processing system to obtain the target historical transaction data corresponding to the business parameters from the REDIS cluster and performing data association on the target real-time data and the target historical transaction data, which can effectively realize financial The batch-stream fusion between the batch processing system and the stream processing system under the large-scale financial data processing architecture can effectively improve the processing efficiency and intelligence of financial business requests with batch-stream fusion requirements, and can effectively improve the reliability and effectiveness of the financial data processing process, and effectively meet the financial business processing scenarios with batch-stream fusion requirements; at the same time, by adopting the REDIS cluster, it is possible to effectively realize the combination of the batch processing system and the stream processing system without changing the respective architectural modes of the batch processing system and the stream processing system, overcoming the shortcomings of the existing technology that batch processing and stream processing require the use of two sets of frameworks, and at the same time making up for the shortcomings of the processing of financial business requests that some businesses need to associate historical data and real-time data, breaking the barriers of the two traditional batch and real-time data frameworks, saving resources while also saving development and maintenance costs. At the same time, compared with the previous calculation of only real-time data, the consideration of historical data can analyze and calculate financial business requests in more dimensions, giving financial customers a better experience, and providing financial institutions or merchants with more intuitive and comprehensive customer behavior. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 It is a schematic diagram of the interaction between the financial data processing apparatus and the client device in an embodiment of the present application.

[0039] Figure 2 It is a first flow chart of the financial data processing method in the embodiment of the present application.

[0040] Figure 3 It is a second flow chart of the financial data processing method in the embodiment of the present application.

[0041] Figure 4 This is a third flow chart of the financial data processing method in the embodiment of the present application.

[0042] Figure 5 This is a fourth flow chart of the financial data processing method in the embodiment of the present application.

[0043] Figure 6 This is the fifth flow chart of the financial data processing method in the embodiment of the present application.

[0044] Figure 7 This is the sixth flow chart of the financial data processing method in the embodiment of the present application.

[0045] Figure 8 This is the seventh flow chart of the financial data processing method in the embodiment of the present application.

[0046] Fig. 9 It is a structural diagram of the financial data processing device in an embodiment of the present application.

[0047] Fig.10 It is a logical function diagram of the financial data processing method of the application example of this application.

[0048] Fig.11 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] It should be noted that the financial data processing method and device disclosed in the present application can be used in the field of big data technology, and can also be used in any field other than the field of big data technology. The application field of the financial data processing method and device disclosed in the present application is not limited.

[0051] Since batch processing and stream processing are two independent frameworks, two different cluster environments need to be maintained during development and operation, which results in high maintenance costs. Different frameworks have different development specifications and receive different types of data, resulting in high data cleaning and conversion costs. At the same time, the existing independently running frameworks cannot obtain each other's data in a timely manner, which makes it impossible to meet the needs of batch-stream fusion, such as when stream processing performs real-time data analysis and needs to refer to previous non-real-time data, or when batch processing performs non-quasi-real-time data analysis and needs to add real-time data. In other words, the existing financial data processing methods have problems such as low financial data processing efficiency, poor reliability, and inability to meet the needs of batch-stream fusion. Based on this, an embodiment of the present application provides a financial data processing method, which determines whether the target real-time event corresponding to the current financial business request involves non-real-time data based on a preset rule engine. If so, the target historical transaction data corresponding to the business parameters of the target real-time event is retrieved, and the target historical transaction data is stored in the REDIS cluster in the batch processing system; the stream processing system is applied to obtain the target real-time data corresponding to the target real-time event, and the target historical transaction data corresponding to the business parameters are obtained from the REDIS cluster; the target real-time data and the target historical transaction data are data-associated, and the processing result data corresponding to the target real-time event is obtained based on the associated target real-time data and the target historical transaction data. By storing the target historical transaction data in the REDIS cluster in the batch processing system, the stream processing system is applied to obtain the target historical transaction data corresponding to the business parameters from the REDIS cluster and the target real-time data and the target historical transaction data are data-associated, which can effectively realize the large-scale financial data processing. The batch-stream fusion between the batch processing system and the stream processing system under the architecture can effectively improve the processing efficiency and intelligence of financial business requests with batch-stream fusion requirements, and can effectively improve the reliability and effectiveness of the financial data processing process, and effectively meet the financial business processing scenarios with batch-stream fusion requirements; at the same time, by adopting the REDIS cluster, it is possible to effectively realize the combination of the batch processing system and the stream processing system without changing the respective architectural modes of the batch processing system and the stream processing system, overcoming the deficiency of the existing technology that batch processing and stream processing require the use of two sets of frameworks, and at the same time making up for the deficiency of financial business request processing that some businesses need to associate historical data and real-time data, breaking the barriers of the traditional batch and real-time data frameworks, saving resources while also saving development and maintenance costs. At the same time, compared with the previous calculation of only real-time data, the consideration of historical data is added, and financial business requests can be analyzed and calculated in more dimensions, giving financial customers a better experience, and providing financial institutions or merchants with more intuitive and comprehensive customer behavior.

[0052] Based on the above content, the present application also provides a financial data processing device for implementing the financial data processing method provided in one or more embodiments of the present application, see Figure 1 The financial data processing device can establish a communication connection with the client device, and the financial data processing device can receive the financial business request sent by the client device. The financial data processing device then calls the batch processing system, stream processing system, distributed file system, etc. preset in the financial institution to perform the financial data processing process for the financial business request, and after obtaining the processing result data of the financial business request, the processing result data is sent to the client device of the financial institution for storage or display, etc.

[0053] In an actual application scenario, the financial data processing device can be implemented using a server; the server can be communicatively connected to at least one client device.

[0054] It is understandable that the client device may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0055] In another practical application scenario, the part of the financial data processing device that performs financial data processing can be executed in the server as described above, or all operations can be completed in the client device. The specific selection can be made based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device can also include a processor for specific processing of financial data processing.

[0056] In a third practical application scenario, the aforementioned financial data processing device may also be a functional module in an architecture such as a stream platform or a stream processing system of a financial institution, etc.

[0057] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0058] The server and the client device may communicate with each other using any suitable network protocol, including network protocols that have not yet been developed on the date of filing this application. The network protocols may include, for example, TCP / IP, UDP / IP, HTTP, HTTPS, etc. Of course, the network protocols may also include, for example, RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer) protocols used on top of the above protocols.

[0059] In one or more embodiments of the present application, non-real-time events refer to events that do not require real-time processing by a stream processing system, which may include quasi-real-time events. The processing time rules of quasi-real-time events are manually set in advance based on a batch processing system, such as batch processing every 10 minutes or every 24 hours, etc., which can be set specifically according to actual application scenarios.

[0060] The details are described in detail through the following embodiments and application examples.

[0061] In order to solve the problems of low efficiency and poor reliability of financial data processing and inability to meet the needs of batch and stream fusion caused by the independence of batch processing and stream processing in the existing financial data processing methods, this application provides an embodiment of a financial data processing method, which is suitable for batch processing scenarios where streaming platform big data uses a streaming data processing framework to process online transaction data and non-real-time batch data in real time, such as commercial banks sending real-time SMS reminders of account balances, real-time interception of blacklisted transactions, and other transaction scenarios. Figure 2 The financial data processing method executed by the financial data processing device specifically includes the following contents:

[0062] Step 110: Based on a preset rule engine, determine whether the target real-time event corresponding to the current financial service request involves non-real-time data. If so, execute step 120.

[0063] Step 120: Retrieve target historical transaction data corresponding to the business parameters of the target real-time event, and store the target historical transaction data in a REDIS cluster in a batch processing system.

[0064] In step 110 and step 120, the Drools rule engine (open source rule engine) can be introduced as the rule engine to perform engine rule routing for the transaction code corresponding to the target real-time event. When a bank transaction data enters the stream platform, the Drools engine first determines whether it is necessary to read the historical transaction data based on the transaction code carried in the bank transaction data. If it is not necessary to read the historical data and only perform real-time calculations, the stream processing system is directly applied to obtain the target real-time data corresponding to the target real-time event, and the processing result data corresponding to the target real-time event is obtained based on the target real-time data. If the Drools engine determines that it is necessary to read the historical data on the basis of reading the real-time data, the data is routed to the real-time rule engine and the non-real-time batch rule engine at the same time.

[0065] Step 200: The application stream processing system obtains the target real-time data corresponding to the target real-time event, and obtains the target historical transaction data corresponding to the business parameters from the REDIS cluster.

[0066] In step 200, the real-time data that needs to read historical data is routed to the non-real-time batch rule engine. When the data enters the open source stream processing framework FLINK computing framework in real time, the parameters marking the requested data features, such as transaction code, transaction number, application name, date, etc., are passed to the batch processing data lake HIVE in the batch processing framework. The reading thread obtains the corresponding table data from the Hadoop distributed file system HDFS. The reading thread puts each row of data into the reading queue in turn. The storage thread reads the data from the corresponding reading queue and updates the remote dictionary service REDIS (RemoteDictionary After storing the data to be retrieved (such as the event number, card number, customer number, etc. that are the same as the transaction within a certain period of time) in the REDIS cluster, the open source stream processing framework FLINK calls its own REDIS class to read the data in the REDIS cluster. By adopting a key-value pair storage database cluster such as the REDIS cluster, the stream processing system can quickly and accurately access the REDIS cluster to quickly obtain historical data in the batch processing system. The stream processing system does not need to access the data storage system or database originally corresponding to the batch processing system. Therefore, there is no need to use complex execution methods to achieve compatibility between the stream processing system and the batch processing system, and the stream processing system can also quickly obtain historical data. At the same time, the security of the internal data of the batch processing system is also guaranteed.

[0067] Step 300: data association is performed on the target real-time data and the target historical transaction data, and processing result data corresponding to the target real-time event is obtained based on the associated target real-time data and target historical transaction data.

[0068] In step 300, the data that was previously directly entered into FLINK can be associated with the rules in the non-real-time rule engine, such as the data with the same event number, card number, and customer number can be associated and calculated to meet business needs.

[0069] From the above description, it can be seen that the financial data processing method provided in the embodiment of the present application, by storing the target historical transaction data in the REDIS cluster in the batch processing system, applying the stream processing system to obtain the target historical transaction data corresponding to the business parameters from the REDIS cluster and perform data association between the target real-time data and the target historical transaction data, can effectively realize the batch-stream fusion between the batch processing system and the stream processing system under the large-scale financial data processing architecture, thereby effectively improving the processing efficiency and intelligence of financial business requests with batch-stream fusion requirements, and can effectively improve the reliability and effectiveness of the financial data processing process, and effectively meet the financial business processing scenarios with batch-stream fusion requirements; at the same time, by adopting Using REDIS cluster, we can effectively realize the combination of batch processing system and stream processing system without changing their respective architectural modes, overcome the deficiency of batch processing and stream processing in the prior art that two sets of frameworks are needed, and make up for the deficiency of financial business request processing that some businesses need to associate historical data and real-time data, break the barriers of traditional batch and real-time data frameworks, save resources and development and maintenance costs, and add historical data into the consideration compared with the previous calculation of real-time data, so as to analyze and calculate financial business requests in more dimensions, give financial customers a better experience, and provide financial institutions or merchants with more intuitive and comprehensive customer behavior.

[0070] In order to improve the accuracy and efficiency of confirming the pending event corresponding to the financial service request as the target real-time event, in an embodiment of the financial data processing method provided in the present application, see Figure 3 , the financial data processing method further specifically includes the following contents before step 100:

[0071] Step 010: Receive a financial service request.

[0072] Step 020: Determine the target processing type of the transaction code corresponding to the financial service request according to the preset transaction code configuration rule.

[0073] Step 030: If the target processing type is the preset first or second category, the pending event corresponding to the financial service request is confirmed as a target real-time event, wherein the first category is a real-time processing type and the second category is a combination of a real-time processing type and a non-real-time processing type.

[0074] For example, sending personal balance change reminder SMS is configured as real-time event rules. Customer marketing makes targeted real-time recommendations and retention based on the customer's transaction behavior over a period of time at the moment the customer makes a transaction. This is configured as real-time event rules + quasi-real-time event rules. The data of the same event number, card number, customer code, etc. within a period of time is calculated together with the real-time transaction.

[0075] From the above description, it can be seen that the financial data processing method provided in the embodiment of the present application can effectively improve the accuracy and efficiency of confirming the pending events corresponding to the financial business request as target real-time events by determining the target processing type of the transaction code corresponding to the financial business request according to the preset transaction code configuration rules, and thus can provide an accurate data basis for the subsequent financial data processing process, so as to further improve the efficiency of the financial data processing process and the reliability of the results.

[0076] In order to improve the reliability and accuracy of non-real-time event processing, an embodiment of the financial data processing method provided in this application is provided. Figure 4 The financial data processing method further specifically includes the following contents after step 020:

[0077] Step 400: If the target processing type is the preset third type, the to-be-processed event corresponding to the financial service request is confirmed as a target non-real-time event, wherein the third type is a non-real-time processing type.

[0078] Step 500: Send the business parameters of the target non-real-time event to a batch processing system, so that the batch processing system retrieves data corresponding to the business parameters of the target non-real-time event within a preset processing time range and obtains processing result data corresponding to the target non-real-time event.

[0079] For example, if the transaction data made by the customer at 9 am needs to be distributed to other applications for loading in the early morning of the next day, it is configured as a quasi-real-time event rule.

[0080] From the above description, it can be seen that the financial data processing method provided in the embodiment of the present application, by sending the business parameters of the target non-real-time event to the batch processing system, so that the batch processing system retrieves the data corresponding to the business parameters of the target non-real-time event within a preset processing time range, and obtains the processing result data corresponding to the target non-real-time event, can effectively improve the reliability and accuracy of non-real-time event processing, and thus can effectively improve the applicability, comprehensiveness and extensiveness of the financial data processing process, so as to further improve the intelligence level of the financial data processing process.

[0081] In order to improve the efficiency, reliability and accuracy of real-time event processing, an embodiment of the financial data processing method provided in this application is provided. Figure 5 The financial data processing method further specifically includes the following contents after step 110:

[0082] Step 130: If it is determined that the target real-time event corresponding to the current financial business request does not involve non-real-time data, the business parameters of the target real-time event are sent to the stream processing system, so that the stream processing system can retrieve the data corresponding to the business parameters of the target real-time event in real time and obtain the processing result data corresponding to the target real-time event.

[0083] From the above description, it can be seen that the financial data processing method provided in the embodiment of the present application, by sending the business parameters of the target real-time event to the stream processing system, so that the stream processing system can retrieve the data corresponding to the business parameters of the target real-time event in real time, and obtain the processing result data corresponding to the target real-time event, can effectively improve the efficiency, reliability and accuracy of real-time event processing, and thus can effectively improve the applicability, comprehensiveness and extensiveness of the financial data processing process, so as to further improve the intelligence level of the financial data processing process.

[0084] In order to improve the reliability and efficiency of the processing process of retrieving the target historical transaction data corresponding to the business parameters of the target real-time event and storing the target historical transaction data in the REDIS cluster in the batch processing system, an embodiment of the financial data processing method provided in this application is provided. Figure 6 , step 120 in the financial data processing method further specifically includes the following contents:

[0085] Step 121: Send the business parameters of the target real-time event to a preset batch data lake;

[0086] Step 122: Retrieve corresponding target historical transaction data from a preset distributed file system according to the business parameters in the batch data lake.

[0087] Step 123: Store the target historical transaction data in the form of key-value pairs in the REDIS cluster in the batch processing system.

[0088] From the above description, it can be seen that the financial data processing method provided in the embodiment of the present application, through the use of a batch data lake, a distributed file system and a REDIS cluster, can improve the reliability and efficiency of the processing process of retrieving the target historical transaction data corresponding to the business parameters of the target real-time event and storing the target historical transaction data in the REDIS cluster in the batch processing system, thereby further improving the processing efficiency and reliability of the financial data processing process with batch-stream fusion requirements.

[0089] In order to improve the reliability and efficiency of data association between the target real-time data and the target historical transaction data, in one embodiment of the financial data processing method provided in the present application, see Figure 7 , the financial data processing method further specifically includes the following contents before step 121:

[0090] Step 124: Send the financial service request to the real-time rule engine corresponding to the stream processing system and the non-real-time batch rule engine corresponding to the batch processing system respectively.

[0091] Correspondingly, step 300 in the financial data processing method further specifically includes the following contents:

[0092] Step 310: According to the rules corresponding to the non-real-time batch rule engine and the rules corresponding to the real-time rule engine, data association is performed between the target historical transaction data corresponding to the non-real-time batch rule engine and the target real-time data corresponding to the real-time rule engine.

[0093] In step 310, the specific method of data association between the target historical transaction data corresponding to the non-real-time batch rule engine and the target real-time data corresponding to the real-time rule engine can be to perform FLINK operator calculation on the target historical transaction data and the target real-time data using a preset FLINK operator calculation method.

[0094] Step 320: Obtain processing result data corresponding to the target real-time event based on the associated target real-time data and target historical transaction data.

[0095] From the above description, it can be seen that the financial data processing method provided in the embodiment of the present application can effectively improve the reliability and efficiency of data association between the target real-time data and the target historical transaction data by data association between the target historical transaction data corresponding to the non-real-time batch rule engine and the target real-time data corresponding to the real-time rule engine according to the rules corresponding to the non-real-time batch rule engine and the rules corresponding to the real-time rule engine, thereby further improving the processing efficiency and reliability of the financial data processing process with batch-stream fusion requirements.

[0096] In order to improve the efficiency and reliability of the stream processing system in acquiring the target real-time data corresponding to the target real-time event, an embodiment of the financial data processing method provided in this application is provided. Figure 8 , step 200 in the financial data processing method specifically includes the following contents:

[0097] Step 210: Apply a preset KAFKA distributed publish-subscribe system to write the target real-time data corresponding to the target real-time event into the stream processing system.

[0098] Step 220: Obtain target historical transaction data corresponding to the business parameters from the REDIS cluster.

[0099] From the above description, it can be seen that the financial data processing method provided in the embodiment of the present application, by applying a preset KAFKA distributed publish-subscribe system to write the target real-time data corresponding to the target real-time event into the stream processing system, can effectively improve the efficiency and reliability of the stream processing system in acquiring the target real-time data corresponding to the target real-time event, and thus can further improve the processing efficiency and reliability of the financial data processing process with batch-stream fusion requirements.

[0100] From the software level, in order to solve the problems of low efficiency and poor reliability of financial data processing and inability to meet the needs of batch and stream fusion caused by the independence of the two frameworks of batch processing and stream processing in the existing financial data processing method, the present application provides an embodiment of a financial data processing device for executing all or part of the contents of the financial data processing method, see Fig. 9 , the financial data processing device specifically includes the following contents:

[0101] The data judgment module 10 is used to judge whether the target real-time event corresponding to the current financial business request involves non-real-time data based on a preset rule engine. If so, the target historical transaction data corresponding to the business parameters of the target real-time event is retrieved and the target historical transaction data is stored in the REDIS cluster in the batch processing system.

[0102] In the data judgment module 10, the Drools rule engine (open source rule engine) can be introduced as the rule engine to perform engine rule routing for the transaction code corresponding to the target real-time event. When a bank transaction data enters the stream platform, the Drools engine first determines whether it is necessary to read the historical transaction data based on the transaction code carried in the bank transaction data. If it is not necessary to read the historical data and only perform real-time calculations, the stream processing system is directly applied to obtain the target real-time data corresponding to the target real-time event, and the processing result data corresponding to the target real-time event is obtained based on the target real-time data. If the Drools engine determines that it is necessary to read the historical data on the basis of reading the real-time data, the data is routed to the real-time rule engine and the non-real-time batch rule engine at the same time.

[0103] The data acquisition module 20 is used to apply a stream processing system to acquire target real-time data corresponding to the target real-time event, and to acquire target historical transaction data corresponding to the business parameters from the REDIS cluster.

[0104] In the data acquisition module 20, the real-time data that needs to read historical data is routed to the non-real-time batch rule engine. When the data enters the open source stream processing framework FLINK computing framework in real time, the parameters marking the requested data features, such as transaction code, transaction number, application name, date, etc., are transmitted to the batch data lake HIVE in the batch processing framework. The reading thread obtains the corresponding table data from the Hadoop distributed file system HDFS, and the reading thread puts each row of data into the reading queue in turn. The storage thread reads data from the corresponding reading queue, and the storage thread updates the remote dictionary service REDIS (Remote Dictionary Server) cluster. After storing the data to be obtained (such as the event number, card number, customer code, etc. that are the same as the transaction within a certain period of time) in the REDIS cluster, the open source stream processing framework FLINK calls its own REDIS class to read the data in the REDIS cluster.

[0105] The data association module 30 is used to perform data association on the target real-time data and the target historical transaction data, and obtain the processing result data corresponding to the target real-time event based on the associated target real-time data and target historical transaction data.

[0106] In the data association module 30, the data that has been directly input into FLINK can be associated according to the rules in the non-real-time rule engine, such as the data with the same event number, card number, and customer number can be associated and calculated to meet business needs.

[0107] The embodiments of the financial data processing device provided in the present application can be specifically used to execute the processing flow of the embodiments of the financial data processing method in the above embodiments. Its functions will not be described in detail here, and reference can be made to the detailed description of the above method embodiments.

[0108] From the above description, it can be seen that the financial data processing device provided in the embodiment of the present application can effectively realize the batch-stream fusion between the batch processing system and the stream processing system under the large-scale financial data processing architecture by storing the target historical transaction data in the REDIS cluster in the batch processing system, applying the stream processing system to obtain the target historical transaction data corresponding to the business parameters from the REDIS cluster and perform data association between the target real-time data and the target historical transaction data, thereby effectively improving the processing efficiency and intelligence of financial business requests with batch-stream fusion requirements, and effectively improving the reliability and effectiveness of the financial data processing process, and effectively meeting the financial business processing scenarios with batch-stream fusion requirements; at the same time, by adopting Using REDIS cluster, we can effectively realize the combination of batch processing system and stream processing system without changing their respective architectural modes, overcome the deficiency of batch processing and stream processing in the prior art that two sets of frameworks are needed, and make up for the deficiency of financial business request processing that some businesses need to associate historical data and real-time data, break the barriers of traditional batch and real-time data frameworks, save resources and development and maintenance costs, and add historical data into the consideration compared with the previous calculation of real-time data, so as to analyze and calculate financial business requests in more dimensions, give financial customers a better experience, and provide financial institutions or merchants with more intuitive and comprehensive customer behavior.

[0109] In order to further illustrate the present solution, the present application also provides a specific application example of a financial data processing method. The application example of the present application is suitable for batch processing scenarios in which streaming platform big data uses a streaming data processing framework to process online transaction data and non-real-time batch data in real time, such as commercial banks sending real-time SMS reminders of account balances, real-time interception of blacklist transactions, and other transaction scenarios.

[0110] The application example of this application overcomes the deficiency of the prior art that batch processing and stream processing require the use of two sets of frameworks, and at the same time makes up for the deficiency that some businesses need to comprehensively consider historical data and real-time data requirements, and provides a batch-stream integrated financial data processing method based on the FLINK open source stream processing framework and the REDIS memory key-value database cluster.

[0111] See also Fig.10 The financial data processing method mentioned in the application example of this application specifically includes the following contents:

[0112] Step 1: The business configures rules according to the transaction code at the front desk (business front desk rule configuration);

[0113] For example, to send a personal balance change reminder SMS, configure it as a real-time event rule.

[0114] For example, customer marketing provides targeted real-time recommendations and retention services based on the customer's transaction behavior over a period of time at the moment the customer makes a transaction. In this case, the real-time event rule + quasi-real-time event rule is configured, and data with the same event number, card number, customer code, etc. over a period of time is calculated together with the real-time transaction.

[0115] For example, if the transaction data made by the customer at 9 am needs to be distributed to other applications for loading at midnight the next day, it is configured as a quasi-real-time event rule.

[0116] Step 2: Introduce the Drools rule engine (open source rule engine) to perform engine rule routing on the data according to the transaction code. When a bank transaction data enters the streaming platform, the Drools engine first determines whether it needs to read historical transaction data based on the transaction code carried in the data. If it does not need to read historical data and only needs to perform real-time calculations, then execute step 3; if the Drools engine determines that it is necessary to read historical data in addition to reading real-time data, then the data is routed to both the real-time rule engine and the quasi-real-time batch rule engine, that is, execute steps 3 and 6;

[0117] Step 3: The data for real-time calculation is routed to the real-time rule engine;

[0118] Step 4: Write data to the corresponding TOPIC of KAFKA;

[0119] Step 5: Go directly to the open source stream processing framework FLINK and use operators to perform calculations to obtain real-time results.

[0120] Step 6: Real-time data that also needs to read historical data is routed to the quasi-real-time batch rule engine;

[0121] Step 7: When the data enters the FLINK computing framework, an open source stream processing framework, in real time, parameters marking the requested data features, such as transaction code, transaction number, application name, date, etc., are passed to HIVE, a batch processing data lake in the batch processing framework;

[0122] Step 8: The reading thread obtains the corresponding table data from the Hadoop distributed file system HDFS, and the reading thread puts each row of data into the reading queue in turn;

[0123] Step 9: The storage thread reads data from the corresponding reading queue and updates the remote dictionary service REDIS (Remote Dictionary Server) cluster.

[0124] Step 10: After storing the data to be retrieved (such as data within a certain period of time with the same event number, card number, customer code, etc. as the transaction) in the REDIS cluster, the open source stream processing framework FLINK calls its own REDIS class to read the data in the REDIS cluster, and associates it with the data that previously entered FLINK directly according to the rules in the quasi-real-time rule engine, such as associating data with the same event number, card number, and customer code to achieve business needs.

[0125] Step 11: At the same time, you can also use user-defined functions (UDFs) to write FLINKSQL for historical data to meet batch requirements. That is, when the data is routed by the Drolls rule engine to the quasi-real-time batch rule engine, the data is loaded into REDIS, and FLINK reads the REDIS data for FLINKSQL batch calculations.

[0126] Based on the above content, the implementation of the specific batch and stream combined data processing example is as follows:

[0127] Step 01. Build a REDIS-based memory database cluster to store key-value data;

[0128] Step 02. Real-time data is written to the open source stream processing framework FLINK through the KAFKA distributed publish-subscribe system;

[0129] Step 03. Non-real-time data transfers data feature parameters to the batch data lake HIVE to obtain corresponding data;

[0130] Step 04. Write the data obtained in step 03 into the REDIS memory database cluster built in step 01 in the form of key-value pairs, as the data source of the open source stream processing framework FLINK;

[0131] Step 05. The data source of FLINK includes the real-time data written by the KAFKA distributed publish-subscribe system in step 02 and the batch data obtained by the REDIS memory database cluster in step 04. If only real-time stream processing is required, the real-time data is used for FLINK operator calculation. If historical data is required for real-time calculation, the two data sources are used for FLINK operator calculation. If only batch processing is required, the data obtained from REDIS is used for FLINKSQL batch calculation.

[0132] From the above content, it can be seen that the batch-stream fusion financial data processing method based on FLINK and REDIS provided by the application example of this application has established a processing method that covers both batch and streaming data, breaking the barriers of the two traditional batch and real-time data frameworks, saving resources while also saving development and maintenance costs. At the same time, compared with the previous calculation of only real-time data, the consideration of historical data can be added, and the data can be analyzed and calculated in more dimensions, giving customers a better experience and giving merchants more intuitive and comprehensive customer behavior. Taking the bank credit card intelligent marketing business as an example, the financial data processing method proposed in this application can be used to analyze customer inflow or loss in real time when customers open or cancel cards at financial institutions, and the data within a certain period of time can be intuitively displayed in batches in the form of reports, so that banks can make targeted adjustments and improvements. At the same time, there is more online data during the day and a larger batch demand at night. Batch-stream fusion is perfectly suitable for this demand and has great promotion value.

[0133] From the hardware level, in order to solve the problems of low efficiency and poor reliability of financial data processing and inability to meet the needs of batch and stream fusion caused by the independence of batch processing and stream processing in the existing financial data processing method, the present application provides an embodiment of an electronic device for implementing all or part of the contents of the financial data processing method, and the electronic device specifically includes the following contents:

[0134] Fig.11 FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig.11 As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig.11 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0135] In one embodiment, the financial data processing function may be integrated into a central processing unit, wherein the central processing unit may be configured to perform the following control:

[0136] Step 110: Based on a preset rule engine, determine whether the target real-time event corresponding to the current financial service request involves non-real-time data. If so, execute step 120.

[0137] Step 120: Retrieve target historical transaction data corresponding to the business parameters of the target real-time event, and store the target historical transaction data in a REDIS cluster in a batch processing system.

[0138] Step 200: The application stream processing system obtains the target real-time data corresponding to the target real-time event, and obtains the target historical transaction data corresponding to the business parameters from the REDIS cluster.

[0139] Step 300: data association is performed on the target real-time data and the target historical transaction data, and processing result data corresponding to the target real-time event is obtained based on the associated target real-time data and target historical transaction data.

[0140] From the above description, it can be seen that the electronic device provided in the embodiment of the present application can effectively realize the batch-stream fusion between the batch processing system and the stream processing system under the large-scale financial data processing architecture by storing the target historical transaction data in the REDIS cluster in the batch processing system, applying the stream processing system to obtain the target historical transaction data corresponding to the business parameters from the REDIS cluster and perform data association between the target real-time data and the target historical transaction data, thereby effectively improving the processing efficiency and intelligence of financial business requests with batch-stream fusion requirements, and effectively improving the reliability and effectiveness of the financial data processing process, effectively meeting the financial business processing scenarios with batch-stream fusion requirements; at the same time, by adopting R The EDIS cluster can effectively realize the combination of batch processing systems and stream processing systems without changing their respective architectural modes, overcoming the deficiency of the existing technology that batch processing and stream processing require the use of two sets of frameworks, while making up for the deficiency of financial business request processing that some businesses need to associate historical data with real-time data, breaking the barriers of the two traditional batch and real-time data frameworks, saving resources while also saving development and maintenance costs. At the same time, compared with the previous calculation of only real-time data, the consideration of historical data can analyze and calculate financial business requests in more dimensions, giving financial customers a better experience, and providing financial institutions or merchants with more intuitive and comprehensive customer behaviors.

[0141] In another embodiment, the financial data processing device may be configured separately from the central processing unit 9100. For example, the financial data processing device may be configured as a chip connected to the central processing unit 9100, and the financial data processing function may be implemented under the control of the central processing unit.

[0142] like Fig.11 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig.11 In addition, the electronic device 9600 may also include Fig.11 For components not shown, reference may be made to the prior art.

[0143] like Fig.11 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0144] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0145] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0146] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0147] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0148] The communication module 9110 is a transmitter / receiver 9110 that sends and receives signals via an antenna 9111. The communication module (transmitter / receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0149] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module, etc. The communication module (transmitter / receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.

[0150] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the financial data processing method in the above embodiments. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, all the steps of the financial data processing method in the above embodiments are implemented by the execution subject being a server or a client. For example, when the processor executes the computer program, the following steps are implemented:

[0151] Step 110: Based on a preset rule engine, determine whether the target real-time event corresponding to the current financial service request involves non-real-time data. If so, execute step 120.

[0152] Step 120: Retrieve target historical transaction data corresponding to the business parameters of the target real-time event, and store the target historical transaction data in a REDIS cluster in a batch processing system.

[0153] Step 200: The application stream processing system obtains the target real-time data corresponding to the target real-time event, and obtains the target historical transaction data corresponding to the business parameters from the REDIS cluster.

[0154] Step 300: data association is performed on the target real-time data and the target historical transaction data, and processing result data corresponding to the target real-time event is obtained based on the associated target real-time data and target historical transaction data.

[0155] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application can effectively realize the batch-stream fusion between the batch processing system and the stream processing system under the large-scale financial data processing architecture by storing the target historical transaction data in the REDIS cluster in the batch processing system, and applying the stream processing system to obtain the target historical transaction data corresponding to the business parameters from the REDIS cluster and perform data association between the target real-time data and the target historical transaction data, thereby effectively improving the processing efficiency and intelligence of financial business requests with batch-stream fusion requirements, and effectively improving the reliability and effectiveness of the financial data processing process, and effectively meeting the financial business processing scenarios with batch-stream fusion requirements; at the same time, by adopting Using REDIS cluster, we can effectively realize the combination of batch processing system and stream processing system without changing their respective architectural modes, overcome the deficiency of batch processing and stream processing in the prior art that two sets of frameworks are needed, and make up for the deficiency of financial business request processing that some businesses need to associate historical data and real-time data, break the barriers of traditional batch and real-time data frameworks, save resources and development and maintenance costs, and add historical data into the consideration compared with the previous calculation of real-time data, so as to analyze and calculate financial business requests in more dimensions, give financial customers a better experience, and provide financial institutions or merchants with more intuitive and comprehensive customer behavior.

[0156] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0158] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0159] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0160] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A financial data processing method, characterized in that: include: Based on the preset rule engine, determine whether the target real-time event corresponding to the current financial business request involves non-real-time data. If so, retrieve the target historical transaction data corresponding to the business parameters of the target real-time event, and store the target historical transaction data in the REDIS cluster in the batch processing system; The application stream processing system obtains the target real-time data corresponding to the target real-time event, and obtains the target historical transaction data corresponding to the business parameter from the REDIS cluster; Performing data association on the target real-time data and the target historical transaction data, and obtaining processing result data corresponding to the target real-time event based on the associated target real-time data and target historical transaction data; Before the preset rule engine determines whether the target real-time event corresponding to the current financial service request involves non-real-time data, the method further includes: Receive financial service requests; Determine the target processing type of the transaction code corresponding to the financial service request according to a preset transaction code configuration rule; the transaction code configuration rule includes: If you want to send a personal balance change reminder SMS, configure it as a real-time event rule; If customer marketing makes targeted real-time recommendations and retention based on the customer's transaction behavior over a period of time at the moment the customer makes a transaction, the configuration is real-time event rules + quasi-real-time event rules; If the transaction data made by the customer at 9 am needs to be distributed to other applications for loading at midnight the next day, configure it as a quasi-real-time event rule; If the target processing type is the preset first or second category, the to-be-processed event corresponding to the financial service request is confirmed as a target real-time event, wherein the first category is a real-time processing type and the second category is a combination of a real-time processing type and a non-real-time processing type.

2. The financial data processing method according to claim 1, characterized in that: Also includes: If the target processing type is the preset third type, confirming the pending event corresponding to the financial service request as a target non-real-time event, wherein the third type is a non-real-time processing type; The business parameters of the target non-real-time event are sent to a batch processing system, so that the batch processing system retrieves data corresponding to the business parameters of the target non-real-time event within a preset processing time range, and obtains processing result data corresponding to the target non-real-time event.

3. The financial data processing method according to claim 1, characterized in that: Also includes: If it is determined that the target real-time event corresponding to the current financial business request does not involve non-real-time data, the business parameters of the target real-time event are sent to the stream processing system so that the stream processing system can retrieve the data corresponding to the business parameters of the target real-time event in real time and obtain the processing result data corresponding to the target real-time event.

4. The financial data processing method according to claim 1, characterized in that: The step of retrieving target historical transaction data corresponding to the business parameters of the target real-time event and storing the target historical transaction data in a REDIS cluster in a batch processing system includes: Sending the business parameters of the target real-time event to a preset batch data lake; Retrieving corresponding target historical transaction data from a preset distributed file system according to the business parameters in the batch data lake; The target historical transaction data is stored in the form of key-value pairs in the REDIS cluster in the batch processing system.

5. The financial data processing method according to claim 4, characterized in that: Before sending the business parameters of the target real-time event to the preset batch data lake, the method further includes: Sending the financial service request to a real-time rule engine corresponding to a stream processing system and a non-real-time batch rule engine corresponding to a batch processing system respectively; Correspondingly, the data association of the target real-time data and the target historical transaction data includes: According to the rules corresponding to the non-real-time batch rule engine and the rules corresponding to the real-time rule engine, data association is performed between the target historical transaction data corresponding to the non-real-time batch rule engine and the target real-time data corresponding to the real-time rule engine.

6. The financial data processing method according to claim 1, characterized in that: The application stream processing system obtains target real-time data corresponding to the target real-time event, including: The preset KAFKA distributed publish-subscribe system is applied to write the target real-time data corresponding to the target real-time event into the stream processing system.

7. A financial data processing device, characterized in that: include: The data judgment module is used to judge whether the target real-time event corresponding to the current financial business request involves non-real-time data based on a preset rule engine. If so, the target historical transaction data corresponding to the business parameters of the target real-time event is retrieved and stored in the REDIS cluster in the batch processing system. A data acquisition module, used for applying a stream processing system to acquire target real-time data corresponding to the target real-time event, and acquiring target historical transaction data corresponding to the business parameters from the REDIS cluster; A data association module, used to perform data association on the target real-time data and the target historical transaction data, and obtain processing result data corresponding to the target real-time event based on the associated target real-time data and target historical transaction data; Before the preset rule engine determines whether the target real-time event corresponding to the current financial service request involves non-real-time data, the method further includes: Receive financial service requests; Determine the target processing type of the transaction code corresponding to the financial service request according to a preset transaction code configuration rule; the transaction code configuration rule includes: If you want to send a personal balance change reminder SMS, configure it as a real-time event rule; If customer marketing makes targeted real-time recommendations and retention based on the customer's transaction behavior over a period of time at the moment the customer makes a transaction, the configuration is real-time event rules + quasi-real-time event rules; If the transaction data made by the customer at 9 am needs to be distributed to other applications for loading at midnight the next day, configure it as a quasi-real-time event rule; If the target processing type is the preset first or second category, the to-be-processed event corresponding to the financial service request is confirmed as a target real-time event, wherein the first category is a real-time processing type and the second category is a combination of a real-time processing type and a non-real-time processing type.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the financial data processing method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the financial data processing method according to any one of claims 1 to 6 is implemented.

Images