Financial data processing method, device and equipment and storage medium

Abstract

The application discloses a financial data processing method, device and equipment and a storage medium, and relates to the field of financial technology, and the method comprises the following steps: after receiving a business processing instruction, acquiring online business data in a preset period of time; converting the online business data into non-relational data; performing business reconstruction on the non-relational data, starting a preset distributed processing node to process the reconstructed non-relational data, and obtaining summary data in the preset period of time. Through data exchange, business reconstruction and distributed parallel calculation on the online business data, the application improves the processing efficiency of large-amount financial data.

Description

Technical Field

[0001] This invention relates to the field of financial technology, and in particular to a financial data processing method, apparatus, device, and storage medium. Background Technology

[0002] In recent years, the internet has been revolutionizing the face of traditional finance. The internet's spirit of "openness, equality, collaboration, and sharing" is permeating traditional financial practices, with both the internet and finance sectors experiencing significant changes. On the one hand, financial institutions are proactively utilizing internet platforms to transform traditional business models; on the other hand, internet companies, relying on technology and platforms, are beginning to penetrate the financial sector. Financial participants have gained more effective direct contact through the internet, resulting in greater transparency, lower intermediary costs, and greater convenience in traditional financial transactions.

[0003] In traditional financial business systems, a centralized processing method is used for periodic batch processing. Due to the huge amount of data, the batch processing time is long and the processing efficiency is low. Summary of the Invention

[0004] The main objective of this invention is to provide a financial data processing method, apparatus, device, and storage medium, which aims to solve the problem of low efficiency in processing large amounts of data in financial business systems.

[0005] To achieve the above objectives, the present invention provides a financial data processing method, the method comprising:

[0006] Upon receiving a service processing instruction, retrieve online service data within a preset time period;

[0007] Convert the online service data into non-relational data;

[0008] The non-relational data is restructured for business purposes, and a preset distributed processing node is started to process the restructured non-relational data to obtain summary data within the preset period.

[0009] Optionally, the step of obtaining online service data within a preset period includes:

[0010] Check whether the online service data in the distributed cache is valid;

[0011] If the online service data in the distributed cache is valid, then the online service data within the preset period time is retrieved from the distributed cache.

[0012] If the online service data in the distributed cache is invalid, then the online service data within the preset period is retrieved from the preset relational database and updated to the distributed cache.

[0013] Optionally, the step of converting the online service data into non-relational data includes:

[0014] The preset data exchange component is invoked to convert the business processing instructions into mapping tasks.

[0015] The online service data is migrated to a preset distributed storage system according to the mapping task, and the online service data in the preset distributed storage system is used as the non-relational data.

[0016] Optionally, the step of restructuring the non-relational data and starting a preset distributed processing node to process the restructured non-relational data includes:

[0017] Based on the business type information in the business processing instruction, the non-relational data is classified to obtain non-relational data of each category;

[0018] Each category of non-relational data is assigned a corresponding preset distributed processing node, and each preset distributed processing node is activated to process the corresponding category of non-relational data.

[0019] Optionally, after the step of starting the preset distributed processing node to process the reconstructed non-relational data and obtain the summary data within the preset period, the method further includes:

[0020] Determine whether the summary data is a preset cache type data;

[0021] If the summary data is not a preset cache type of data, then the summary data is stored in a preset online database;

[0022] If the summary data is a preset cache type of data, then the summary data is stored in the preset distributed storage system.

[0023] Optionally, after the step of storing the summary data in the preset distributed storage system if the summary data is a preset cache type, the method further includes:

[0024] Upon receiving a data call request from the online service processing system, the summary data is sent to the online service processing system through a preset data interface, wherein the preset data interface is set based on the preset distributed storage system.

[0025] Optionally, after the step of initiating the preset distributed processing node to process the reconstructed non-relational data, the method further includes:

[0026] If it is detected that the preset distributed processing node has not returned the summary data within a preset time period, then the idle distributed processing node with the highest business relevance to the processing task of the preset distributed processing node is determined.

[0027] The idle distributed processing node is invoked to process the reconstructed non-relational data.

[0028] Furthermore, to achieve the above objectives, the present invention also provides a financial data processing apparatus, the financial data processing apparatus comprising:

[0029] The acquisition module is used to acquire online service data within a preset period after receiving a service processing instruction;

[0030] The conversion module is used to convert the online service data into non-relational data;

[0031] The processing module is used to reconstruct the business logic of the non-relational data, start a preset distributed processing node to process the reconstructed non-relational data, and obtain summary data within the preset period.

[0032] The acquisition module is also used for:

[0033] Check whether the online service data in the distributed cache is valid;

[0034] If the online service data in the distributed cache is valid, then the online service data within the preset period time is retrieved from the distributed cache.

[0035] If the online service data in the distributed cache is invalid, then the online service data within the preset period is retrieved from the preset relational database and updated to the distributed cache.

[0036] The conversion module is also used for:

[0037] The preset data exchange component is invoked to convert the business processing instructions into mapping tasks.

[0038] The online service data is migrated to a preset distributed storage system according to the mapping task, and the online service data in the preset distributed storage system is used as the non-relational data.

[0039] The processing module is also used for:

[0040] Based on the business type information in the business processing instruction, the non-relational data is classified to obtain non-relational data of each category;

[0041] Each category of non-relational data is assigned a corresponding preset distributed processing node, and each preset distributed processing node is activated to process the corresponding category of non-relational data.

[0042] The distributed data processing device may also include a second storage module for:

[0043] Determine whether the summary data is a preset cache type data;

[0044] If the summary data is not a preset cache type of data, then the summary data is stored in a preset online database;

[0045] If the summary data is a preset cache type of data, then the summary data is stored in the preset distributed storage system.

[0046] The second storage module is also used for:

[0047] Upon receiving a data call request from the online service processing system, the summary data is sent to the online service processing system through a preset data interface, wherein the preset data interface is set based on the preset distributed storage system.

[0048] The processing module is also used for:

[0049] If it is detected that the preset distributed processing node has not returned the summary data within a preset time period, then the idle distributed processing node with the highest business relevance to the processing task of the preset distributed processing node is determined.

[0050] The idle distributed processing node is invoked to process the reconstructed non-relational data.

[0051] In addition, to achieve the above objectives, the present invention also provides an electronic device, the electronic device comprising: a memory, a processor, and a financial data processing program stored on the memory and executable on the processor, the financial data processing program being configured to implement the steps of the financial data processing method as described above.

[0052] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a financial data processing program, which, when executed by a processor, implements the steps of the financial data processing method described above.

[0053] Upon receiving a business processing instruction, this invention acquires online business data within a preset period, converts the online business data into non-relational data, reconstructs the business logic of the non-relational data, and initiates preset distributed processing nodes to process the reconstructed non-relational data, thereby obtaining summary data within the preset period. By converting online business data into non-relational data, this invention provides data type support for distributed data processing. Furthermore, this invention reconstructs the business logic of the non-relational data to adapt to the characteristics of distributed computing, further reducing batch processing runtime and improving data processing efficiency on the basis of distributed computing. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of the structure of an electronic device in the hardware operating environment involved in the embodiments of the present invention;

[0055] Figure 2 This is a flowchart illustrating the first embodiment of the financial data processing method of the present invention;

[0056] Figure 3 This is a flowchart illustrating the second embodiment of the financial data processing method of the present invention;

[0057] Figure 4 This is a flowchart illustrating the third embodiment of the financial data processing method of the present invention;

[0058] Figure 5 This is a flowchart illustrating the fourth embodiment of the financial data processing method of the present invention;

[0059] Figure 6 This is a schematic diagram of the financial data processing device of the present invention.

[0060] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0061] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0062] In financial business scenarios, there are many situations that require processing large amounts of financial data. The processing speed of traditional financial business platforms is gradually unable to keep up with the pace of data growth, resulting in very low data processing efficiency.

[0063] The main technical solution of this invention is as follows: after receiving a business processing instruction, online business data within a preset period is obtained; the online business data is converted into non-relational data; the non-relational data is reconstructed, and a preset distributed processing node is started to process the reconstructed non-relational data to obtain summary data within the preset period.

[0064] Reference Figure 1 , Figure 1 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of the present invention.

[0065] like Figure 1 As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0066] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0067] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and a financial data processing program.

[0068] exist Figure 1 In the illustrated electronic device, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the electronic device of the present invention can be disposed in the electronic device, and the electronic device calls the financial data processing program stored in the memory 1005 through the processor 1001 and executes the following steps:

[0069] Upon receiving a service processing instruction, retrieve online service data within a preset time period;

[0070] Convert the online service data into non-relational data;

[0071] The non-relational data is restructured for business purposes, and a preset distributed processing node is started to process the restructured non-relational data to obtain summary data within the preset period.

[0072] Optionally, the processor 1001 may invoke the financial data processing program stored in the memory 1005 and further execute the following steps:

[0073] Check whether the online service data in the distributed cache is valid;

[0074] If the online service data in the distributed cache is valid, then the online service data within the preset period time is retrieved from the distributed cache.

[0075] If the online service data in the distributed cache is invalid, then the online service data within the preset period is retrieved from the preset relational database and updated to the distributed cache.

[0076] Optionally, the processor 1001 may invoke the financial data processing program stored in the memory 1005 and further execute the following steps:

[0077] The preset data exchange component is invoked to convert the business processing instructions into mapping tasks.

[0078] The online service data is migrated to a preset distributed storage system according to the mapping task, and the online service data in the preset distributed storage system is used as the non-relational data.

[0079] Optionally, the processor 1001 may invoke the financial data processing program stored in the memory 1005 and further execute the following steps:

[0080] Based on the business type information in the business processing instruction, the non-relational data is classified to obtain non-relational data of each category;

[0081] Each category of non-relational data is assigned a corresponding preset distributed processing node, and each preset distributed processing node is activated to process the corresponding category of non-relational data.

[0082] Optionally, the processor 1001 may invoke the financial data processing program stored in the memory 1005 and further execute the following steps:

[0083] Determine whether the summary data is a preset cache type data;

[0084] If the summary data is not a preset cache type of data, then the summary data is stored in a preset online database;

[0085] If the summary data is a preset cache type of data, then the summary data is stored in the preset distributed storage system.

[0086] Optionally, the processor 1001 may invoke the financial data processing program stored in the memory 1005 and further execute the following steps:

[0087] Upon receiving a data call request from the online service processing system, the summary data is sent to the online service processing system through a preset data interface, wherein the preset data interface is set based on the preset distributed storage system.

[0088] Optionally, the processor 1001 may invoke the financial data processing program stored in the memory 1005 and further execute the following steps:

[0089] If it is detected that the preset distributed processing node has not returned the summary data within a preset time period, then the idle distributed processing node with the highest business relevance to the processing task of the preset distributed processing node is determined.

[0090] The idle distributed processing node is invoked to process the reconstructed non-relational data.

[0091] This invention provides a financial data processing method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating a first embodiment of a financial data processing method according to the present invention.

[0092] In this embodiment, the financial data processing method includes:

[0093] Step S10: After receiving the service processing instruction, obtain the online service data within a preset period.

[0094] This embodiment uses a banking system as an example. Online transactions can be considered as transactions processed at the bank's counter during working hours, generating online transaction data during the process. After the bank closes, the system can perform periodic batch processing on this online transaction data, such as end-of-day processing, weekend processing, or month-end processing. The preset period can be set according to batch processing needs; for example, for end-of-day processing, the preset period could be set to 00:00:00 to 23:59:59 of the previous day.

[0095] Step S20: Convert the online service data into non-relational data;

[0096] The financial data processing architecture in this embodiment can be composed of three parts: data exchange, Hadoop cluster, and data interface.

[0097] Online business data can be stored in relational databases, which can maintain data consistency, but are not good at writing large amounts of data or handling simple queries that require fast results. Online business data can be converted into non-relational data, which is achieved by the data exchange part of the financial data processing framework.

[0098] As an example, step S20, the step of converting the online service data into non-relational data, includes:

[0099] Step a: Invoke a preset data exchange component to convert the business processing instructions into mapping tasks. The preset data exchange component can be Sqoop. Sqoop is primarily used to transfer data between Hadoop and relational databases or mainframes, importing data from a relational database management system into the Hadoop Distributed File System (HDFS), or transforming and exporting data from Hadoop to a relational database management system. Sqoop can be invoked via the Java API (Application Programming Interface). Sqoop can convert business processing instructions into Map tasks in Hadoop. Each Map task reads a portion of data from the relational database, allowing multiple Map tasks to replicate concurrently, quickly copying the entire data to HDFS (Hadoop Distributed File System) in Hadoop.

[0100] Step b involves migrating the online service data to a pre-defined distributed storage system according to the mapping task, and using the online service data in the pre-defined distributed storage system as the non-relational data. The pre-defined distributed storage system can be HDFS or HBase, which resides within a Hadoop cluster and provides distributed storage services. After migrating the online service data to the pre-defined distributed storage system, the online service data stored in the relational database is transformed into non-relational data stored in the pre-defined distributed system, providing data support for subsequent distributed computing within the Hadoop cluster.

[0101] Step S30: Perform business reconstruction on the non-relational data, start the preset distributed processing node to process the reconstructed non-relational data, and obtain the summary data within the preset period.

[0102] In the process of distributed processing of non-relational data, the characteristics of distributed computing can be adapted to restructure the business of non-relational data. For example, data that requires simple queries or simple calculations can be merged to give full play to the characteristics of distributed computing.

[0103] As an example, after the step of starting the pre-defined distributed processing nodes to process the reconstructed non-relational data, it may also include:

[0104] Step c: If it is detected that the preset distributed processing node has not returned the summary data within a preset time period, then determine the idle distributed processing node with the highest business relevance to the processing task of the preset distributed processing node.

[0105] Step d: Invoke the idle distributed processing node to process the reconstructed non-relational data.

[0106] In processing financial data using distributed parallel computing, each pre-defined distributed processing node can periodically return its data processing results and its own status. Pre-defined distributed processing nodes may also fail to respond to data processing tasks; in this case, they can be considered to be in a silent state. If a pre-defined distributed processing node assigned a data processing task remains silent for more than a preset time period, it can be considered to have failed. The data processing task is then reallocated. During reallocation, we first identify currently idle distributed processing nodes that are not processing data. Then, based on the business type in the data processing task, we identify other data processing tasks with the highest business relevance to that business type. We then find idle distributed processing nodes associated with these other data processing tasks with the highest business relevance and call these idle distributed processing nodes to process the unfinished data processing tasks of the failed pre-defined distributed processing nodes. Reallocating data processing tasks based on business relevance improves data processing efficiency while making reasonable use of node resources.

[0107] In this embodiment, after receiving a business processing instruction, the system acquires online business data generated within a preset period, converts the online business data into non-relational data, reconstructs the non-relational data, and starts a preset distributed processing node to process the reconstructed non-relational data, thereby obtaining summary data within the preset period. By converting the online business data into non-relational data, the system provides data type support for distributed data processing. The system also reconstructs the non-relational data to adapt to the characteristics of distributed computing, reducing batch processing runtime and improving data processing efficiency.

[0108] Furthermore, in the second embodiment of the financial data processing method of the present invention, referring to Figure 3 The method includes:

[0109] Step S11: Detect whether the online service data in the distributed cache is valid;

[0110] As an example, before the step of detecting whether the online service data in the distributed cache is valid, the following may also be included:

[0111] Step e: When online services generate online service data, the online service data is stored in a preset relational database and a distributed cache, respectively.

[0112] In banking systems, a large number of online transactions are processed every business day. The online transaction data generated by these transactions can first be stored in a relational database. Relational databases store data in rows and columns, which ensures the correctness and reliability of transactions.

[0113] For distributed caching, it is equivalent to a database infrastructure, rather than being part of memory as in traditional systems. Distributed caching can be used in conjunction with relational databases. After online business data is written to the relational database, the distributed cache can update or delete its own existing online business data, keeping it synchronized with the online business data in the relational database. If the relational database needs to be rolled back, the distributed cache will also be rolled back synchronously.

[0114] In this embodiment, an RMDB (Remote Management Database) can be configured to store online service data. After receiving a service processing instruction, the RMDB checks whether the online service data in the distributed cache that works with the RMDB is valid. Redis can be used as the specific carrier of the distributed cache.

[0115] Step S12: If the online service data in the distributed cache is valid, then retrieve the online service data within the preset period from the distributed cache.

[0116] If the online service data in the distributed cache includes online service data within a preset period, then the online service data in the distributed cache can be considered valid and can be directly retrieved from the distributed cache. The online service data in the distributed cache originates from a relational database.

[0117] Step S13: If the online service data in the distributed cache is invalid, retrieve the online service data within a preset period from the relational database and update the online service data within the preset period to the distributed cache.

[0118] If the online business data in the distributed cache is empty or has expired, it can be considered invalid. In this case, online business data within a preset period can be retrieved from the relational database. By using a distributed cache in conjunction with a relational database, after updating the distributed cache with online business data within a preset period, subsequent online business data can be directly read from the distributed cache, improving data access efficiency. A distributed cache can alleviate the access pressure on the relational database, handling situations requiring large volumes of data access within a short period.

[0119] In another possible implementation, when acquiring online service data within a preset period, the online service data can be read directly from the distributed cache. If the read fails, the data can be read from a preset relational database, and the online service data in the preset relational database can be updated to the distributed cache.

[0120] In this embodiment, using a relational database and a distributed cache together for storing and accessing online business data can improve data access efficiency and shorten the preparation time before data processing.

[0121] Furthermore, in the third embodiment of the distributed data processing method of the present invention, referring to Figure 4 The method includes:

[0122] Step S21: Based on the business type information in the business processing instruction, classify the non-relational data to obtain non-relational data of each category;

[0123] Batch processing tasks encompass various business types, such as interest rate calculation, accrual, interest settlement, accounting entries, and general ledger balances. When classifying non-relational data, it can be categorized into related and unrelated types. For example, in a complete business process, there are five processes: A, B, C, D, and E. According to the business processing logic, processes A, B, C, and D must be processed sequentially, while the processing order of process E can be flexibly adjusted. In this case, the non-relational data for processes A, B, C, and D is related non-relational data, while the non-relational data for process E is unrelated non-relational data.

[0124] In traditional business processes, business types with complex computational logic require following fixed steps. In this embodiment, business processes can be restructured based on processing requests in the business processing instructions, shortening the waiting time for non-related, non-relational data before processing.

[0125] Step S22: Assign corresponding preset distributed processing nodes to each category of non-relational data, and start each preset distributed processing node to process the corresponding category of non-relational data.

[0126] This embodiment employs distributed parallel computing for data processing, with data storage and processing services provided by a Hadoop cluster. Hadoop, written in Java, is an open-source framework that stores massive amounts of data and runs distributed analytics applications on a distributed server cluster. Its core components are HDFS and MapReduce. HDFS is used for distributed data storage and retrieval. MapReduce is a computing framework that distributes computational tasks to servers within the cluster for execution.

[0127] Non-relational data is stored in HDFS or HBase within the Hadoop cluster and processed by the MapReduce mechanism. When allocating pre-defined distributed processing nodes, corresponding pre-defined distributed processing nodes can be assigned to both related and unrelated non-relational data. Furthermore, multiple processing nodes can be allocated to each category of non-relational data to avoid situations where the processing volume of a single node is too large, thus affecting the overall processing efficiency.

[0128] Specifically, in one possible implementation, corresponding to the A, B, C, D and E services listed in step S21, the processing of service B requires non-relational data from service E. Different preset distributed processing nodes can be allocated to service E and services A, B, C and D. Service E is computed in parallel with service A. Then, during the processing of service B, the pre-computed data from service E can be used directly, shortening the processing time of the entire business process.

[0129] In another possible implementation, each business process may contain multiple sub-business processes. When there are enough preset distributed processing nodes, each sub-business can be assigned a corresponding preset distributed processing node. For example, business E includes E1, E2, and E3. Business A needs data from E1, and business B needs data from E3. E1, E2, and E3 are respectively assigned corresponding preset distributed processing nodes, which can also shorten the processing time of the entire business process.

[0130] In this embodiment, business restructuring of non-relational data is performed to adapt to the parallel processing characteristics of distributed computing, which can reduce the running time of batch processing and improve data processing efficiency.

[0131] Furthermore, in the fourth embodiment of the distributed data processing method of the present invention, referring to Figure 5 The method includes:

[0132] Step S31: Determine whether the summary data is a preset cache type data;

[0133] Summary data can be understood as data generated after a business process is completed. In a periodic batch processing scenario, multiple business processes may be handled, and the summary data may be used in subsequent processes. Before performing periodic batch processing, preset cached data types, such as interest rate data, can be set.

[0134] Step S32: If the summary data is not a preset cache type data, then store the summary data in a preset online database;

[0135] Compared to the preset cached data type, other types of summary data are accessed less frequently and can be stored in a preset online database. This data, such as settlement time and settlement balance, can be updated within the previous period. As needed, the summary data can be updated to the online database. When online transactions begin during working hours, the online business processing system can directly access the summary data in the online database.

[0136] Step S33: If the summary data is a preset cache type data, then store the summary data in the preset distributed storage system;

[0137] Pre-defined cached data types typically have high access frequency during periodic batch processing. The pre-defined distributed storage system can be HDFS or HBase. HDFS or HBase is characterized by high query efficiency for large datasets; storing pre-defined cached data in HDFS or HBase not only adapts to the characteristics of distributed storage but also alleviates the storage and access pressure on the online database.

[0138] Step S34: After receiving a data call request from the online service processing system, the summary data is sent to the online service processing system through a preset data interface, wherein the preset data interface is set based on the preset distributed storage system.

[0139] The summary data generated after batch processing is stored in HDFS or HBase. HDFS or HBase is part of the Hadoop cluster and can provide a data interface for the online business processing system. The online business processing system can directly call the summary data through the data interface during online transactions.

[0140] The data processing architecture in this embodiment can be interconnected with online transactions through a data interface. When an online transaction needs to query summary data stored in HDFS or HBase, it can send a data call request to Hadoop through the data interface, and Hadoop will return the summary data corresponding to the data call request. HDFS or HBase are distributed storage systems with high efficiency for querying large amounts of data, which does not affect the processing speed of online transactions.

[0141] In this embodiment, the summary data is stored in different storage locations according to its different types, which can adapt to the different data access needs of online transactions. For the summary data stored in the distributed storage system, a data interface is provided for the online transaction system to call the summary data, which can reduce the data access pressure of the online database. Moreover, the distributed storage system has high efficiency in querying large amounts of data, and does not affect the data access efficiency of online transactions when reducing the database access pressure.

[0142] The present invention also provides a financial data processing device, such as... Figure 6 As shown, the financial data processing device includes:

[0143] The acquisition module 101 is used to acquire online service data within a preset period after receiving a service processing instruction;

[0144] The conversion module 102 is used to convert the online service data into non-relational data;

[0145] Processing module 103 is used to perform business reconstruction on the non-relational data, start a preset distributed processing node to process the reconstructed non-relational data, and obtain summary data within the preset period.

[0146] Module 101 is also used for:

[0147] Check whether the online service data in the distributed cache is valid;

[0148] If the online service data in the distributed cache is valid, then the online service data within the preset period time is retrieved from the distributed cache.

[0149] If the online service data in the distributed cache is invalid, then the online service data within the preset period is retrieved from the preset relational database and updated to the distributed cache.

[0150] Conversion module 102 is also used for:

[0151] The preset data exchange component is invoked to convert the business processing instructions into mapping tasks.

[0152] The online service data is migrated to a preset distributed storage system according to the mapping task, and the online service data in the preset distributed storage system is used as the non-relational data.

[0153] Processing module 103 is also used for:

[0154] Based on the business type information in the business processing instruction, the non-relational data is classified to obtain non-relational data of each category;

[0155] Each category of non-relational data is assigned a corresponding preset distributed processing node, and each preset distributed processing node is activated to process the corresponding category of non-relational data.

[0156] The distributed data processing device may also include a second storage module for:

[0157] Determine whether the summary data is a preset cache type data;

[0158] If the summary data is not a preset cache type of data, then the summary data is stored in a preset online database;

[0159] If the summary data is a preset cache type of data, then the summary data is stored in the preset distributed storage system.

[0160] The second storage module is also used for:

[0161] Upon receiving a data call request from the online service processing system, the summary data is sent to the online service processing system through a preset data interface, wherein the preset data interface is set based on the preset distributed storage system.

[0162] Processing module 103 is also used for:

[0163] If it is detected that the preset distributed processing node has not returned the summary data within a preset time period, then the idle distributed processing node with the highest business relevance to the processing task of the preset distributed processing node is determined.

[0164] The idle distributed processing node is invoked to process the reconstructed non-relational data.

[0165] The present invention also provides an electronic device comprising: a memory, a processor, and a financial data processing program stored in the memory and executable on the processor, the financial data processing program being configured to implement the steps of the financial data processing method described above. Specific embodiments of the electronic device of the present invention are described in conjunction with the embodiments of the financial data processing method described above, and will not be repeated here.

[0166] The present invention also provides a computer-readable storage medium storing a financial data processing program, which, when executed by a processor, implements the steps of the financial data processing method described above. Specific embodiments of the computer-readable storage medium of the present invention are described in the various embodiments of the above-described financial data processing method, and will not be repeated here.

[0167] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0168] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0169] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0170] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A financial data processing method, characterized in that, The financial data processing method includes the following steps: When online services generate online service data, the online service data is stored in a preset relational database and a distributed cache, respectively. The distributed cache is synchronized with the preset relational database and is subject to rollback synchronization. Upon receiving a business processing instruction, the system checks whether the online business data in the distributed cache is valid. The online business refers to the business processed at the bank's counter during working hours, and the business processing instruction refers to the bank's periodic daily settlement, weekend settlement, and month-end settlement instructions. If the online service data in the distributed cache is valid, then retrieve the online service data within a preset period from the distributed cache; If the online service data in the distributed cache is invalid, the online service data within the preset period is retrieved from the preset relational database and updated to the distributed cache. The preset data exchange component is invoked to convert the business processing instructions into mapping tasks. The online service data is migrated to a preset distributed storage system according to the mapping task, and the online service data in the preset distributed storage system is treated as non-relational data, wherein the preset distributed storage system is HDFS or HBase; The non-relational data is restructured for business purposes, and a preset distributed processing node is started to process the restructured non-relational data to obtain summary data within the preset period. If it is detected that the preset distributed processing node has not returned the summary data within a preset time period, then the idle distributed processing node with the highest business relevance to the processing task of the preset distributed processing node is determined. The idle distributed processing node is invoked to process the reconstructed non-relational data. The step of restructuring the non-relational data and activating preset distributed processing nodes to process the restructured non-relational data includes: Based on the business type information in the business processing instruction, the non-relational data is classified to obtain non-relational data of each category; Each category of non-relational data is assigned a corresponding preset distributed processing node, and each preset distributed processing node is activated to process the corresponding category of non-relational data.

2. The financial data processing method as described in claim 1, characterized in that, After the step of starting the preset distributed processing node to process the reconstructed non-relational data and obtaining the summary data within the preset period, the method further includes: Determine whether the summary data is a preset cache type data; If the summary data is not a preset cache type of data, then the summary data is stored in a preset online database; If the summary data is a preset cache type of data, then the summary data is stored in the preset distributed storage system.

3. The financial data processing method as described in claim 2, characterized in that, After the step of storing the summary data in the preset distributed storage system if the summary data is of a preset cache type, the method further includes: Upon receiving a data call request from the online service processing system, the summary data is sent to the online service processing system through a preset data interface, wherein the preset data interface is set based on the preset distributed storage system.

4. A financial data processing device, characterized in that, The financial data processing device includes: The acquisition module is used to store online business data generated by online business transactions into a preset relational database and a distributed cache, respectively. The distributed cache is synchronized and rollback synchronized with the preset relational database. Upon receiving a business processing instruction, the module checks whether the online business data in the distributed cache is valid. The online business is a counter service processed during bank working hours, and the business processing instruction is a periodic daily settlement, weekend settlement, and month-end settlement instruction from the bank's business system. If the online business data in the distributed cache is valid, the module retrieves online business data within a preset period from the distributed cache. If the online business data in the distributed cache is invalid, the module retrieves the online business data within the preset period from the preset relational database and updates the distributed cache with the online business data within the preset period. A conversion module is used to convert the online business data into non-relational data. Specifically, the conversion module is used to call a preset data exchange component to convert the business processing instructions into mapping tasks; and to migrate the online business data to a preset distributed storage system according to the mapping tasks, and to treat the online business data in the preset distributed storage system as non-relational data, wherein the preset distributed storage system is HDFS or HBase. The processing module is used to reconstruct the business logic of the non-relational data, start preset distributed processing nodes to process the reconstructed non-relational data, and obtain summary data within the preset period. Specifically, the processing module is used to classify the non-relational data according to the business type information in the business processing instruction to obtain non-relational data of each category; allocate corresponding preset distributed processing nodes to each category of non-relational data, and start each preset distributed processing node to process the corresponding category of non-relational data. The processing module is further configured to, if it is detected that the preset distributed processing node has not returned the summary data within a preset time period, determine the idle distributed processing node with the highest business relevance to the processing task of the preset distributed processing node; and call the idle distributed processing node to process the reconstructed non-relational data.

5. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a financial data processing program stored in the memory and executable on the processor, the financial data processing program being configured to implement the steps of the financial data processing method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a financial data processing program, which, when executed by a processor, implements the steps of the financial data processing method as described in any one of claims 1 to 3.

Images