Dynamic Cache Financial Risk Control In-process Data Processing Method and System

Abstract

The present application provides a dynamic cache financial risk control in-process data processing method and system, which relates to the field of data management and can be applied to the fields of financial risk control and other fields. The method includes: obtaining a data writing request provided by an in-process risk control application, and parsing to obtain cache data and a cache identifier according to the data writing request; writing the cache data into a local cache cluster, and querying a locally pre-stored storage policy according to the cache identifier; and synchronizing the cache data to other cache clusters in a park associated with the data synchronization component by a data synchronization component according to the query result.

Description

Technical Field

[0001] This application relates to the field of data management and can be applied to the fields of financial risk control and other fields. In particular, it refers to a dynamic caching method and system for processing financial risk control in-process data. Background Art

[0002] In the context of the migration of the bank system from an IBM mainframe to a platform distributed node cluster architecture, as an important part of financial informatization construction, the cache needs to face challenges such as complex data structures and consistency, as well as high system availability and scalability brought about by the continuous growth of the number of scenario accesses and the transformation of risk control strategies. However, the native Redis distributed cache architecture can no longer meet the current financial service requirements in terms of availability, scalability, and storage capacity.

[0003] According to the distributed CAP theory, it is necessary to make a trade-off between data consistency and availability on the premise of ensuring partition tolerance. For example, during major promotions such as Double Eleven and June 18th, the trading volume increases, and the system hopes to temporarily sacrifice a small amount of consistency to greatly improve system availability. After the peak of availability requirements, it will return to the original cache solution. Due to the complex types of data in the field of in-process financial risk control and the frequent involvement of policy interventions such as financial supervision, how to dynamically adjust the data reading, writing, and distribution strategies in a timely manner according to the needs of supervision and business, and to meet the system availability and consistency requirements of the current environment to the greatest extent is an urgent problem to be solved at present. Summary of the Invention

[0004] The purpose of this application is to provide a dynamic caching method and system for processing financial risk control in-process data, to solve the challenges of system availability and consistency faced by traditional distributed caches in the field of in-process risk control, to achieve the dynamic adjustment ability of cache strategies through a more flexible architecture, to more timely meet the needs of supervision and business, and to ensure the stable operation of the financial system.

[0005] To achieve the above object, the dynamic caching method for processing financial risk control in-process data provided by this application specifically includes: obtaining a data write request provided by an in-process risk control application, parsing the cache data and cache identifier according to the data write request; writing the cache data into the local cache cluster, and querying the pre-stored storage strategy locally according to the cache identifier; synchronizing the cache data to other cache clusters in the park associated with the data synchronization component by the data synchronization component according to the query result.

[0006] In the above dynamic caching method for processing financial risk control in-process data, optionally, querying the pre-stored storage strategy according to the cache identifier further includes: when the corresponding storage strategy is not queried according to the cache identifier, extracting the corresponding storage strategy from the data write request; binding the storage strategy to the cache identifier and storing it locally.

[0007] In the above data processing method for dynamic cache financial risk control during events, optionally, synchronizing the cache data from the data synchronization component to other cache clusters associated with the data synchronization component according to the query result further includes: when the corresponding storage policy is not queried according to the cache identifier, synchronizing the cache data from the data synchronization component to other cache clusters associated with the data synchronization component according to the storage policy in the data write request.

[0008] In the above data processing method for dynamic cache financial risk control during events, optionally, the method further includes: obtaining an adjustment policy, parsing the adjustment policy to obtain a batch adjustment instruction or a single processing instruction; adjusting the pre-stored storage policies in each park according to the batch adjustment instruction or the single processing instruction.

[0009] In the above data processing method for dynamic cache financial risk control during events, optionally, the method further includes: monitoring the capacity status of the cache clusters in the parks associated with the data synchronization component; comparing the capacity status with a preset threshold, and generating a warning message according to the comparison result.

[0010] In the above data processing method for dynamic cache financial risk control during events, optionally, the data synchronization component receives and synchronizes the cache data through a Kafka message queue.

[0011] This application also provides a data processing system for dynamic cache financial risk control during events. The system includes: a client, a data synchronization management component, and multiple park data cache devices; the park data cache device includes a proxy component and at least one cache cluster, and each cache cluster is connected to the data synchronization management component respectively; the client is used to receive user access data through the risk control application during events, generate a data write request according to the access data, and provide the data write request to the proxy component; the proxy component is used to parse the received data write request to obtain cache data and a cache identifier; write the cache data into the locally connected cache cluster, and query the pre-stored storage policy locally according to the cache identifier; provide the storage policy and the cache data to the data synchronization management component; the data synchronization management component is used to synchronize the cache data to the cache clusters of other park data cache devices according to the query result.

[0012] In the above data processing system for dynamic cache financial risk control during events, optionally, the system further includes a public support component and an operation and maintenance component; the operation and maintenance component is used to monitor the capacity status of the cache clusters in the parks associated with the data synchronization component; the public support component is used to compare the capacity status with a preset threshold, and generate a warning message according to the comparison result.

[0013] The present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method when executing the computer program.

[0014] The present application also provides a computer-readable storage medium, which stores a computer program for executing the above method.

[0015] The present application also provides a computer program product, comprising a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0016] The beneficial technical effect of this application is that it can timely and dynamically adjust the data reading, writing and allocation strategies according to actual needs, so as to meet the system availability and consistency requirements of the current environment to the greatest extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute a limitation of the present application. In the drawings:

[0018] Figure 1 A flowchart of a method for dynamically caching financial risk control data during processing provided in one embodiment of the present application;

[0019] Figure 2 A schematic diagram of a storage policy self-update process according to an embodiment of the present application;

[0020] Figure 3 This is a logical diagram of a method for processing data during dynamic caching of financial risk control provided in one embodiment of the present application;

[0021] Figure 4 A schematic diagram of a process for actively adjusting a storage policy according to an embodiment of the present application;

[0022] Figure 5 A schematic diagram of a dual-active active-standby asynchronous mode in the same city provided by an embodiment of the present application;

[0023] Figure 6 A schematic diagram of storage strategy adjustment provided in one embodiment of the present application;

[0024] Figure 7 A schematic diagram of a single cluster active-standby mode provided in an embodiment of the present application;

[0025] Figure 8 A schematic diagram of the early warning process provided in one embodiment of the present application;

[0026] Figure 9 This is a schematic diagram of the structure of a dynamic caching financial risk control in-process data processing system provided by one embodiment of the present application;

[0027] Figure 10 A schematic structural diagram of an electronic device provided by an embodiment of the present application. Specific embodiments

[0028] The following will combine the accompanying drawings and embodiments to detail the implementation manner of the present application, so as to fully understand how the present application uses technical means to solve technical problems and the implementation process of achieving technical effects and implement accordingly. It should be noted that as long as there is no conflict, the various embodiments in the present application and the various features in each embodiment can be combined with each other, and the formed technical solutions are all within the protection scope of the present application.

[0029] In addition, the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0030] Please refer to Figure 1 As shown, the dynamic cache financial risk control in-process data processing method provided by the present application specifically includes:

[0031] S101 Obtain a data writing request provided by the in-process risk control application, and parse to obtain cache data and a cache identifier according to the data writing request;

[0032] S102 Write the cache data into the local cache cluster, and query the locally pre-stored storage policy according to the cache identifier;

[0033] S103 Synchronize the cache data from the data synchronization component to other cache clusters in the park associated with the data synchronization component according to the query result.

[0034] Among them, the data synchronization component receives and synchronizes the cache data through the Kafka message queue, can achieve service degradation after a storage node fails, and append data and realize data backhaul after the node failure is restored.

[0035] Specifically, in actual work, the storage strategy may include a single cluster master-slave mode, a same-city multi-active master-slave (synchronous / asynchronous) mode, and a remote multi-active master-slave (synchronous / asynchronous) mode; most of the data in the in-process risk control are read and written in terms of customer code or account card. In the multi-master mode, write conflicts can be avoided to the greatest extent. For a very small number of write conflicts, they will be resolved through the "Last Write Win" strategy in the data synchronization management component, and the relevant Redis cluster will be notified, and records will be generated at the same time for business personnel or developers to check. The staff can choose to set the storage method of any cached data according to actual needs, or use batch definition or a certain type of specified setting to achieve flexible storage of cached data. Each storage strategy has different advantages, and the staff can choose the setting based on actual needs. This application does not make further restrictions here.

[0036] Single-cluster master-slave mode: has better performance, does not involve synchronization waiting time, and occupies fewer resources. However, when a cluster fails, a small amount of data loss may occur, and the service may become unavailable. It is suitable for non-critical data and data with more reads and fewer writes. In-city multi-active master-slave mode: In the asynchronous case, the read and write performance is higher than that of a single cluster, but in a few cases, write conflicts will cause write performance degradation or data loss. In the synchronous case, the write performance is poor, the read performance is good, and the data security is higher. Off-site multi-active master-slave mode: in the event of a city-level disaster, the service is still available. In the asynchronous case, the read and write performance is higher than that of a single cluster, but in a few cases, write conflicts will cause write performance degradation or data loss. In the synchronous case, the write performance is poor, the read performance is good, and the data security is extremely high.

[0037] Please refer to Figure 2 As shown, in one embodiment of the present application, querying the pre-stored storage strategy according to the cache identifier further includes:

[0038] S201: when no corresponding storage policy is found according to the cache identifier, extracting the corresponding storage policy from the data write request;

[0039] S202: Bind the storage policy and the cache identifier and store them locally.

[0040] Furthermore, in another embodiment of the present application, synchronizing the cache data from the data synchronization component to other cache clusters associated with the data synchronization component according to the query results may also include: when no corresponding storage policy is queried according to the cache identifier, synchronizing the cache data from the data synchronization component to other cache clusters associated with the data synchronization component according to the storage policy in the data write request.

[0041] The overall process can be referred to Figure 3As shown in the figure, in actual work, when the SDK sends a write request to the Redis proxy layer, the Redis proxy layer not only passes the write request to the current Redis cluster, but also queries the storage policy of this key. If this storage policy already exists, it uses this policy. If not, it uses the storage policy sent by the SDK and notifies the data synchronization management component to synchronize data with other clusters.

[0042] Please refer to Figure 4 As shown, in one embodiment of the present application, the method further includes:

[0043] S401 obtains an adjustment strategy, and obtains a batch adjustment instruction or a single processing instruction according to the adjustment strategy;

[0044] S402 adjusts the pre-stored storage strategy of each park according to the batch adjustment instruction or the single processing instruction.

[0045] In actual work, please refer to Figures 5 to 7 As shown in the figure, when adjusting the dual-active master-slave asynchronous mode in the same city to the single-cluster master-slave synchronous mode, business personnel adjust the key cache strategy in batches or individually through the Redis key management system. After receiving the update request, the data synchronization management component will notify each related node and automatically adjust the corresponding data. Figure 5 As shown, when the in-process risk control application provides a write request, the proxy component writes the relevant cache data to the cache cluster on one hand, and sends the synchronization data and policy to the data synchronization management component on the other hand. The data synchronization management component performs data and policy synchronization with the proxy component of the same-city campus B based on the received information. After the same-city campus B writes the relevant cache data, it returns the result to the data synchronization management component, completing the same-city active-active master-slave asynchronous data caching.

[0046] Please refer to Figure 6 As shown, when the staff needs to adjust the relevant storage policy, namely the key policy, the storage policy can be sent to the data synchronization management component through a key management system via an http request. The data synchronization management component will notify the connected parks to make corresponding modifications. For example, the agent components of Park A and Park B in the same city, the agent components of the park in the same city will then change the local pre-stored content according to the storage policy and key policy and feedback the change results.

[0047] Please refer to Figure 7 As shown in the figure, during the master-slave synchronization process of a single cluster, the write request is mainly provided by the in-process risk control application. After the proxy component writes the corresponding cache data into the cache cluster according to the write request, the proxy component feeds back the result based on the write situation. Subsequently, the proxy component feeds back the result to the in-process risk control application.

[0048] Please refer to Figure 8As shown, in one embodiment of the present application, the method may further include:

[0049] S801 monitors the capacity status of the cache cluster in the park associated with the data synchronization component;

[0050] S802 compares the capacity status with a preset threshold, and generates warning information according to the comparison result.

[0051] Specifically, in actual work, a monitoring mechanism can be established to judge the storage status of the cache cluster of each park in real time or quasi-real time. When the storage status of any park is abnormal, processing can be arranged in advance to prevent the problem of inconsistent cache data.

[0052] Please refer to Figure 9 As shown, the present application also provides a dynamic cache financial risk control in-process data processing system, the system comprising: a client, a data synchronization management component and multiple campus data cache devices; the campus data cache device comprises an agent component and at least one cache cluster, each cache cluster is respectively connected to the data synchronization management component; the client is used to receive user access data through the in-process risk control application, generate a data write request based on the access data, and provide the data write request to the agent component; the agent component is used to parse the received data write request to obtain cache data and cache identification; write the cache data into a locally connected cache cluster, and query the locally pre-stored storage policy according to the cache identification; provide the storage policy and the cache data to the data synchronization management component; the data synchronization management component is used to synchronize the cache data to the cache cluster of other campus data cache devices according to the query results.

[0053] In the above embodiment, the system may also include a public support component and an operation and maintenance component; the operation and maintenance component is used to monitor the capacity status of the cache cluster in the park associated with the data synchronization component; the public support component is used to compare the capacity status with a preset threshold and generate early warning information based on the comparison result.

[0054] In actual work, the logic implemented by each component in the above-mentioned dynamic caching financial risk control in-process data processing system is as follows:

[0055] Client: Provides access interface of distributed cache platform for applications;

[0056] Proxy component: Responsible for request parsing and forwarding, link tracking and recording; centralized key management, key caching and mapping, and implementation of customized caching strategies: single cluster active-standby mode, same-city multi-active active-standby (synchronous / asynchronous) mode, and remote multi-active active-standby (synchronous / asynchronous) mode;

[0057] Data Synchronization Management Component: Uses Kafka for receiving and pushing data messages, can achieve service degradation after a storage node fails, and after the node failure is recovered, append data to achieve data backhaul. Forward data according to the key corresponding policy; support policy adjustment by operation and maintenance personnel or business personnel, and notify each agent component; handle write conflicts;

[0058] Common Support Component: Responsible for monitoring the status of each park node and providing functions such as monitoring and alarming;

[0059] Operation and Maintenance Component: Responsible for collecting and summarizing the performance capacity data of application requests in the cache cluster, and performing statistical analysis and visual display.

[0060] The beneficial technical effects of this application are as follows: It can dynamically adjust the data reading, writing, and distribution policies in a timely manner according to actual needs, and meet the system availability and consistency requirements of the current environment to the greatest extent.

[0061] This application also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the above method is implemented.

[0062] This application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program for executing the above method.

[0063] This application also provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by the processor, the steps of the above method are implemented.

[0064] As Figure 10 shown, the electronic device 600 may further include: a communication module 110, an input unit 120, an audio processing unit 130, a display 160, and a power supply 170. It should be noted that the electronic device 600 does not necessarily have to include Figure 10 all the components shown in; in addition, the electronic device 600 may further include Figure 10 components not shown in, and reference can be made to the prior art.

[0065] As Figure 10 shown, the central processing unit 100 is sometimes also called a controller or an operation control, and may include a microprocessor or other processor devices and / or logic devices. The central processing unit 100 receives inputs and controls the operations of the various components of the electronic device 600.

[0066] Among them, the memory 140 can be, for example, one or more of a buffer, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory, or other suitable devices. It can store the above-mentioned failure-related information, and can also store a program for executing relevant information. And the central processing unit 100 can execute the program stored in the memory 140 to implement information storage or processing, etc.

[0067] The input unit 120 provides an input to the central processing unit 100. The input unit 120 is, for example, a key or a touch input device. The power supply 170 is used to supply power to the electronic device 600. The display 160 is used to display display objects such as images and texts. The display can be, for example, an LCD display, but is not limited thereto.

[0068] The memory 140 can be a solid-state memory. For example, it can be a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It can also be a memory that stores information even when power is off, can be selectively erased and has more data. An example of this memory is sometimes called an EPROM, etc. The memory 140 can also be some other type of device. The memory 140 includes a buffer memory 141 (sometimes called a buffer). The memory 140 can include an application / function storage unit 142, which is used to store application programs and function programs or the processes for operating the electronic device 600 through the central processing unit 100.

[0069] The memory 140 can also include a data storage unit 143, which is used to store data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 144 of the memory 140 can include various drivers of the electronic device for communication functions and / or for executing other functions of the electronic device (such as a messaging application, an address book application, etc.).

[0070] The communication module 110 is a transmitter / receiver 110 that transmits and receives signals via the antenna 111. The communication module (transmitter / receiver) 110 is coupled to the central processing unit 100 to provide an input signal and receive an output signal, which can be the same as the case of a conventional mobile communication terminal.

[0071] Based on different communication technologies, in the same electronic device, multiple communication modules 110 can be provided, such as a cellular network module, a Bluetooth module, and / or a wireless local area network module, etc. The communication module (transmitter / receiver) 110 is also coupled to a speaker 131 and a microphone 132 via an audio processor 130 to provide an audio output via the speaker 131 and receive an audio input from the microphone 132, so as to implement normal telecommunication functions. The audio processor 130 may include any suitable buffers, decoders, amplifiers, etc. In addition, the audio processor 130 is also coupled to a central processor 100, so that recording can be performed on the local machine through the microphone 132, and the sound stored on the local machine can be played through the speaker 131.

[0072] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one or more of the processes Figure 1 or multiple processes and / or blocks

[0074] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one or more of the processes Figure 1 or multiple processes and / or blocks

[0075] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions for implementing the process Figure 1 in one process or a plurality of processes and / or blocks Figure 1 steps for the functions specified in one block or a plurality of blocks.

[0076] The specific embodiments described above further elaborate the purpose, technical solution and beneficial effects of the present application. It should be understood that the above are only specific embodiments of the present application and are not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A dynamic caching method for processing financial risk control in-process data, characterized in that The method includes: Obtaining a data writing request provided by the in - process risk control application, and parsing to obtain cached data and a cache identifier according to the data writing request; Writing the cached data into the local cache cluster, and querying the locally pre - stored storage policy according to the cache identifier; wherein, the storage policy includes single - cluster master - standby mode, multi - active master - standby mode in the same city, and multi - active master - standby mode across different regions; Synchronizing the cached data from the data synchronization component to other cache clusters in the park associated with the data synchronization component according to the query result; Monitoring the capacity status of the cache clusters in the park associated with the data synchronization component; Comparing the capacity status with a preset threshold, and generating a warning message according to the comparison result; The method further includes: Obtaining an adjustment policy, and parsing to obtain a batch adjustment instruction or a single - processing instruction according to the adjustment policy; Adjusting the locally pre - stored storage policies in each park according to the batch adjustment instruction or the single - processing instruction.

2. The dynamic cache financial risk control in-process data processing method according to claim 1, wherein Querying the locally pre - stored storage policy according to the cache identifier further includes: When the corresponding storage policy is not queried according to the cache identifier, extracting the corresponding storage policy from the data writing request; Binding the storage policy with the cache identifier and storing it locally.

3. The dynamic cache financial risk control in-process data processing method according to claim 2, wherein Synchronizing the cached data from the data synchronization component to other cache clusters associated with the data synchronization component according to the query result further includes: When the corresponding storage policy is not queried according to the cache identifier, synchronizing the cached data from the data synchronization component to other cache clusters associated with the data synchronization component according to the storage policy in the data writing request.

4. The dynamic cache financial risk control in-process data processing method according to claim 1, wherein The data synchronization component receives and synchronizes the cached data through a Kafka message queue.

5. A dynamic caching financial risk control in-process data processing system, characterized in that, The system includes: a client, a data synchronization management component, and multiple park data cache devices; it also includes a public support component and an operation and maintenance component; The park data cache device includes a proxy component and at least one cache cluster, and each cache cluster is connected to the data synchronization management component respectively; The client is used to receive user - accessed data through the in - process risk control application, generate a data writing request according to the access data, and provide the data writing request to the proxy component; The proxy component is used to parse to obtain cached data and a cache identifier according to the received data writing request; write the cached data into the local cache cluster, and query the locally pre - stored storage policy according to the cache identifier; provide the storage policy and the cached data to the data synchronization management component; wherein, the storage policy includes single - cluster master - standby mode, multi - active master - standby mode in the same city, and multi - active master - standby mode across different regions; The data synchronization management component is used to synchronize the cached data to the cache clusters of other park data cache devices according to the query result; The operation and maintenance component is used to monitor the capacity status of the cache clusters in the park associated with the data synchronization component; The public support component is used to compare the capacity status with a preset threshold, and generate a warning message according to the comparison result; The data synchronization management component is further configured to obtain an adjustment policy, parse and obtain a batch adjustment instruction or a single processing instruction according to the adjustment policy; and adjust the storage policies pre-stored in each park according to the batch adjustment instruction or the single processing instruction.

6. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program for the processor to execute the method according to any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 4.

Images