A cross-bank fund clearing processing method and system based on big data

Abstract

The present application relates to the technical field of cross-line fund clearing, and discloses a cross-line fund clearing processing method and system based on big data, which comprises the following steps: firstly, the transmission path of a standardized clearing message is extracted to generate a routing identifier, which is then distributed to an account index; subsequently, the message is divided into directional queues according to the fund direction and the routing identifier, and the routing retention degree of each queue affected by the transmission delay of the equipment is calculated; the current fund window is generated by combining the account available position with the routing retention degree; during the clearing cycle, the de-retention sorting is performed on the outgoing queue, and the messages with high retention degree are preferentially stripped and sent to the accounting preparation area; finally, the standard accounting is completed and the result is output. The present application effectively identifies and actively reduces the message misplacement retention problem caused by the time consumption difference of each node equipment, and improves the fund circulation rate and overall processing efficiency within the same clearing batch.

Description

Technical Field

[0001] This invention relates to the field of interbank fund clearing technology, and more specifically, to a method and system for interbank fund clearing based on big data. Background Technology

[0002] In current interbank fund clearing scenarios, business messages from participating banks need to pass through multiple independent nodes, such as local front-end machines, communication gateways, and signature verification devices, before finally being transferred to the central clearing account book to complete debit and credit operations. Due to objective differences in the processing bandwidth and response time of various hardware nodes and dedicated network lines, outgoing and incoming funds messages will experience varying degrees of asynchronous delay in the transmission link. When transaction concurrency increases or certain encryption gateways experience a sudden surge in load, some clearing messages passing through specific routing nodes will inevitably experience transmission timing misalignment, thus missing the preset periodic matching window.

[0003] Existing clearing systems typically employ a release logic based on the absolute time sequence of messages arriving at the central endpoint or prioritizing messages by transaction amount. This conventional queue scheduling method only focuses on endpoint timeline parameters or transaction detail parameters, failing to compensate for the underlying device latency experienced by messages from a global perspective. This leads to a common anomaly: even if a clearing account's inflows are sufficient to cover outflows, meaning it meets the net clearing conditions, the system may still determine insufficient positions or forcibly suspend relevant messages to the next cycle because matching fund detail messages are affected by the processing time difference of gateways or encryption devices and fail to fall synchronously into the same matching cycle. This situation results in the invalidation of local positions, increases the queuing depth of interbank clearing systems during peak processing periods, and reduces the availability and flow efficiency of funds within the same clearing batch. Summary of the Invention

[0004] This invention provides a cross-bank fund clearing and processing method and system based on big data, which solves the technical problems mentioned in the background art.

[0005] Firstly, a big data-based interbank funds clearing processing method is applied to an interbank funds clearing system comprising a front-end server, communication gateway, signature verification equipment group, regional processing entry point, central queue entry point, and clearing account book, including:

[0006] Extract the clearing account number carried in the standardized clearing message entering the interbank fund clearing system, establish routing features representing the physical transmission path of the standardized clearing message, and allocate the standardized clearing message to the account index of the clearing account corresponding to the clearing account number;

[0007] Based on the account index, the standardized clearing message is divided into direction queues according to the fund direction, the routing feature, and the clearing cycle number. The direction queues with the fund direction being debit are divided into outflow queues, and the direction queues with the fund direction being credit are divided into inflow queues.

[0008] To guide fund release rearrangement to proactively mitigate dwell interference caused by physical entry structure, the routing dwell degree, which characterizes the degree of structural delay, is calculated for the standardized clearing messages within each of the said directional queues.

[0009] Based on the current available position of the clearing account and the routing stagnation, a current funding window is generated to constrain the funding release boundary;

[0010] Within each clearing cycle, the withdrawal queue entering the current period's funds window is sorted to remove lingering issues, and the stripped standardized clearing messages are sent to the accounting preparation area.

[0011] Standard accounting is performed on the standardized clearing messages in the accounting preparation area, converting them into accounting records and batch summary records on the clearing account book;

[0012] Based on the completed accounting records and batch summary records, the interbank fund clearing processing results are output.

[0013] Secondly, a cross-bank fund clearing and processing system based on big data includes:

[0014] The account index configuration module is used to extract the clearing account number carried in the standardized clearing message entering the interbank fund clearing network, establish routing features representing the physical transmission path of the standardized clearing message, and allocate the standardized clearing message to the account index of the clearing account corresponding to the clearing account number.

[0015] The direction queue segmentation module is used to segment the standardized clearing message into direction queues based on the account index, according to the fund direction, the routing feature, and the clearing cycle number. The direction queues with the fund direction being debit are divided into outgoing queues, and the direction queues with the fund direction being credit are divided into incoming queues.

[0016] The routing retention calculation module is used to calculate the routing retention degree, which represents the degree of structural delay, of the standardized clearing messages in each of the direction queues in order to guide the rearrangement of fund release and actively resolve the retention interference caused by the physical entry structure.

[0017] The funds window generation module is used to generate a current funds window that constrains the funds release boundary based on the current available position of the clearing account and the routing retention rate.

[0018] The de-delay release module is used to perform de-delay sorting on the withdrawal queue that enters the current period's funds window in each clearing cycle, and send the stripped standardized clearing messages into the accounting preparation area.

[0019] The standard accounting conversion module is used to perform standard accounting on the standardized clearing messages in the accounting preparation area, converting them into accounting records and batch summary records on the clearing account book;

[0020] The processing result output module is used to output the interbank fund clearing processing result based on the completed accounting records and the batch summary records.

[0021] The beneficial effects of this invention include: by introducing routing delay as a core parameter for fund release rearrangement, it changes the traditional clearing logic that solely relies on queue scheduling based on time or amount. This invention effectively identifies and measures the queuing misalignment phenomenon caused by differences in processing time between the front-end server, communication gateway, and signature verification equipment, resulting in some messages repeatedly missing the clearing matching cycle. By generating a current period's fund window and implementing delay-free sorting, it proactively prioritizes the release of withdrawal requests significantly affected by system link latency, alleviating the short-term delays and position occupancy issues that still occur in clearing accounts even when net clearing conditions are met, thereby improving the availability of funds within a single cycle in the interbank fund clearing system. Attached Figure Description

[0022] Figure 1 This is a flowchart of a cross-bank fund clearing and processing method based on big data according to the present invention. Detailed Implementation

[0023] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0024] Example 1: As Figure 1 As shown, a big data-based interbank funds clearing method is applied to an interbank funds clearing system that includes a front-end server, communication gateway, signature verification equipment group, regional processing entry point, central queue entry point, and clearing account book, comprising:

[0025] Extract the clearing account number carried in the standardized clearing message entering the interbank fund clearing system, establish routing features representing the physical transmission path for the standardized clearing message, and allocate the standardized clearing message to the account index of the clearing account corresponding to the clearing account number;

[0026] Based on the account index, the standardized clearing message is divided into direction queues according to the fund direction, the routing feature, and the clearing cycle number. The direction queue with the fund direction being debit is divided into the outflow queue, and the direction queue with the fund direction being credit is divided into the inflow queue.

[0027] To guide the rearrangement of fund releases in order to proactively mitigate dwell interference caused by physical entry structures, the routing dwell degree, which characterizes the degree of structural delay, is calculated for the standardized clearing messages within each of the said directional queues.

[0028] Based on the current available position of the clearing account and the routing stagnation, a current funding window is generated to constrain the funding release boundary;

[0029] Within each clearing cycle, the withdrawal queue entering the current period's funds window is sorted to remove lingering issues, and the stripped standardized clearing messages are sent to the accounting preparation area.

[0030] Standard accounting is performed on the standardized clearing messages in the accounting preparation area, converting them into accounting records and batch summary records on the clearing account book;

[0031] Based on the completed accounting records and batch summary records, the interbank fund clearing processing results are output.

[0032] S201 records the time of the transaction when a standardized clearing message enters the front-end server.

[0033] The trigger condition for a service occurrence is the moment when the standardized clearing message is parsed by the application layer of the front-end machine, not the moment the message arrives at the network interface or is received by the receive buffer. The service occurrence time is stored in Unix timestamp format with millisecond precision, using the UTC time standard. All nodes in the system uniformly use the NTPv4 protocol for time synchronization, with a synchronization frequency of once every 5 minutes, requiring that the time synchronization error between nodes does not exceed 1 millisecond.

[0034] The unit of routing latency is milliseconds, and it is consistent with the timestamp precision of the business occurrence time, signature verification completion time, and central queue entry time. All time-related calculations are based on millisecond-level timestamps.

[0035] After being processed by the signature verification equipment group, S202 records the signature verification completion time and writes the signature verification equipment group identifier.

[0036] A signature verification device group refers to a single, independent signature verification hardware processing device, not a cluster concept. Each device has independent processing capacity and a queuing queue. The trigger condition for signature verification completion time is the moment the front-end node receives the processing result returned by the signature verification device. The signature verification device group identifier is a 16-bit unsigned integer, globally unique throughout the entire clearing system, used to uniquely identify each signature verification device.

[0037] S203 writes the central queue entry time when the standardized clearing message arrives at the central queue entry.

[0038] The trigger condition for the central queue entry time is the moment when a standardized clearing message is successfully inserted into the doubly linked list data structure of the central queue, not the moment the message arrives at the central queue network interface. The central queue entry time is also stored in Unix timestamp format with millisecond precision, maintaining the same format as the business occurrence time and the signature verification completion time.

[0039] S204 compresses the paths of the corresponding front-end machine number, gateway number, signature verification device group identifier, area entry identifier, and central queue entry identifier into a routing identifier, which serves as the routing feature.

[0040] The front-end machine ID is a 16-bit unsigned integer, globally unique; the gateway ID is a 16-bit unsigned integer, globally unique; the area entry identifier is an 8-bit unsigned integer, globally unique; and the central queue entry identifier is an 8-bit unsigned integer, globally unique. The input order of these five identifiers must strictly follow the sequence: front-end machine ID, gateway ID, signature verification device group identifier, area entry identifier, and central queue entry identifier; this order cannot be changed. The SHA-256 hash algorithm is used to hash the concatenated identifier byte stream. The first 16 bytes of the hash result are used as the routing identifier, converted to a 64-bit hexadecimal string, and globally unique within the entire clearing system. To address hash collisions, a chaining method is used. When identical hash values ​​appear, an incrementing sequence number suffix is ​​added to the end of the routing identifier to ensure its uniqueness.

[0041] S205 assigns standardized clearing messages with different routing identifiers to the account index of the corresponding clearing account according to the clearing account number.

[0042] The clearing account number is a 64-bit long integer, globally unique. The account index is implemented using a ConcurrentHashMap data structure in Java, with the clearing account number as the key and an ArrayList storing standardized clearing message objects as the value. A segmented locking mechanism ensures thread safety in high-concurrency scenarios. Message allocation is performed immediately after the route identifier is generated. During the allocation process, a write lock is acquired on the corresponding account index entry to prevent data inconsistency caused by concurrent modifications.

[0043] S301 reads the account index and locates the corresponding clearing account by clearing account number.

[0044] At the start of each settlement cycle, an incremental read approach is used to retrieve the account index, reading only the standardized settlement messages newly added within the previous settlement cycle to avoid the performance overhead of a full read. During the read process, the `get` method of a `ConcurrentHashMap` is used to quickly locate the corresponding message list based on the settlement account number. Read locks are acquired during the read process to ensure data consistency.

[0045] S302 includes debit standardized clearing messages in the disbursement queue and credit standardized clearing messages in the inflow queue according to the direction of funds.

[0046] The funds direction field in the standardized clearing message is a single-digit integer. A value of 1 indicates a debit, meaning funds flow out of the clearing account, while a value of 2 indicates a credit, meaning funds flow into the clearing account. Both the withdrawal queue and the deposit queue are implemented using a LinkedList data structure, supporting insertion and deletion operations with O(1) time complexity, and also supporting subsequent reordering operations. The queue's lifecycle is the current clearing cycle; it is created at the beginning of the clearing cycle and automatically destroyed at the end of the cycle, releasing the occupied memory resources.

[0047] S303 extracts the absolute value of the amount in the standardized clearing message.

[0048] The settlement amount field in the standardized settlement message is a 64-bit long integer, in cents, with a value range of -999999999999 to 9999999999999. The absolute value of the amount is obtained by performing a Math.abs operation on the settlement amount field, resulting in a non-negative long integer with a precision of cents, retaining two decimal places.

[0049] S304 uses the clearing cycle number as a cross-section to group consecutive standardized clearing messages under the same routing identifier into a queue, thus obtaining a directional queue.

[0050] The settlement cycle number is a 10-digit string, generated by adding a 3-digit cycle number to the settlement date in the format yyyyMMdd. For example, 20240530001 represents the first settlement cycle starting on May 30, 2024. The start and end times of the settlement cycle are on the hour. For example, when the settlement cycle length is 5 seconds, 00:00:00.000 to 00:00:04.999 is the first cycle, and 00:00:05.000 to 00:00:09.999 is the second cycle. Cycle switching uses a soft handover mechanism; the process switches to the next cycle only after the previous cycle's packets are processed. Continuous standardized settlement packets refer to packets whose service occurrence times are consecutive, and the time interval between two adjacent packets does not exceed 300 seconds. Packets meeting this condition under the same route identifier are grouped into the same directional queue.

[0051] S305 retains an amount sequence consisting of absolute amounts in each direction queue, as well as four internal sequences: queue entry sequence, signature verification completion sequence, and transaction occurrence sequence.

[0052] All four internal sequences use ArrayList. <long>The data structure is implemented such that the indexes of the sequences correspond one-to-one with the indices of the messages in the direction queue. The amount sequence stores the absolute value of the amount of all messages in the direction queue; the queue entry sequence stores the entry time of all messages into the central queue; the signature verification completion sequence stores the signature verification completion time of all messages; and the business occurrence sequence stores the business occurrence time of all messages. The sequences are sorted according to the order in which the messages were assigned to the direction queue, ensuring an accurate correspondence between elements within the sequence and the messages.

[0053] In order to eliminate the system time consumption interference caused by the signature verification process, S401 uses the difference between the entry time of the central queue and the completion time of signature verification to extract the routing delay that represents the actual transmission time of the physical network.

[0054] The formula for calculating routing latency is:

[0055] ;

[0056] in, The routing delay for the i-th standardized clearing message is... The entry time of the central queue for the i-th standardized clearing message. The signature verification completion time for the i-th standardized clearing message;

[0057] In order to correct the periodic misalignment caused by the physical ingress structure, S402 extracts the deviation characteristics of the routing delay from the preset reference delay corresponding to the routing identifier, and determines the periodic deviation that characterizes the missed release phase of the message by combining the preset clearing cycle length.

[0058] The formula for calculating period deviation is:

[0059] ;

[0060] in, For the periodic deviation of the i-th standardized clearing message, The routing identifier for the i-th standardized clearing message

[0061] The statistical period to which the i-th standardized clearing message belongs;

[0062] The statistical period is divided into whole-hour segments, with a length of 1 hour. The statistical period number is a string in the format yyyyMMddHH. For route identifier During the statistical period The corresponding preset baseline latency; the preset baseline latency is calculated by taking all route latency samples under this route identifier within the statistical period, excluding outliers that deviate from the mean 3σ, and then calculating the arithmetic mean of the remaining samples; the preset baseline latency is updated once per hour, and the default baseline latency of newly opened routes adopts the average baseline latency of routes of the same type in the same area; The preset settlement cycle length; This represents the mathematical modulo operation, ensuring that the result is a non-negative number; the period deviation ranges from 0 to 1.

[0063] The outlier exclusion rule for the preset baseline delay is to calculate the arithmetic mean and standard deviation of all routing delay samples within the statistical period, mark samples that are greater than the mean plus 3 times the standard deviation or less than the mean minus 3 times the standard deviation as outliers and exclude them, and use only the remaining valid samples to calculate the final preset baseline delay.

[0064] In order to quantify the queuing impact caused by congestion in the signature verification link, S403 determines the retention coefficient, which characterizes the degree of structural delay of a single message, by comprehensively considering the congestion distribution status and periodic deviation of the real-time queuing number of the corresponding signature verification equipment group compared with the baseline queuing number.

[0065] The formula for calculating the retention coefficient is:

[0066] ;

[0067] in, Let be the retention coefficient of the i-th standardized clearing message. For the signature verification equipment group corresponding to the i-th standardized clearing message, For the corresponding signature verification equipment group During the statistical period The number of real-time queues within;

[0068] The real-time queue count is obtained via the internal API of the signature verification device and is updated every 100 milliseconds. For the corresponding signature verification equipment group During the statistical period The baseline queue number is the arithmetic mean of the real-time queue numbers within the statistical period, and it is updated hourly. This is the zero-prevention constant for this formula, and its value is [value missing]. This is used to prevent calculation errors when the denominator is zero.

[0069] The natural logarithm operation is used, and the upper limit of the retention coefficient is set to 10. When the calculation result exceeds 10, 10 is taken as the final retention coefficient to prevent the coefficient from being too large due to abnormal congestion. The value of the retention coefficient is greater than or equal to 1.

[0070] In order to guide the rearrangement of fund release to actively resolve congestion, S404 uses the absolute value of the amount as the influence weight to merge each congestion coefficient, and under the condition of introducing a zero-prevention constant to prevent calculation anomalies, it obtains the routing congestion degree, which characterizes the degree of influence of the physical entry point on the overall queue.

[0071] The formula for calculating route dwell time is:

[0072]

[0073] in, The routing latency of the direction queue. For the corresponding clearing account, For funding direction, For routing identifier, This is the liquidation cycle number;

[0074] The unique identifier of the direction queue is generated by concatenating the clearing account number, fund direction, routing identifier, and clearing cycle number. The set of messages in the direction queue. Let be the absolute value of the amount in the i-th standardized clearing message. This represents the summation of all standardized clearing messages in the destination queue. This is the zero-prevention constant for this formula, and its value is [value missing]. The unit is cents, used to prevent calculation errors where the denominator is zero;

[0075] S501 reads the current balance in the clearing account book and deducts the minimum balance control value, the amount to be deposited, and the intraday available financing amount to obtain the basic movable position.

[0076] The current balance is the real-time updated account fund balance in the clearing account book, in cents. The minimum balance control value is the minimum amount of funds that a clearing account must retain, as uniformly stipulated by regulatory authorities, and is updated daily. The pre-deposit amount is frozen funds pending clearing, including only frozen funds that have been submitted but not yet cleared; the pre-deposit period is 24 hours, after which it will be automatically unfrozen. The intraday available financing limit is a temporary financing limit dynamically adjusted based on the clearing account's credit rating, updated daily. The basic movable position is calculated once at the beginning of each clearing cycle and updated in real time.

[0077] S502 reads the routing delays for each inflow queue and each outflow queue of the same clearing account.

[0078] The routing latency is stored in the member variables of the directional queue object. When reading, the member variables of the directional queue can be accessed directly without accessing the database, thus ensuring real-time reading.

[0079] S503 marks the withdrawal queues whose routing delays have reached the preset delay threshold as priority release groups, and marks the deposit queues that have entered the central queue entry as depositable groups.

[0080] The default value for the preset retention threshold is 0.7. During peak business hours (9:00-12:00 and 14:00-17:00 daily), it is adjusted to 0.6, and during off-peak business hours (0:00-8:00 and 18:00-24:00 daily), it is adjusted to 0.8, achieving dynamic adjustment. The criterion for a message entering the central queue is that it has been successfully inserted into the central queue and the entry time has been recorded. The validity period of the markers for the priority release group and the eligible settlement group is the current settlement cycle; they are automatically cleared after the cycle ends.

[0081] S504, within the same clearing cycle, combines the basic available position to form a three-segment funding window, including the withdrawal limit, the inflow range, and the reserved balance, which serves as the current funding window.

[0082] The formula for calculating the withdrawal limit is:

[0083] ;

[0084] in, This is the maximum withdrawal limit. Based on movable positions The set of deposit queues for eligible groups. This represents the total amount in the eligible deposit queue, with 0.9 being a risk discount factor used to mitigate the risk of potential deposit failures.

[0085] The lower limit of the deposit range is 0, and the upper limit is the total amount in the deposit queue of the eligible deposit group. The reserved balance is equal to the minimum balance control value. The execution order of the funds window is to process deposit operations first, and then process withdrawal operations, using a two-stage submission mechanism. When a deposit operation fails, the already processed withdrawal operation is automatically rolled back to ensure fund security.

[0086] The risk discount factor of 0.9 is the system default value and can be adjusted through the configuration file. The adjustment range is from 0.8 to 1.0. The lower the value, the lower the tolerance for the risk of deposit failure and the larger the reserved safety buffer.

[0087] S601 is organized into a primary sequence based on the priority of regulatory business.

[0088] The regulatory business priority field in the standardized clearing message is a single-digit integer, ranging from 1 to 5, where 1 represents the highest priority and 5 represents the lowest priority. Specifically, the priority classification is as follows: 1 corresponds to emergency disaster relief funds, 2 to large-amount real-time payments, 3 to ordinary commercial payments, 4 to small-amount batch payments, and 5 to batch disbursement and payment services. The priority classification table can be modified via configuration file to meet different regulatory requirements. The primary sequence is the initial sequence formed by arranging all pending disbursements according to regulatory business priority from highest to lowest.

[0089] S602 sorts the routing delay values ​​of the direction queues in descending order within the same priority.

[0090] For withdrawal queues with the same regulatory business priority, the TimSort sorting algorithm is used to sort them from high to low according to the routing delay value. The TimSort algorithm is a stable sorting algorithm that can ensure the efficiency and stability of sorting in high-concurrency scenarios.

[0091] S603 When the routing delay values ​​of the two directional queues are the same, they are arranged according to the order of entry time of the central queue.

[0092] For withdrawal queues with the same routing delay value, they are arranged in ascending order of entry time from the center queue of the directional queue. When the entry time of the center queue is also exactly the same, they are arranged in ascending lexicographical order of the unique identifier of the directional queue to ensure the uniqueness of the sorting result.

[0093] When multiple standardized clearing messages exist in the same direction queue, S604 will strip them in a fixed order: the business occurrence sequence, the signature verification completion sequence, and the central queue entry sequence.

[0094] For multiple messages within the same directional queue, they are strictly sorted according to priority. First, they are sorted from earliest to latest according to the time of the transaction occurrence sequence. When the transaction occurrence times are the same, they are sorted from earliest to latest according to the time of the signature verification completion sequence. When the signature verification completion times are also the same, they are sorted from earliest to latest according to the time of entry into the central queue. When all three times are exactly the same, they are sorted in ascending lexicographical order according to the payment instruction number. Messages are then stripped and processed for fund release according to this sorting result.

[0095] For each standardized clearing message that is stripped from the S605, the amount is deducted from the current funds window of the corresponding clearing account, a corresponding release number is generated, and the standardized clearing message carrying the release number is written into the accounting preparation area.

[0096] The release sequence number is a unique, incrementing integer within each clearing cycle, starting from 1. The accounting preparation area is implemented using a ConcurrentLinkedQueue data structure, with each clearing account corresponding to an independent accounting preparation area, ensuring thread safety. The maximum capacity of the accounting preparation area is 10,000 messages. When the capacity is full, the processing thread enters a waiting state until free capacity becomes available. For each message stripped, the clearing amount for that message is deducted from the current period's cash window's withdrawal limit. When a new eligible deposit message arrives, the current period's cash window's withdrawal limit is dynamically updated.

[0097] The maximum capacity of the accounting preparation area is 10,000 records, which is the system default value. It can be dynamically adjusted according to the server hardware configuration and business volume, with an adjustment range of 1,000 to 100,000 records. When the capacity reaches the limit, new message stripping operations will be blocked until a processed message is removed from the accounting preparation area.

[0098] S701 parses the standardized clearing messages in the accounting preparation area and extracts the clearing date, payment instruction number, paying clearing bank, receiving clearing bank, paying clearing account, receiving clearing account, clearing amount, and debit / credit direction.

[0099] The standardized clearing message adopts the ISO8583 format. The position and length of each field are as follows: Clearing Date (Field 3, 8 bits), Payment Instruction Number (Field 11, 19 bits), Payment Clearing Action (Field 32, 12 bits), Receipt Clearing Action (Field 33, 12 bits), Payment Clearing Account (Field 4, 19 bits), Receipt Clearing Account (Field 5, 19 bits), Clearing Amount (Field 49, 12 bits), and Debit / Credit Direction (Field 50, 1 bit). The parsing process strictly follows the field definitions to ensure the accuracy of the parsing results.

[0100] S702 reads the standardized clearing messages in the accounting preparation area one by one according to the release sequence number.

[0101] A batch reading approach is adopted, processing 100 messages at a time before moving on to the next batch, thereby improving system throughput. The reading order strictly follows the release sequence number from smallest to largest to ensure that the accounting order matches the fund release order.

[0102] The system default value for batch reading messages is 100 messages. It can be adjusted via configuration file, ranging from 10 to 1000 messages. The batch size setting needs to balance system throughput and processing latency. The larger the batch, the higher the throughput, but the average processing latency of a single message will also increase accordingly.

[0103] S703 generates debit entries for payment clearing accounts and credit entries for collection clearing accounts.

[0104] The accounting subjects uniformly adopt the 10101 subject stipulated by the clearing system. The debit entry for the payment clearing account is an increase on the debit side to record the outflow of funds, and the credit entry for the collection clearing account is an increase on the credit side to record the inflow of funds.

[0105] The S704 synchronously writes the settlement date, payment instruction number, settlement amount, lending direction, settlement cycle number, routing identifier, posting time, and release sequence number to complete the posting record.

[0106] The posting time is a Unix timestamp indicating the system executed the posting operation, with millisecond precision. Posting records are stored in an InnoDB table in MySQL. The table structure includes an `id` primary key field, as well as all the fields mentioned above, and supports transactions and indexes. Distributed transactions employ a two-phase commit mechanism: the `prepare` phase locks the payment and receipt accounts, and the `commit` phase updates the balances of both accounts simultaneously. Automatic rollback occurs on failure, ensuring data atomicity.

[0107] S705 summarizes the debit, credit, and net amounts for each cost cycle by the paying and receiving clearing banks, completing the batch summary record.

[0108] The system employs a real-time accounting and aggregation approach. After each message is recorded, the corresponding clearing bank's debit, credit, and net amounts are updated, eliminating the need for a full iteration at the end of the cycle. Batch aggregation records are stored in a summary table in MySQL, with one record per clearing bank per clearing cycle.

[0109] S706 rewrites the processing status of the standardized clearing message to the clearing status of the current cycle and generates a corresponding receipt containing the clearing receipt number.

[0110] The processing status field has values ​​of 0 (pending processing), 1 (cleared in this period), and 2 (processing failed). The clearing receipt number is a globally unique 32-bit hexadecimal string generated based on the payment instruction number and clearing date. The receipt is in XML format and sent to the corresponding clearing bank via HTTPS. If sending fails, it automatically retryes 3 times, with a 1-second interval between each retry. After each failed retry, it is stored in a retry queue, and a background thread retryes every 5 minutes.

[0111] The retry parameters for sending clearing receipts can be adjusted via configuration file. The default number of immediate retries is 3, with a 1-second interval between each retrieval. The scanning interval of the background failure retry queue is 5 minutes, and the maximum number of retries is 10. Receipts that fail to be sent after more than 10 retries will be permanently stored in the failure log and a manual alarm notification will be triggered.

[0112] S801 extracts the following fields from accounting records, batch summary records, and corresponding receipts: settlement date, settlement cycle number, payment instruction number, paying clearing bank, receiving clearing bank, paying clearing account, receiving clearing account, settlement amount, debit / credit direction, accounting time, processing status, and settlement receipt number.

[0113] The extraction order strictly follows the order of the fields mentioned above. The data type conversion rules are as follows: time types are converted to strings in the format yyyy-MM-ddHH:mm:ss.SSS, amount types are converted to strings with two decimal places, and number types are converted to strings to ensure compatibility between different systems.

[0114] S802 generates a transaction-by-transaction clearing result file based on the clearing cycle number, and simultaneously generates a summary file of loan and borrowing amounts at the clearing bank level, which is then output as the cross-bank fund clearing result.

[0115] The results of each transaction clearing process are in JSON format, encoded in UTF-8, with LF line breaks, and named as `clearing_result_clearingcycle_number.json`. The summary file of loan and borrowing amounts at the clearing bank level is in CSV format, encoded in UTF-8, with LF line breaks, and named as `clearing_summary_clearingcycle_number.csv`. Output is performed immediately after the end of each clearing cycle and transferred to the designated server directory of each clearing bank via SFTP. Files are also backed up to the HDFS distributed file system, with a 90-day backup retention period to ensure data security and traceability.

[0116] The system's modules communicate using the gRPC protocol, with interface definitions strictly adhering to the Protobuf specification. The interface between the front-end module and the signature verification module takes a standardized settlement message as input and outputs a processed message and timestamp. The interface between the signature verification module and the central queue module takes a message with a routing identifier as input and outputs the queue insertion result. The complete data flow is as follows: a message is sent from the participating bank to the front-end, recording the transaction time; it is forwarded through the gateway to the signature verification device, recording the signature verification completion time; then it is sent to the central queue, recording the central queue entry time; a routing identifier is generated; it is assigned to the account index; the direction queue is split; the routing retention rate is calculated; a current period funds window is generated; a retention sort is performed; the message is stripped to the accounting preparation area; standard accounting is performed; batch summary records are generated; and finally, the settlement result file is output.

[0117] Regarding error handling, when message parsing fails, error code 9001 is returned, written to the error log, and the sender is notified to retransmit. When accounting fails, the distributed transaction is rolled back, the message processing status is marked as failed, and a retry mechanism is triggered. In the event of a system crash, after restarting, data in memory is restored via Write-Ahead Log (WAL), and unprocessed messages are replayed to ensure no data loss. For high-concurrency processing, a load balancing mechanism is employed to evenly distribute messages to different processing nodes. A flow control mechanism limits the number of messages processed per second to the system's maximum processing capacity. An overload protection mechanism rejects new connections when the system load exceeds a threshold to prevent system crashes.

[0118] The HDFS distributed file system is backed up hourly, and the locally generated liquidation result files are synchronously uploaded to the HDFS cluster. At the same time, an off-site backup mechanism is used to synchronize the backup data to an off-site backup center more than 500 kilometers away from the main data center. The backup data is retained for 90 days, and after that, it is automatically archived to the offline storage system. The offline storage data is retained for 5 years.

[0119] It should be noted that the system adopts a distributed microservice architecture. The account index configuration module, direction queue splitting module, route retention calculation module, fund window generation module, retention release module, standard accounting conversion module, and processing result output module are all deployed as independent microservice instances. Service discovery and load balancing between microservice instances are achieved through a registry center, and horizontal scaling is supported to cope with the growth of business volume.

[0120] It should be noted that the system logs adopt a hierarchical recording mechanism, divided into four levels: DEBUG, INFO, WARN, and ERROR. The production environment records INFO and higher level logs by default. The log content includes timestamp, module name, thread ID, log level, and detailed log information. The log files are split by day, with a separate log file generated each day. The local log retention period is 30 days, and the logs are also synchronized to the ELK log analysis platform for centralized storage and analysis. The platform logs are retained for 90 days.

[0121] It should be noted that the Write-Ahead Log (WAL) is written to the WAL file every time a fund release operation and an accounting operation is performed. Detailed operation information is synchronously written to the WAL file. The WAL file is appended, and each WAL file is 1GB in size. A new WAL file is automatically generated when the file is full. The system recovery process after a crash is as follows: Upon restarting, the latest WAL file is read first. All unfinished transactions are replayed in the order of operations. The account indexes, direction queues, accounting preparation areas, and clearing account books in memory are restored to their pre-crash state. Then, unfinished clearing tasks are processed.

[0122] It should be noted that the gRPC interfaces between the various modules of the system are defined using Protobufv3. Taking the interface between the account index configuration module and the directional queue splitting module as an example, the Protobuf message structure is defined as follows:

[0123] messageClearingMessage{

[0124] int64clearing_account_number=1;

[0125] stringrouting_identifier=2;

[0126] int64business_occurrence_time=3;

[0127] int64signature_verification_completion_time=4;

[0128] int64central_queue_entry_time=5;

[0129] int64amount_absolute_value=6;

[0130] int32fund_direction=7;

[0131] stringclearing_cycle_number=8;

[0132] int32regulatory_business_priority=9;

[0133] stringpayment_instruction_number=10;

[0134] }

[0135] messageDirectionQueueRequest{

[0136] repeatedClearingMessagemessages=1;

[0137] }

[0138] messageDirectionQueueResponse{

[0139] int32code=1;

[0140] stringmessage=2;

[0141] }

[0142] serviceDirectionQueueService{

[0143] rpcSplitDirectionQueue(DirectionQueueRequest)returns(DirectionQueueResponse);

[0144] }

[0145] It should be noted that the core performance requirements of the system are: a single node can process no less than 10,000 clearing messages per second, the average clearing latency is no more than 50 milliseconds, the processing latency of 99.9% of clearing messages is no more than 200 milliseconds, the system availability is no less than 99.99%, and the unplanned downtime per year is no more than 52.56 minutes.

[0146] Example 2: A cross-bank fund clearing and processing system based on big data, comprising:

[0147] The account index configuration module is used to extract the clearing account number carried in the standardized clearing message entering the interbank fund clearing network, establish routing features representing the physical transmission path of the standardized clearing message, and allocate the standardized clearing message to the account index of the clearing account corresponding to the clearing account number.

[0148] The direction queue segmentation module is used to segment the standardized clearing message into direction queues based on the account index, according to the fund direction, the routing feature, and the clearing cycle number. The direction queues with the fund direction being debit are divided into outgoing queues, and the direction queues with the fund direction being credit are divided into incoming queues.

[0149] The routing retention calculation module is used to calculate the routing retention degree, which represents the degree of structural delay, of the standardized clearing messages in each of the direction queues in order to guide the rearrangement of fund release and actively resolve the retention interference caused by the physical entry structure.

[0150] The funds window generation module is used to generate a current funds window that constrains the funds release boundary based on the current available position of the clearing account and the routing retention rate.

[0151] The de-delay release module is used to perform de-delay sorting on the withdrawal queue that enters the current period's funds window in each clearing cycle, and send the stripped standardized clearing messages into the accounting preparation area.

[0152] The standard accounting conversion module is used to perform standard accounting on the standardized clearing messages in the accounting preparation area, converting them into accounting records and batch summary records on the clearing account book;

[0153] The processing result output module is used to output the interbank fund clearing processing result based on the completed accounting records and the batch summary records.

[0154] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.< / long>

Claims

1. A big data-based cross-bank fund clearing processing method applied to a cross-bank fund clearing system comprising a front-end machine, a communication gateway, a group of signing and verifying devices, a regional processing portal, a central queue portal and a clearing account book, characterized in that, include: Extract the clearing account number carried in the standardized clearing message entering the interbank fund clearing system, establish routing features representing the physical transmission path of the standardized clearing message, and allocate the standardized clearing message to the account index of the clearing account corresponding to the clearing account number; Based on the account index, the standardized clearing message is divided into direction queues according to the fund direction, the routing feature, and the clearing cycle number. The direction queues with the fund direction being debit are divided into outflow queues, and the direction queues with the fund direction being credit are divided into inflow queues. To guide fund release rearrangement to proactively mitigate dwell interference caused by physical entry structure, the routing dwell degree, which characterizes the degree of structural delay, is calculated for the standardized clearing messages within each of the said directional queues. Based on the current available position of the clearing account and the routing stagnation, a current funding window is generated to constrain the funding release boundary; Within each clearing cycle, the withdrawal queue entering the current period's funds window is sorted to remove lingering issues, and the stripped standardized clearing messages are sent to the accounting preparation area. Standard accounting is performed on the standardized clearing messages in the accounting preparation area, converting them into accounting records and batch summary records on the clearing account book; Based on the completed accounting records and batch summary records, the interbank fund clearing processing results are output.

2. The cross-bank funds clearing processing method based on big data according to claim 1, characterized in that, The step of establishing routing features characterizing the physical transmission path for the standardized clearing message and allocating the standardized clearing message to the account index of the clearing account corresponding to the clearing account number includes: Record the time of the transaction when the standardized clearing message enters the front-end server; After processing by the signature verification equipment group, the signature verification completion time is recorded and written into the signature verification equipment group identifier. Write the central queue entry time when the standardized clearing message arrives at the central queue entry; The paths of the corresponding front-end machine number, gateway number, signature verification device group identifier, area entry identifier, and central queue entry identifier are compressed into a routing identifier, which serves as the routing feature. The standardized clearing messages with different routing identifiers are assigned to the account index corresponding to the clearing account according to the clearing account number. 3.The cross-bank funds clearing processing method based on big data according to claim 2, characterized in that, The step of dividing the standardized clearing messages into directional queues based on the account index according to the fund direction, the routing characteristics, and the clearing cycle number includes: Read the account index and locate the corresponding clearing account by the clearing account number; According to the stated funding direction, the standardized clearing message for debiting is included in the disbursement queue, and the standardized clearing message for crediting is included in the inflow queue; Extract the absolute value of the amount from the standardized settlement message; Using the clearing cycle number as a cross-section, consecutive standardized clearing messages under the same routing identifier are grouped into a queue to obtain the directional queue; Each direction queue retains an amount sequence consisting of the absolute value of the amount, as well as four internal sequences: queue entry sequence, signature verification completion sequence, and business occurrence sequence.

4. The cross-bank funds clearing processing method based on big data according to claim 3, characterized in that, The calculation of the routing retention rate, which characterizes the degree of structural delay for the standardized clearing messages within each of the directional queues, includes: To eliminate system time consumption interference caused by signature verification processing, the difference between the entry time of the central queue and the completion time of signature verification is used to extract the routing delay that represents the actual transmission time of the physical network. To correct the periodic misalignment caused by the physical entry structure, the deviation characteristics of the routing delay from the preset reference delay corresponding to the routing identifier are extracted, and the periodic deviation characterizing the missed release phase of the message is determined by combining the preset clearing cycle length. In order to quantify the queuing impact caused by congestion in the signature verification link, the retention coefficient, which characterizes the degree of structural delay of a single message, is determined by combining the congestion distribution of the real-time queuing number of the signature verification device group relative to the baseline queuing number and the period deviation. To guide the rearrangement of fund release to proactively resolve congestion, the absolute value of the amount is used as the influence weight to fuse the congestion coefficients. Under the condition of introducing a zero-prevention constant to prevent calculation anomalies, the routing congestion degree, which characterizes the degree of influence of the physical entry point on the overall queue, is obtained.

5. The cross-bank funds clearing processing method based on big data according to claim 4, characterized in that, The step of generating a current funding window that constrains the funding release boundary based on the current available position of the clearing account and the routing stagnation includes: Read the current balance in the clearing account book, and deduct the minimum balance control value, the amount deposited, and the intraday available financing amount to obtain the basic movable position; Read the routing delays corresponding to each of the inflow queues and each of the outflow queues for the same clearing account; The withdrawal queues whose routing delay reaches a preset delay threshold are marked as priority release groups, and the deposit queues that have entered the central queue entry are marked as depositable groups. Within the same clearing cycle, the basic movable position is combined to form a three-segment funding window, including the withdrawal limit, the inflow range, and the reserved balance, which serves as the current funding window.

6. The cross-bank funds clearing processing method based on big data according to claim 5, characterized in that, Within each clearing cycle, the process of performing a de-de-listing sort on the disbursement queue entering the current period's funds window, and sending the stripped standardized clearing messages to the accounting preparation area, includes: A primary sequence is formed based on the priority of regulatory business; Within the same priority, the routes are sorted in descending order of their routing latency values. When the routing latency values ​​of the two directional queues are the same, they are arranged according to the order in which the central queue entered. When multiple standardized clearing messages exist in the same direction queue, they are stripped in a fixed order: the business occurrence sequence, the signature verification completion sequence, and the central queue entry sequence. For each standardized clearing message stripped, its amount is deducted from the current funds window of the corresponding clearing account, a corresponding release sequence number is generated, and the standardized clearing message carrying the release sequence number is written into the accounting preparation area. 7.The cross-bank funds clearing processing method based on big data according to claim 6, characterized in that, The standard accounting process for the standardized clearing messages in the accounting preparation area, which is then converted into accounting records and batch summary records on the clearing account book, includes: Parse the standardized clearing message in the accounting preparation area to extract the clearing date, payment instruction number, paying clearing bank, receiving clearing bank, paying clearing account, receiving clearing account, clearing amount, and debit / credit direction; Read the standardized settlement messages in the accounting preparation area one by one according to the release sequence number; A debit entry is generated for the payment clearing account, and a credit entry is generated for the collection clearing account; The settlement date, payment instruction number, settlement amount, lending direction, settlement cycle number, routing identifier, accounting time, and release sequence number are written synchronously to complete the accounting record; The batch summary record is completed by summing the debit, credit, and net amounts of the payment clearing bank and the receiving clearing bank for the cost cycle. The processing status of the standardized clearing message is rewritten to the clearing status of the current cycle, and a corresponding receipt containing the clearing receipt number is generated. 8.The cross-bank funds clearing processing method based on big data according to claim 7, characterized in that, The output of interbank fund clearing results based on the completed accounting records and batch summary records includes: Extract the following fields from the accounting records, batch summary records, and corresponding receipts: settlement date, settlement cycle number, payment instruction number, paying clearing bank, receiving clearing bank, paying clearing account, receiving clearing account, settlement amount, debit / credit direction, accounting time, processing status, and settlement receipt number. Generate a transaction-by-transaction clearing result file according to the clearing cycle number, and simultaneously generate a summary file of loan and borrowing amounts at the clearing bank level, which is then output as the cross-bank fund clearing result. 9.A big data-based cross-bank funds clearing processing system, characterized by, include: The account index configuration module is used to extract the clearing account number carried in the standardized clearing message entering the interbank fund clearing network, establish routing features representing the physical transmission path of the standardized clearing message, and allocate the standardized clearing message to the account index of the clearing account corresponding to the clearing account number. The direction queue segmentation module is used to segment the standardized clearing message into direction queues based on the account index, according to the fund direction, the routing feature, and the clearing cycle number. The direction queues with the fund direction being debit are divided into outgoing queues, and the direction queues with the fund direction being credit are divided into incoming queues. The routing retention calculation module is used to calculate the routing retention degree, which represents the degree of structural delay, of the standardized clearing messages in each of the direction queues in order to guide the rearrangement of fund release and actively resolve the retention interference caused by the physical entry structure. The funds window generation module is used to generate a current funds window that constrains the funds release boundary based on the current available position of the clearing account and the routing retention rate. The de-delay release module is used to perform de-delay sorting on the withdrawal queue that enters the current period's funds window in each clearing cycle, and send the stripped standardized clearing messages into the accounting preparation area. The standard accounting conversion module is used to perform standard accounting on the standardized clearing messages in the accounting preparation area, converting them into accounting records and batch summary records on the clearing account book; The processing result output module is used to output the interbank fund clearing processing result based on the completed accounting records and the batch summary records.

Images