A distributed counting system and a counting method

Abstract

The application discloses a distributed counting system and a counting method. The system comprises a plurality of distributed service nodes, a log node and a counting device. Each distributed service node is configured to write the latest value of a sub-counter into a log file on the log node when a preset sub-counter value writing condition is met. The counting device is configured to obtain the sub-counter values of the distributed service nodes from the log file respectively, and determine the total counter value of the distributed system according to the sub-counter values. The application can effectively overcome the risk of instability of the overall system caused by the multi-node shared counter scheme in the multi-distributed system, and ensure the overall high availability of the distributed system under the premise of meeting the counting requirement.

Description

Technical Field

[0001] This invention relates to the field of distributed processing technology, and more specifically to a distributed counting system and counting method. Background Technology

[0002] Counters are widely used in many application systems, including distributed systems, to trigger related subsequent business logic. In distributed systems, because each node executes business functions in a distributed manner, different nodes can affect a particular counter result. Common solutions use shared counters to synchronize the counter values ​​across the entire distributed system. However, using shared counters necessitates introducing other service components into each application system to ensure reliable access or updates to the shared counter by multiple distributed nodes, thus increasing the overall system instability.

[0003] For distributed application systems with extremely high availability requirements, new solutions are needed to implement counter schemes in distributed systems in a more stable and reliable manner. Summary of the Invention

[0004] This disclosure provides a distributed counting system and method in which each distributed business node counts independently and writes the count to a log. The log extraction method is used to obtain the sub-counter values ​​of each node, and then the total count value of the overall distributed system is determined. This method can effectively overcome the risk of overall system instability caused by multi-node shared counter schemes.

[0005] This disclosure provides a distributed counting system, including: a plurality of distributed service nodes, log nodes, and a counting device;

[0006] Each of the distributed service nodes is configured to write the latest value of the counter to the log file on the log node when the preset counter value writing condition is met.

[0007] The counting device is configured to: obtain the sub-counter values ​​of each of the distributed business nodes from the log file; and determine the total count value of the preset counter in the distributed business system based on the sub-counter values.

[0008] This disclosure also provides a distributed counting method, including:

[0009] Each distributed service node writes the latest value of the counter to the log file on the log node when the preset conditions for writing the counter value are met.

[0010] The counting device obtains the sub-counter values ​​of each of the distributed business nodes from the log file; and determines the total count value of the preset counter in the distributed business system based on the sub-counter values.

[0011] After reading and understanding the accompanying diagrams and detailed descriptions, the other aspects can be understood. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of the structure of a distributed counting system according to an embodiment of the present disclosure;

[0014] Figure 2 This is a schematic diagram of the structure of another distributed counting system in an embodiment of this disclosure;

[0015] Figure 3 This is a flowchart of a distributed counting method according to an embodiment of the present disclosure.

[0016] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0018] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0019] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0020] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0021] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0022] This disclosure provides a distributed counting system, such as... Figure 1 As shown, it includes: several distributed business nodes 10, log nodes 20, and a counting device 30;

[0023] Each of the distributed service nodes 10 is configured to write the latest value of the counter to the log file on the log node 20 when the preset counter value writing condition is met.

[0024] The counting device 30 is configured to: obtain the sub-counter values ​​of each of the distributed business nodes 10 from the log file; and determine the total count value of the preset counter in the distributed business system based on the sub-counter values.

[0025] It should be noted that the distributed system includes one or more business nodes, and the application systems within these business nodes perform local counter counting operations. All operations on the local counter by the application system are memory operations, requiring no additional components. After counting, the application system to which the counter belongs writes the latest value of its counter to the log file on log node 20 in real time or periodically, according to a preset log specification. In this embodiment, the application system can be equivalently replaced by an application function module, and is not specifically limited to either an application system or an application module.

[0026] Each business node runs one or more application systems (or modules), each application system (or module) operates one or more minute counters, and the latest value of the one or more minute counters operated by each application system (or module) is written to a log file, or written to multiple log files separately.

[0027] In some exemplary embodiments, the counting device 30 obtains the sub-counter values ​​of each of the distributed service nodes from the log file, including: using a log extraction tool to obtain the sub-counter values ​​of each of the distributed service nodes from the log file.

[0028] For example, the log extraction tool is Flume software, that is, the counting device 30 uses Flume to collect the sub-counter values ​​from one or more log files on the log node.

[0029] The log extraction tool executes an extraction task to obtain one or more score counter values ​​when the extraction trigger condition is met.

[0030] In some exemplary embodiments, the extraction triggering condition includes at least one of the following: a preset event occurs; a preset extraction interval arrives. For example, if the preset extraction interval is 30 seconds, then every 30 seconds, a log extraction tool is used to extract all or incremental minute counter values ​​from the node's log file.

[0031] It should be noted that Flume is a highly available, highly reliable, distributed system for massive log collection, aggregation, and transmission. Flume supports customizing various data senders within the log system for data collection. Using Flume, based on the defined log files and log writing specifications, it can accurately collect target data. In the embodiments of this disclosure, Flume is used to collect the values ​​of one or more minute counters written in the log files of the collection nodes. After configuring the extraction task according to Flume's relevant configuration requirements, the collection of minute counter values ​​can be achieved. The specific configuration method is not limited or protected by this application. Optionally, other log extraction tools can be used to replace Flume to extract minute counter value information from log files; it is not limited to any specific extraction tool.

[0032] In some exemplary embodiments, the log node 20 may include one or more. Service nodes 10 may write log files to their respective corresponding log nodes 20 according to system configuration.

[0033] In some exemplary embodiments, the conditions for writing the counter value include one or more of the following:

[0034] The value of the minute counter changes;

[0035] The preset write interval has arrived;

[0036] Preset events occur on distributed business nodes.

[0037] For example, each business node writes the latest value of its in-memory counter to the log file of the log node every 2 seconds. For example, when a certain statistical statistic reaches a set quantity on a distributed business node, the latest count value is written, and so on. It should be noted that the conditions for writing counter values ​​to different business nodes, and even different applications within the same business node, can be the same or different, and the write intervals can be the same or different; it is not limited to a specific method.

[0038] In some exemplary embodiments, the preset counter includes one or more.

[0039] In some exemplary embodiments, the counting device 30 is further configured to determine the total count value of the preset counter in the overall distributed system based on the obtained sub-counter values ​​of the business nodes, according to the correspondence between the preset counter and multiple sub-counters.

[0040] For example, a distributed system includes three business nodes: node 1, node 2, and node 3. The application system running on node 1 operates on the first consumer counter's sub-counter 1; the application system running on node 2 operates on the first consumer counter's sub-counter 1 and also on the second consumer counter's sub-counter 2; the application system running on node 3 operates on the first consumer counter's sub-counter 1 and also on the second consumer counter's sub-counter 2. The entire distributed system has pre-set counters including the first consumer counter and the second consumer counter. Therefore, the value of the first consumer counter in the entire distributed system = the value of the first consumer counter's sub-counter 1 on node 1 + the value of the first consumer counter's sub-counter 1 on node 2 + the value of the first consumer counter's sub-counter 1 on node 3; the value of the second consumer counter in the entire distributed system = the value of the second consumer counter's sub-counter 2 on node 2 + the value of the second consumer counter's sub-counter 2 on node 3.

[0041] It should be noted that the relationship between the total count value of the preset counter in a distributed system and the count values ​​of the sub-counters on one or more business nodes is flexibly determined according to actual counting requirements, and is not limited to the specific method in the example above. Based on the above example, those skilled in the art can understand the corresponding determination method, which will not be elaborated upon here.

[0042] In some exemplary embodiments, the system further includes a search server 40; the search server 40 is configured to store score counter information and to return search results based on search criteria.

[0043] Accordingly, the counting device is further configured to: obtain search results from the search server 40 according to the search conditions corresponding to the preset counter, and determine the total count value of the preset counter in the distributed business system according to the search results.

[0044] In some exemplary embodiments, the sub-counter information includes at least: service node, counter identifier, and latest count value.

[0045] In some exemplary embodiments, the search server 40 is an Elastic Search (ES) server. That is, one or more counter values ​​obtained from log files are uploaded to the ES server for storage, so as to determine the total count value of a preset counter in the distributed system after retrieving search results from the ES server.

[0046] For example, the counter information obtained from 3 business nodes and saved to the ES server is as follows:

[0047] Node 1, sub-counter 1 of the first consumer counter, value 1; Node 2, sub-counter 1 of the first consumer counter, value 2, sub-counter 2 of the second consumer counter, value 3; Node 3, sub-counter 1 of the first consumer counter, value 4, sub-counter 2 of the second consumer counter, value 5.

[0048] If the search criteria are "the first consumer counter's sub-counter 1", then the search results obtained from the ES server are: Node 1, the first consumer counter's sub-counter 1, value 1; Node 2, the first consumer counter's sub-counter 1, value 2; Node 3, the first consumer counter's sub-counter 1, value 4. Therefore, based on the search results, the total count value of the first consumer counter is determined to be value 1 + value 2 + value 4.

[0049] It should be noted that the search server is not limited to an Elasticsearch server; other search servers may also be used. In some exemplary embodiments, a database server can also be used as a search server. Accordingly, the search criteria and search results are matched with the selected search server. The description of the search criteria and search results above is only for illustrative purposes and does not limit the scope of the related steps. The search criteria and search results matched by the search server are determined by the selected search server and are not within the scope of protection or limitation of this application. Those skilled in the art can implement the solutions proposed in the embodiments of this disclosure based on the relevant implementation schemes.

[0050] The distributed counting scheme provided in this disclosure can perform system-level counting in a timely, accurate, and stable manner, and further realize subsequent business functions based on the counting results. For example, the total count value of the preset counters in the distributed system determined by the counting device can trigger other business processes when set conditions are met. For example, in online transaction or online payment systems, the huge volume of online transactions inevitably leads to the use of a distributed system to utilize multiple nodes to execute related business logic in a distributed manner. Each node independently completes the assigned tasks, and the same type of business may be executed by multiple nodes in a distributed manner. It can be seen that the total count value of a certain type of business is affected by multiple distributed nodes. According to the counting method provided in this disclosure, the total count value of a certain type of business can be accurately and timely determined to accurately and timely trigger subsequent business processes. For example, business alarms can be triggered based on the total count value of unprocessed records in the overall distributed system; batch transaction summaries can be performed based on the total count value of the current batch of completed transactions in the overall distributed system. This disclosure provides a counting scheme in a distributed system, without limiting how the determined count value can be further used. Those skilled in the art can implement subsequent processes according to business needs.

[0051] As can be seen, the distributed counting system provided in this embodiment does not employ a method of multiple service nodes sharing a counter. This reduces conflicts or instability caused by distributed service nodes operating the shared counter, and also reduces the system overhead that may be caused by resolving access conflicts to the shared counter. Distributed service nodes maintain stable operation by operating their sub-counters locally and recording logs. Third-party log extraction tools are used to obtain the sub-counter values ​​of each service node, which are then aggregated to obtain the total count value of the target counter in the entire distributed system. According to the counting scheme provided in this embodiment, both the timeliness and accuracy of the overall technology of the preset counter in the distributed system are guaranteed, as well as the stability of the operation of each node, reducing the operational overhead of each node under the shared counter scheme. For distributed service systems with huge concurrency and extremely high availability requirements, the counting scheme provided in this embodiment has significant advantages.

[0052] In some exemplary embodiments, such as Figure 2 As shown, the distributed business system is an online payment system;

[0053] The plurality of distributed business nodes 10 include: a plurality of transaction closing business nodes 101 and a plurality of final state notification sending nodes 102;

[0054] Each of the transaction closing business nodes 101 is configured to: when the first counter value writing condition is met, use the cumulative number of closed transactions as the latest value of the closing transaction counter and write it to the log file on the log node;

[0055] Each of the final state notification sending nodes 102 is configured to: when the second sub-counter value writing condition is met, use the cumulative number of currently sent final state notifications as the latest value of the final state notification sub-counter and write it into the log file on the log node; the first sub-counter value writing condition and the second sub-counter value writing condition may be the same or different.

[0056] The counting device 30 is further configured to: obtain the latest value of each customs clearance transaction counter from the log file, and determine the total number of customs clearance transactions in the online payment system based on the latest value of each customs clearance transaction counter; obtain the latest value of each final status notification counter from the log file, and determine the total number of final status notifications sent by the online payment system based on the latest value of each final status notification counter; and determine the transaction processing status of the online payment system based on the total number of customs clearance transactions and the total number of final status notifications sent.

[0057] In some exemplary embodiments, each of the transaction closing order business nodes 101 is further configured to: when the final state notification delay switch is turned on, save the generated final state notifications to be sent corresponding to the closing order transaction to a delayed sending queue; when the number of final state notifications in the delayed sending queue reaches a preset cache quantity threshold, update the value of the closing order transaction counter using the preset cache quantity threshold, and save the final state notifications in the delayed sending queue to a storage device;

[0058] Each of the final state notification sending nodes 102 is further configured to: when the final state notification delay switch is off, acquire the final state notification in the delay sending queue and / or the storage device and send it out.

[0059] It should be noted that the final status notification delay switch in the online payment system is generally off, and the final status notification for a transaction can be sent immediately after its generation. In some business scenarios, such as during critical periods, delayed and rate-controlled sending is required. In this case, the final status notification delay switch is turned on, and each transaction closing business node 101 performs closing business processing and generates final status notifications to be sent, but these notifications need to be sent with a delay. In some exemplary embodiments, these final status notifications to be sent are submitted to a delayed sending queue for waiting to be sent; in some exemplary embodiments, when the accumulated messages in the queue reach a preset cache quantity threshold, these final status notifications to be sent are serialized and saved to disk. In some exemplary embodiments, the delayed sending queue includes one or more; that is, each transaction closing business node 101 corresponds to one delayed sending queue, or multiple transaction closing business nodes 101 jointly correspond to one or more delayed sending queues, not limited to a specific implementation method.

[0060] When the final state notification delay switch is turned off, each final state notification sending node 102 retrieves the final state notification to be sent from the delayed sending queue or disk and sends it.

[0061] During this process, each transaction closing business node 101 counts the transaction score counter in its own memory and writes the latest value to the log file of the log node when the write condition is met; each final state notification sending node 102 counts the final state notification score counter in its own memory and writes the latest value to the log file of the log node when the write condition is met. The counting device 30 can determine the transaction processing status of the entire online payment system in real time, such as the current sending status of final state notifications, that is, understand the backlog of final state notifications in the overall system or the sending speed. In some exemplary embodiments, final state notification sending nodes 102 can also be dynamically added according to the backlog situation to improve the sending speed.

[0062] In some exemplary embodiments, the writing conditions for each transaction closing business node 101 further include: when the number of final state notifications cached in the delayed sending queue is greater than or equal to a preset cache quantity threshold, the cumulative number of its own closed transactions is used as the latest value of the closing transaction counter and written to the log file on the log node. The serialization timing of the final state notification delayed sending queue can be used as the trigger timing for updating the value of the closing transaction counter. Furthermore, since a closing transaction generates a final state notification to be sent, and the cumulative number of generated final state notifications is equal to the cumulative number of closed transactions, the value of the closing transaction counter can be increased using the preset cache quantity threshold.

[0063] In some exemplary embodiments, the preset customs clearance transaction counter includes: customs clearance transaction counters corresponding to several banks; the preset final status notification counter includes: final status notification counters corresponding to several banks;

[0064] The counting device 30 is further configured to: obtain the transaction score counter values ​​of each bank corresponding to each transaction closing business node from the log file and save them to the search server 40; and obtain the final state notification score counter values ​​of each bank corresponding to each final state notification sending node from the log file and save them to the search server 40.

[0065] Each transaction closing business node 101 counts the transaction closing counters for each bank, and each final state notification sending node 102 counts the final state notification counters for each bank. The extracted counter information is then saved to the search server 40 for subsequent finer-grained counting and aggregation.

[0066] In some exemplary embodiments, the counter information includes at least: business node, counter identifier, bank identifier, and latest count value.

[0067] In some exemplary embodiments, the counting device 30 is further configured to: determine the transaction processing status of each bank based on the transaction counter values ​​of each bank and the final status notification counter values ​​of each bank stored in the search server 40.

[0068] In some exemplary embodiments, the counting device 30 is further configured to: save the counter information to the search server 40; submit search conditions to the search server 40 to obtain corresponding search results; and determine the transaction processing status of one or more banks based on the search results.

[0069] In some exemplary embodiments, the counter information includes: business node identifier, counter identifier, bank identifier, latest count value, and update time; the search criteria include at least: bank identifier.

[0070] For example, the minute counter information saved on search server 40 is as follows:

[0071]

[0072] Among them, counter identifier 1 is the customs clearance transaction counter, which corresponds to each customs clearance business node 101 and is called the customs clearance transaction sub-counter; counter identifier 2 is the final state notification counter, which corresponds to each final state notification sending node 102 and is called the final state notification sub-counter.

[0073] For example, if the bank identifier in the search criteria is BankID1, the counting device 30 submits the search criteria to the search server 40. The search server 40 searches its stored counter information based on this bank identifier and returns the total count value of the customs clearance transaction counter (i.e., the total number of customs clearance transactions for that bank) and the total count value of the final status notification counter (i.e., the total number of final status notifications sent for that bank). Specifically, the total count value returned by the search server for counter identifier 1 is 500 + 4000 + 7000, and the total count value for counter identifier 2 is 200 + 550 + 6000. Based on these two total count values, the counting device 30 can determine the current overall status of final status notifications sent for that bank, i.e., 4750 final status notifications have not yet been sent.

[0074] In some exemplary embodiments, the search server can further search and statistically analyze the changes in the total value corresponding to counter identifier 1 within a unit of time based on historical counter information, for example, it can count how many new final status notifications are added every 10 minutes; and / or, search and statistically analyze the changes in the total count value corresponding to counter identifier 2 within a unit of time, for example, it can count how many final status notifications are sent every 10 minutes, and so on.

[0075] It should be noted that the above example illustrates some of the search results that search server 40 can provide. Search server 40 can also define other search conditions and corresponding search results. The counting device 30 simply needs to submit the corresponding search conditions. Detailed examples of more search conditions and corresponding search results are not described here. Those skilled in the art can determine the corresponding implementation method as needed based on the above example.

[0076] As can be seen, the counting device can determine the backlog of final status notifications to be sent by each bank in real time, or the speed at which final status notifications are sent. Based on this, it is possible to further determine whether the relevant receiving systems of the connected banks are functioning normally or whether they meet business processing requirements, so as to further adjust the configuration of business nodes or notify banks to investigate or improve processing capabilities.

[0077] This disclosure also provides a distributed counting method, such as... Figure 3 As shown, it includes:

[0078] Step 310: Under the condition that the preset counter value writing conditions are met, each distributed service node writes the latest value of the counter to the log file on the log node.

[0079] Step 320: The counting device obtains the sub-counter values ​​of each of the distributed business nodes from the log file; and determines the total count value of the preset counter in the distributed business system based on the sub-counter values.

[0080] In some exemplary embodiments, each of the distributed service nodes includes one or more application systems; each application system includes one or more counters.

[0081] In some exemplary embodiments, the log nodes include one or more; the log files on each log node include one or more.

[0082] In some exemplary embodiments, the log node and the service node can be the same physical node; or they can be different physical nodes.

[0083] The preset conditions for writing the counter value include at least one of the following:

[0084] The preset log writing event occurs;

[0085] The preset log writing interval has arrived.

[0086] For example, if the default log write event is a change in the minute counter, it means that the latest value will be written to the log file when the minute counter changes.

[0087] As can be seen, although the counter value in the business node affects the total count value of the preset counters in the entire distributed system, the application in the business node does not operate on the shared counter in the overall distributed system; it only operates on its own counter in local memory. This avoids conflicts caused by multiple nodes operating on the shared counter and significantly reduces the system overhead of node counter operations. The node's operation on the counter is timely, accurate, and stable. Furthermore, there is no need to introduce other components to assist in the operation of the shared counter in the distributed system, greatly improving the operational stability of the entire distributed system.

[0088] Example 1

[0089] In a distributed online payment system, when the transaction closing service executes tasks, these tasks are randomly distributed to different service instances (deployed on different distributed nodes). These instances will inevitably differ in terms of accepting banks and transaction volumes. Under normal circumstances, after completing the transaction closing process for its assigned transaction flow, an instance (node) will send a notification of the transaction's final state to the target system.

[0090] However, during certain special periods, the transmission of some notification information may be suspended, delayed, or slowed down. For example, during critical periods, the business system needs to delay and control the rate of sending final status notifications for closing transactions. When the final status notification delay switch is turned on, the final status notifications for closing transactions are delayed. After the instance (node, also known as the transaction closing business node) executing the closing transaction reaches the number of delayed flush transactions, the delay queue will be serialized to the local disk. Since one final status notification corresponds to one closing transaction, the value of the closing transaction counter can be updated, that is, the closing transaction counter cached by the corresponding bank is increased simultaneously.

[0091] When resuming final status notification sending, the instance (node) executing delayed final status notification sending will send the delayed final status notification. Each completed final status notification send will increment its corresponding bank's final status notification counter. The business system needs to monitor the status of cached final status notifications in real time. When the cached customs clearance transaction counter matches the final status notification counter, the final status notification sending is considered complete. Specifically, the instance (node) executing customs clearance transactions includes one or more nodes; the instance (node) executing delayed final status notification sending includes one or more nodes; a node can include either an instance executing customs clearance transactions or an instance executing delayed final status notification sending; or, a node can include both an instance executing customs clearance transactions and an instance executing delayed final status notification sending. As can be seen, in this distributed online payment system, the total cached transaction counter value is the sum of the cached transaction sub-counter values ​​on multiple nodes, and the total final notification counter value is the sum of the final notification sub-counter values ​​on multiple nodes; the total cached transaction counter value for a certain bank is the sum of the cached transaction sub-counter values ​​for that bank on multiple nodes, and the total final notification counter value for that bank is the sum of the final notification sub-counter values ​​for that bank on multiple nodes.

[0092] The total cached transaction counter, the total final status notification counter, the total cached transaction counter of a bank in the distributed online payment system, and the total final status notification counter of a bank in the distributed online payment system are all embodiments of the preset counters described in this disclosure.

[0093] The counting methods provided in this example include:

[0094] Step 100: After the node increases the closed transaction counter or the final state notification counter of its corresponding bank, it starts a scheduled task to write the information of these two counters to the log file every 10 seconds according to the formatted log convention.

[0095] Step 200: Based on the preset extraction interval of 15 seconds, extract the cached transaction counter values ​​and / or final state notification counter values ​​of multiple nodes from the log files of multiple nodes using Flume, and send them to the ES server.

[0096] Step 300: The business monitoring system executes a search task every 20 seconds to obtain the values ​​of the above counters for all nodes from ES, and aggregates and sums them according to multiple dimensions such as bank and node to determine the total cached transaction counter value for each bank, the total final status notification counter value for each bank, the total cached transaction counter value for the entire online payment system, and the total final status notification counter value for the entire online payment system.

[0097] Step 400: Based on the total cached transaction counter value of each bank and the total final status notification counter value of the corresponding bank, determine whether the delayed final status notification sending speed of each bank is normal and whether it can be completed within the expected time; or, based on the total cached transaction counter value of the system and the total final status notification counter value of the system, determine whether the delayed final status notification sending speed of the overall system is normal and whether it can be completed within the expected time, etc.

[0098] It should be noted that step 400 above is merely an illustrative example of how the total count value of multiple preset counters in a distributed system for online payments can be further used. Based on this example, those skilled in the art can preset one or more other counters in the system as needed, determine the corresponding total count value, and perform subsequent business processing based on the determined total count value, without being limited to the specific aspects shown in this example.

[0099] As can be seen, the distributed counting scheme proposed in this disclosure abandons the centralized counting method using shared counters in traditional distributed service counting schemes. This effectively reduces the counting complexity of distributed nodes, decreases the coupling between nodes and shared counters, and significantly reduces the overall system complexity. Each node operates its own sub-counter, avoiding access conflicts and system overhead caused by sharing counters among multiple nodes. This improves the stability of node business operations and enhances the overall stability and high availability of the distributed system. Especially for payment systems with massive transaction volumes and high concurrency, using the scheme proposed in this disclosure is the cornerstone of ensuring the high availability of the overall system.

[0100] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0101] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made under the concept of the present invention using the contents of the present invention specification and drawings, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A distributed counting system, characterized in that, include: Several distributed business nodes, log nodes, and counting devices; Each of the distributed service nodes is configured to: write the latest value of the counter to the log file on the log node when the preset counter value writing condition is met; the counting device is configured to: obtain the counter value of each of the distributed service nodes from the log file respectively; and determine the total count value of the preset counter in the distributed service system based on the counter values ​​of each of the distributed service nodes. The distributed business system is an online payment system. The distributed business nodes include: several transaction closing business nodes and several final state notification sending nodes; Each of the transaction closing business nodes is configured as follows: when the first counter value writing condition is met, the cumulative number of closed transactions is used as the latest value of the closing transaction counter and written to the log file on the log node. Each of the final state notification sending nodes is configured to: when the condition for writing the second counter value is met, use the cumulative number of final state notifications that have been sent as the latest value of the final state notification counter and write it into the log file on the log node; The counting device is further configured to: obtain the latest value of each transaction counter from the log file, and determine the total number of transactions in the online payment system based on the latest value of each transaction counter; obtain the latest value of each final notification counter from the log file, and determine the total number of final notifications sent by the online payment system based on the latest value of each final notification counter; and determine the transaction processing status of the online payment system based on the total number of transactions and the total number of final notifications sent.

2. The system as described in claim 1, characterized in that, The counting device obtains the counter values ​​of each of the distributed service nodes from the log file, including: The minute counter values ​​are obtained from the log files using a log extraction tool.

3. The system as described in claim 1, characterized in that, The system also includes: a search server; The counting device is further configured to: save the values ​​of each of the sub-counters to the search server; The counting device is further configured to: obtain search results from the search server according to the search conditions corresponding to the preset counter, and determine the total count value of the preset counter in the distributed business system according to the search results.

4. The system as described in any one of claims 1-3, characterized in that, The conditions for writing the counter value include one or more of the following: The value of the minute counter changes; The preset write interval has arrived; Pre-defined events occur on distributed business nodes.

5. The system as described in claim 1, characterized in that, Each of the transaction closing order business nodes is further configured to: when the final state notification delay switch is turned on, save the generated final state notifications to be sent corresponding to the closing order transaction to the delayed sending queue; when the number of final state notifications in the delayed sending queue reaches a preset cache quantity threshold, update the value of the closing order transaction counter using the preset cache quantity threshold, and save the final state notifications in the delayed sending queue to the storage device; Each of the final state notification sending nodes is further configured to: when the final state notification delay switch is off, acquire the final state notification in the delay sending queue and / or the storage device and send it out.

6. The system as described in claim 1, characterized in that, The customs clearance transaction counter includes: customs clearance transaction counters corresponding to several banks; The final state notification counter includes: final state notification counters corresponding to several banks; The counting device is further configured to: obtain the transaction score counter values ​​of each bank corresponding to each transaction closing business node from the log file and save them to the search server; and obtain the final state notification score counter values ​​of each bank corresponding to each final state notification sending node from the log file and save them to the search server.

7. The system as described in claim 6, characterized in that, The counting device is further configured to determine the transaction processing status of each bank based on the transaction counter values ​​of each bank's customs declaration and the final status notification counter values ​​of each bank stored in the search server.

8. The system as described in claim 6, characterized in that, The counting device is further configured to: save the counter information to the search server; submit search conditions to the search server to obtain the corresponding search results; and determine the transaction processing status of one or more banks based on the search results. The counter information includes: business node identifier, counter identifier, bank identifier, latest count value, and update time; The search criteria include at least: bank identifiers.

9. A distributed counting method, characterized in that, include: Each distributed service node writes the latest value of the counter to the log file on the log node when the preset conditions for writing the counter value are met. The counting device obtains the sub-counter values ​​of each of the distributed business nodes from the log file; and determines the total count value of the preset counter in the distributed business system based on the sub-counter values. The distributed business system is an online payment system. The distributed business nodes include: transaction closing business nodes and final state notification sending nodes; Each of the transaction closing business nodes is configured as follows: when the first counter value writing condition is met, the cumulative number of closed transactions is used as the latest value of the closing transaction counter and written to the log file on the log node. Each of the final state notification sending nodes is configured to: when the condition for writing the second counter value is met, use the cumulative number of final state notifications that have been sent as the latest value of the final state notification counter and write it into the log file on the log node; The counting device is further configured to: obtain the latest value of each transaction counter from the log file, and determine the total number of transactions in the online payment system based on the latest value of each transaction counter; obtain the latest value of each final notification counter from the log file, and determine the total number of final notifications sent by the online payment system based on the latest value of each final notification counter; and determine the transaction processing status of the online payment system based on the total number of transactions and the total number of final notifications sent.

Images