Resource manager for transaction processing systems

Abstract

A Resource Manager (RM) instance is associated with each TPS member of a Transaction Processing System (TPS) group. Each RM instance monitors the performance of the associated TPS member. If a TPS member becomes unavailable for any reason (a failed TPS), the associated RM instance broadcasts the state of the failed TPS to the RMs associated with the "surviving" members of the group. The RM instances associated with the surviving members initiate a series of actions that reduce the resources used by the surviving TPS members. As a result, the surviving TPS members are better able to handle the additional workload imposed on them due to the unavailability of the failed TPS. Once the failed TPS is brought back online and available again (or a replacement TPS is brought online), the RM instances associated with the surviving members perform actions to undo the resource usage reduction tasks, and the TPS group returns to the nominal configuration.

Description

Background Art

[0001] The present invention relates generally to the field of transaction processing, and more particularly to transaction processing system responses to constrained resource events.

[0002] A transaction processing system (TPS) receives transaction requests and processes them in near real time. Transactions can involve banking, credit / debit card, database, and / or ticket sales, to name a few. A TPS can consist of many individual systems working together to process a large number of incoming transaction requests within response time specifications. Characteristics of a TPS include continuous availability, data integrity, even if some TPS components fail, and the ability to scale up or down as workloads change. Summary of the Invention

[0003] According to one aspect of the present invention, there is a method, computer program product, and / or system that performs the following operations (not necessarily in the following order): (i) determining that a first status of a first transaction processing system (TPS) member of a TPS group is "unavailable"; (ii) in response to determining the first status of the first TPS member, broadcasting a first message to a second TPS member in the TPS member group, wherein the first message includes information about the first status of the first TPS member; and (iii) in response to receiving the first message, implementing a resource usage reduction action by the second TPS member. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 is a block diagram of a system according to at least one embodiment of the present invention;

[0005] Figure 2 is a flow chart illustrating a method performed, at least in part, according to at least one embodiment of the present invention;

[0006] Figure 3 is a block diagram illustrating the machine logic (e.g., software) portion of a system according to at least one embodiment of the present invention;

[0007] Figure 4 is a block diagram of a transaction processing system according to at least one embodiment of the present invention;

[0008] Figure 5A is a flow chart illustrating a method performed, at least in part, according to at least one embodiment of the present invention;

[0009] Figure 5B is a flow chart illustrating a method performed, at least in part, according to at least one embodiment of the present invention;

[0010] Figure 5Cis a flow chart illustrating a method performed, at least in part, according to at least one embodiment of the present invention;

[0011] Figure 5D is a flow chart illustrating a method performed, at least in part, according to at least one embodiment of the present invention; and

[0012] Figure 5E is a flow chart illustrating a method performed, at least in part, in accordance with at least one embodiment of the present invention. DETAILED DESCRIPTION

[0013] Some embodiments of the present invention add a resource manager (RM) component to a transaction processing system (TPS). More specifically, an instance of the RM is hosted by each TPS member of a TPS group. Each RM instance monitors the performance of the hosted TPS. If a TPS member fails, or becomes unavailable to process incoming transactions, the RM instance hosted by the failed TPS broadcasts the status of the failed TPS to the other (surviving) members of the TPS group. The RM instances hosted by the surviving members of the TPS group initiate a series of actions that reduce the resources used by the respectively corresponding surviving members so that the surviving members can better handle the additional workload imposed on them due to the unavailability of the failed TPS. Once the failed TPS (or replacement TPS) is brought back online and available again, the RM instances hosted by the surviving members of the TPS group perform actions to undo the resource usage reduction tasks, and the TPS group returns to the nominal configuration.

[0014] This detailed description section is divided into the following subsections: (i) Hardware and Software Environment; (ii) Example Embodiments; (iii) Further Comments and / or Examples; and (iv) Definitions.

[0015] I. Hardware and Software Environment

[0016] The present invention may be a system, method and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or multiple media) having computer-readable program instructions thereon, the computer-readable program instructions being used to cause a processor to perform various aspects of the present invention.

[0017] Computer-readable storage media can be a tangible device that can retain and store the instructions used by the instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), static random access memories (SRAM), portable compact disc read-only memories (CD-ROM), digital versatile disks (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or raised structures in grooves with instructions recorded thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein should not be interpreted as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated by waveguides or other transmission media (e.g., light pulses by optical fiber cables), or electrical signals transmitted by wires.

[0018] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the corresponding computing / processing device.

[0019] The computer-readable program instructions for performing the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages, such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, to perform various aspects of the present invention, an electronic circuit comprising, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may execute the computer-readable program instructions to personalize the electronic circuit by utilizing the state information of the computer-readable program instructions.

[0020] Various aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0021] These computer-readable program instructions can be provided to a processor of a computer or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create components for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can direct the computer, programmable data processing device and / or other equipment to operate in a specific manner, so that the computer-readable storage medium having the instructions stored therein includes an article of manufacture, which includes instructions for implementing various aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0022] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0023] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each frame in the flow chart or block diagram can represent a module, segment or part of an instruction, which includes one or more executable instructions for realizing the specified (multiple) logical functions. In some alternative embodiments, the function noted in the frame may not occur in the order noted in the figure. For example, the two frames shown in succession can actually be implemented as a step, simultaneously, substantially simultaneously, in a manner that partially or entirely overlaps, or these frames can sometimes be performed in reverse order, depending on the function involved. It will also be noted that the combination of the frames in the block diagram and / or flow chart illustration and the block diagram and / or flow chart illustration can be implemented by a dedicated hardware-based system that performs a specified function or action or performs a combination of dedicated hardware and computer instructions.

[0024] Embodiments of possible hardware and software environments for the software and / or method according to the present invention will now be described in detail with reference to the accompanying drawings. Figure 1 1 is a functional block diagram showing various parts of the networked computer system 100, including: storage subsystem 102; client computers 104, transaction processing system (TPS) 106; communication network 114; resource manager (RM) server computer 200; communication unit 202; processor set 204; input / output (I / O) interface set 206; memory 208; persistent storage 210; display 212; external devices 214; random access memory (RAM) 230; cache 232; and resource manager (RM) 300.

[0025] The storage subsystem 102 is representative in many respects of various computer subsystems in the present invention. Accordingly, several portions of the storage subsystem 102 will now be discussed in the following paragraphs.

[0026] The storage subsystem 102 may be a laptop computer, a tablet computer, a netbook computer, a personal computer (PC), a desktop computer, a personal digital assistant (PDA), a smart phone, or any programmable electronic device capable of communicating with the client subsystem via the communication network 114. The RM 300 is a collection of machine-readable instructions and / or data used to create, manage, and control certain software functions, which are discussed in detail below in the exemplary embodiments subsection of this detailed description.

[0027] The storage subsystem 102 can communicate with other computer subsystems via a communication network 114. For example, the communication network 114 can be a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination of both, and can include wired, wireless, or fiber optic connections. In general, the communication network 114 can be any combination of connections and protocols that support communication between server and client subsystems.

[0028] The storage subsystem 102 is shown as a block diagram with many double arrows. These double arrows (without separate reference numerals) represent a communication structure that provides communication between the various components of the storage subsystem 102. The communication structure can be implemented using any architecture designed to transfer data and / or control information between processors (such as microprocessors, communication and network processors, etc.), system memory, peripheral devices, and any other hardware components within the system. For example, the communication structure can be implemented at least in part using one or more buses.

[0029] Memory 208 and persistent storage 210 are computer-readable storage media. In general, memory 208 may include any suitable volatile or non-volatile computer-readable storage media. It should also be noted that currently and / or in the near future: (i) external device 214 may be able to provide some or all of the memory for storage subsystem 102; and / or (ii) devices external to storage subsystem 102 may be able to provide memory for storage subsystem 102.

[0030] RM 300 is stored in persistent storage 210 for access and / or execution by one or more corresponding computer processor sets 204, typically through one or more memories of memory 208. Persistent storage 210: (i) is at least more permanent than a signal in transit; (ii) stores the program (including its soft logic and / or data) on tangible media (such as magnetic or optical domains); and (iii) is substantially less permanent than persistent storage. Alternatively, data storage may be more permanent and / or permanent than the type of storage provided by persistent storage 210.

[0031] RM 300 may include machine-readable and executable instructions and / or substantive data (i.e., the type of data stored in a database). In this particular embodiment, persistent storage 210 includes a magnetic hard drive. To name a few possible variations, persistent storage 210 may include a solid-state hard drive, a semiconductor memory device, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), flash memory, or any other computer-readable storage medium capable of storing program instructions or digital information.

[0032] The media used by persistent storage 210 may also be removable. For example, a removable hard drive may be used for persistent storage 210. Other examples include optical and magnetic disks, thumb drives, and smart cards that are inserted into a drive for transfer to another computer-readable storage medium that is also part of persistent storage 210.

[0033] In these examples, communication unit 202 provides communications with other data processing systems or devices external to storage subsystem 102. In these examples, communication unit 202 includes one or more network interface cards. Communication unit 202 can provide communications using one or both of physical and wireless communication links. Any software modules discussed herein can be downloaded to a persistent storage device (such as persistent storage device 210) via a communication unit (such as communication unit 202).

[0034] The I / O interface set 206 allows for input and output of data with other devices that may be locally connected to the server computer 200 in data communication. For example, the I / O interface set 206 provides a connection to an external device 214. The external device 214 may include devices such as a keyboard, a keypad, a touch screen, and / or some other suitable input device. The external device 214 may also include a portable computer-readable storage medium such as a thumb drive, a portable optical or magnetic disk, and a memory card. Software and data used to implement embodiments of the present invention, such as RM 300, may be stored on such a portable computer-readable storage medium. In these embodiments, the relevant software may (or may not) be loaded in whole or in part onto the persistent storage device 210 via the I / O interface set 206. The I / O interface set 206 also connects to the display device 212 using data communication.

[0035] Display device 212 provides a mechanism for displaying data to a user and may be, for example, a computer monitor or a smartphone display screen.

[0036] The programs described herein are identified based on the applications in which they are implemented in specific embodiments of the invention. However, it should be understood that any specific program terminology herein is used for convenience only, and thus, the present invention should not be limited to use solely in any specific application identified and / or implied by such terminology.

[0037] The description of various embodiments of the present invention has been provided for the purpose of illustration, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, practical applications, or improvements over existing technologies in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

[0038] II. Example Embodiments

[0039] Figure 2 A flow chart 250 describing a method according to the present invention is shown. Figure 3 A resource manager (RM) 300 is shown for performing at least some of the method operations of flowchart 250. In the course of the following paragraphs, and with extensive reference to Figure 2 (for method action blocks) and Figure 3 (for software blocks) to discuss the method and associated software.

[0040] Processing begins at operation S255, where the guard module 302 of the resource manager (RM 300) determines that the status of the first transaction processing system (first TPS) member associated with the RM 300 is "unavailable". The first TPS is a member of a group of TPS members that work together to handle incoming transaction requests. In some embodiments, the RM 300 is hosted by the first TPS, and each TPS member of the group hosts a separate instance of the RM 300. Alternatively, in some embodiments, one or more instances of the RM 300 reside outside the associated TPS member (e.g., on separate computer hardware), so that if a TPS member fails, the RM 300 will be able to continue operating and will not be affected by the TPS failure.

[0041] RM 300 monitors the first TPS member for conditions that affect its availability to process transaction requests, including both incoming requests and those already in progress. RM 300 monitors the first TPS member by observing one or more characteristics associated with the performance and / or availability of the first TPS member. Example characteristics include: the rate of incoming transactions, where a sudden and / or unexpected drop in the rate may indicate a network failure; the time to process a transaction, where, for example, an unusually long time to process a transaction may indicate that the central processing unit (CPU) is throttling due to temperature issues; CPU clock frequency; the frequency, number, and / or rate at which errors occur; CPU utilization; memory read and / or write latency, or any other metric associated with memory access; storage read and / or write latency, or any other metric associated with storage access; and / or network communication status, to name a few characteristics.

[0042] Examples of conditions that may affect the availability of TPS members include (but are not limited to): network failures or outages; hardware failures (such as processor, storage or memory device failures, or power outages); software failures (such as operating system, virtual machine, or application crashes); and / or excessive workloads that cause TPS failures due to exhaustion of compute, memory, and / or storage resources.

[0043] The guardian module 302 also monitors notification messages received by the first TPS or RM 300, wherein the notification indicates the status (available, unavailable, etc.) of other TPS members of the group.

[0044] Processing continues at operation S260 where, in response to determining that the first TPS has become unavailable, the broadcast module 304 of the RM 300 sends a notification to all other members of the TPS group (including the second TPS member) The notification includes information about the failure of the first TPS member.

[0045] Alternatively, in some embodiments, manually or automatically generated commands direct the broadcast module 304 to send notifications to members of the TPS group, directing them to enter "constrained" mode. While not in response to a TPS member failure, such commands can be issued to better enable the TPS group to handle particularly heavy workloads, such as may occur periodically during certain "busy" times. This resource consumption reduction action can enable the TPS system to handle busy times without requiring additional TPS processing resources.

[0046] Processing continues at operation 265, where, in response to receiving the notification, the constraint module 306 of the RM 300 associated with the second TPS member implements an action (and / or causes the second TPS member to implement an action) that reduces resource usage by the second TPS member. In other words, the constraint module 306 causes the second TPS member to reconfigure into "constrained mode" and thereby consume fewer resources.

[0047] Once in "constraint mode," the second TPS member reduces resource consumption by taking at least the following actions: making the transaction instance block (TIB) lighter to use less memory; implementing a delay factor to help control transaction flow; disabling diagnostic tracing; disabling TPS dumps (at least for non-termination issues); and canceling idle transactions at a higher frequency.

[0048] Processing continues at operation 270 where the guardian module 302 of the RM 300 determines that the first TPS status has returned to “available”.

[0049] Alternatively, in some embodiments, the guardian module 302 of the RM 300 determines that a new TPS member has been added to the TPS group, and the status of the new TPS member is "available," thereby replacing the first TPS member.

[0050] Processing continues at operation S272, where, in response to determining that the first TPS status has returned to "available" (or a new TPS member has become available in the TPS group), the broadcast module 304 of the RM 300 sends a notification to all other members of the TPS group (including the second TPS member). The notification includes information about the return of the first TPS member to availability (or replacement).

[0051] Processing continues at operation 280 where, in response to receiving the notification, the constraint module 306 of the RM 300 associated with the second TPS member reverses (undoes) the actions implemented in operation S265 above and thereby returns the second TPS member to the standard processing configuration.

[0052] III. Further Comments and Examples

[0053] A transaction processing system (TPS) consists of multiple physical systems working as a group to process incoming transaction workloads in real time. Individual transactions can be placed into one or more transaction queues to facilitate sharing of the transaction workload among TPS group members. Each transaction can be processed by any of the physical systems to meet the required response time standards.

[0054] Each transaction processed by the TPS consumes system resources within the group member that processes the transaction. The infrastructure for commonly processed transactions is replicated across TPS members. However, for efficiency or other reasons, subsets of transactions (priority transactions) can be directed to certain TPS members for processing. The physical system resources used by the TPS include the resources required to process common transactions (processed by any TPS member) and the additional resources required to process priority transactions.

[0055] When a TPS group member suffers a catastrophic failure (meaning that the member is unable to process transactions), the shared transaction workload must be distributed among the surviving TPS group members, placing an additional burden on the physical system resources associated with the surviving members. Furthermore, each TPS group member creates infrastructure for processing transactions that were previously prioritized only on the failed TPS group member. The resulting additional prioritized transaction infrastructure requires a greater level of physical system resources to be utilized in the remaining TPS group members.

[0056] If a TPS group member fails, the TPS group enters a "critical period," which begins when the TPS member fails and ends when the failed TPS member recovers and restarts, or is replaced by a new member added to the group. During this critical period, a sudden increase in the required physical resources placed on the surviving TPS group members can cause the surviving TPS group members to experience their own catastrophic failures in a cascading effect due to reaching the resource limits of their physical systems.

[0057] Some embodiments of the present invention include methods and systems for creating a resource manager (RM) that allows members of a TPS to work together to handle large web-related banking workloads and survive catastrophic failures of one or more members of the TPS group.

[0058] The Resource Manager is a new component of each member of a TPS group. In the event of a catastrophic failure, the RM streamlines system resource utilization on the remaining TPS group members to prevent system resource exhaustion and subsequent catastrophic failure of those members.

[0059] The RM component is designed to operate in four modes: (i) ready mode; (ii) broadcast mode; (iii) constrained mode; and (iv) return to ready mode. The modes are described as follows:

[0060] Ready ModeThe RM is initialized and waits for notification of: (i) catastrophic failures from the TPS system on which it is installed; and (ii) unavailable status messages sent from TPS group members in broadcast mode. In some embodiments, the RM instance hosted on the TPS system includes a daemon process that performs one or more response actions when triggered by a notification message from the master TPS system or from another instance of the RM.

[0061] Broadcast Mode ——RM enters broadcast mode to send TPS member "unavailable" and "available" status messages to the surviving TPS group members.

[0062] In response to a TPS member experiencing a failure, the failed TPS member invokes the RM instance hosted by the failed TPS member and instructs the RM to enter "broadcast mode" to send an "unavailable status" message to surviving TPS group members. In some embodiments, the RM instance enters "broadcast mode" in response to detecting a system termination process or other indication of a failure of the TPS on which the RM instance is installed.

[0063] In response to the completion of the system restart process for the failed TPS member, the RM broadcast mode sends an "Available Status" message to the surviving TPS group members. The "Available Status" message signals the availability of the previously failed TPS member to resume processing the transaction workload within the TPS group.

[0064] Once the RM has completed sending the "Available Status" message, the RM re-enters "Ready Mode".

[0065] Constraint Mode In response to receiving an "unavailable status" message corresponding to the failed TPS group member, the surviving TPS group member enters constraint mode. In constraint mode, the RM controls and reduces the system resources used by the surviving TPS group member. Constraint mode enables the surviving TPS systems to absorb the increased transaction workloads routed to them, which would normally have been routed to and processed by the failed TPS group member.

[0066] The TPS group members remain in the constrained mode during a critical time period, which is a period of time beginning when the failed TPS group member fails and ending with the subsequent recovery and restart of the previously failed TPS group member. More specifically, the critical time period (with respect to each surviving TPS group member) begins when a surviving TPS group member receives an "unavailable status" message and ends when the surviving TPS group member receives an "available status" message. In some embodiments, the surviving TPS group member receives an "available status" message from the new TPS member that is added to the group that is (and may be) designated as the replacement for the failed member.

[0067] In some embodiments of the present invention, the constraint mode includes tasks for reducing usage of system resources during "critical times" (while TPS group members are unavailable) by the continued presence of TPS group members.

[0068] 1. Make transaction instance blocks lighter.

[0069] 2. Activate flood monitors with weighting factors.

[0070] 3. Activate transaction flow control using the delay time factor.

[0071] 4. Disable TPS diagnostic tracing.

[0072] 5. Disable TPS dump for non-termination issues.

[0073] 6. Start canceling idle transactions more frequently.

[0074] In some embodiments, the RM provides an option (via a command interface or via automation) to dynamically activate and deactivate "Constraint Mode" for any selected period of time to reduce system resource usage during that period of time. The RM user may also choose to activate "Constraint Mode" on a permanent, full-time basis to reduce daily system resource usage.

[0075] The following will be about Figure 4 and Figures 5A-5E , describing in more detail the tasks outlined above for constraint mode.

[0076] Return to ready mode Once the surviving TPS group members have received all "Available Status" messages, the RM enters "Return to Ready Mode" to undo the tasks executed during "Constrained Mode." Once the RM completes the "Return to Ready Mode" process, the RM (and therefore the associated TPM) re-enters "Ready Mode."

[0077] Figure 4is a block diagram illustrating a high availability system 400 according to at least one embodiment of the present invention. High availability system 400 includes: any number of mobile and / or fixed user devices 405; a sysplex distributor 410; any number of Transmission Control Protocol / Internet Protocol (TCP / IP) gateways, including a first gateway 411, a second gateway 412, and an Mth gateway 413; a transaction processing system (TPS) group 425, including any number of TPS members, including a first TPS 421, a second TPS 422, and an Nth TPS 423; and a transaction queue 415. Note: In some embodiments, the number of TCP / IP gateways (M) may or may not be equal to the number of TPS members (N).

[0078] Users generate transactions through user devices 405 (mobile and / or fixed, such as smartphones, tablets, laptop and desktop computers, smart watches, televisions, etc.), and there can be any number of such systems operating simultaneously, especially in large enterprise-level systems such as online payment systems or media streaming services.

[0079] In some embodiments, a transaction arrives at the sysplex dispatcher 410. The sysplex dispatcher 410 routes the transaction to any available TCP / IP gateway. The TCP / IP gateway then routes the transaction to any available transaction processing system (TPS) member of the TPS group 425. (Note: The term "Sysplex" may be subject to trademark rights in various jurisdictions around the world and is used herein only with reference to appropriately named products or services by mark, to the extent such trademark rights may exist).

[0080] To maintain high availability, the members of the TPS group 425 work together to avoid having a single point of failure. Each TPS member can have a preferred set of (one or more) input transaction types for which the member is optimized to process efficiently. The TCP / IP gateway routes incoming transactions of a given type to the TPS member optimized for that transaction type.

[0081] Any TPS member can make transactions available to any TPS member of the group for processing on the transaction queue 415. Any TPS member can create a transaction instance block (TIB) to represent an incoming transaction or process a transaction from another TPS member of the group. Each TIB consumes system resources.

[0082] Now consider a scenario where the first TPS 421 becomes unavailable due to a system failure, a network failure, or any other reason that prevents the first TPS 421 from processing the transactions assigned to it. Such failures are sometimes referred to herein as "emergency situations." In response to the failure, the system will have been directed to the first TPS 421 (see Figure 4 , bold line 424) to one or more surviving TPS members, such as the second TPS 422 and the Nth TPS 423. The first TPS 421 can also redistribute the workload to other TPS members of the TPS group 425 by placing the workload on the transaction queue 415.

[0083] Redirected transactions impose a greater workload on the surviving TPS members that receive the redirected transactions. Some of the surviving TPS members that receive the increased workload can potentially exhaust system resources (at least in part due to the increased workload), causing them to crash. Each surviving TPS member that receives the redirected workload is allocated a TIB associated with the redirected workload, which is associated with the transactions placed on the transaction queue 415 from the first TPS 421. In embodiments that process a large number of transactions, the correspondingly large number (possibly thousands or more) of new TIBs associated with the redirected workload can overwhelm the surviving TPS members.

[0084] In some embodiments, such as large transaction processing systems, restarting an unavailable or failed TPS member may take five to ten minutes. During this critical time window, the surviving TPS members may be affected enough to cause a single TPS system failure or even a cascading failure of the TPS group 425.

[0085] Surviving TPS members running out of system resources can cause the creation of a large number of new TIBs corresponding to the workload redirected to the surviving members. If a TPS member fails, a cascading failure sequence can begin, in which workload piles up on the surviving TPS members, causing more members to fail in turn, redirecting increasingly heavier workloads to a decreasing number of surviving members. In this type of scenario, the entire TPS system can quickly fall in a falling domino fashion, all initially triggered by the failure of one TPS element.

[0086] Some embodiments of the present invention install an instance of a "Resource Manager" (RM 601) for each member of the TPS group 425. The RM 601 helps the TPS group 425 survive if one or more TPS members become unavailable.

[0087] In some embodiments, RM 601 includes software components installed in each respective TPS group member to streamline the use of system resources required to process transactions during an emergency and avoid cascading outages as described above.

[0088] RM 601 has four processing modes: (i) Broadcast; (ii) Constraint; (iii) Return to Ready; and (iv) Ready. The functions of the four modes are described below:

[0089] (i) Broadcast Mode - In response to entering the initial dump formatting phase associated with the downtime of the failed TPS member, a failed TPS member of a TPS group invokes a corresponding instance of RM 601. RM 601 broadcasts a notification message to some or all TPS members of the group. The broadcast notification indicates the "unavailable" status of the failed TPS member. Furthermore, once the TPS member completes its restart and becomes available again, RM 601 broadcasts a notification message to the group indicating the "available" status of the previously failed TPS member and the now-recovered TPS member.

[0090] In some embodiments, RM 601 broadcasts an "unavailable" notification to TPS members with the same preferences as the failed TPS. That is, if the failed TPS has a preference for processing credit card payment transactions (is optimized), RM 601 broadcasts an "unavailable" notification to surviving TPS members that also have a preference for processing credit card payment transactions. Simultaneously, the gateways that receive and distribute transactions to TPS group members redirect credit card payment transactions to TPS members with the same preferences as the failed TPS member. This minimizes the adverse impact on processing credit card payment transactions in the absence of the failed TPS member. In some embodiments, if a surviving TPS member is in danger of becoming overwhelmed by the additional workload, or is actually overwhelmed by the additional workload (for example, if the average time to process incoming transactions increases above a predetermined threshold), some of the additional workload can be redirected to other TPS members, regardless of their preferences. In this case, RM 601 broadcasts an "unavailable" notification to the other TPS members.

[0091] (ii) Constrained Mode - In response to receiving a broadcast notification about a failed TPS member, the surviving TPS members enter Constrained Mode. When operating in Constrained Mode, the TPS members control system resources in a manner that manages the increased workload and prevents system failures during the critical window (while the failed TPS member is unavailable).

[0092] (iii) Return to Ready Mode - Once the (previously) failed TPS member becomes available again, (or a replacement TPS member is introduced into the group, the surviving TPS members enter "Return to Ready Mode" to undo the special tasks performed in Constrained Mode.

[0093] (iv) Ready Mode - In "Ready Mode," no action is required from the RM 601 other than being ready to respond to a failure or impending failure of a TPS member of the group. The RM 601 detects an impending failure of a TPS group member based in part on actual resource usage compared to the maximum resource usage available to the TPS group member. If the actual resource usage is changing in an increasing direction and remains above an "action" threshold for a predetermined time span, the RM 601 determines an impending failure.

[0094] In some embodiments of the present invention, RM 601 performs “special” tasks when in constraint mode (see item (ii) above), including: (a) making transaction instance blocks lighter; (b) activating a flood monitor with a weighting factor; and (c) activating transaction flow control with a delay time factor:

[0095] (a) Make transaction instance blocks lighter.

[0096] Each TIB consists of a base component and an on-demand component. Transaction processing always requires the base component. The on-demand component is used for specific operations, such as receiving transaction input and delivering transaction output.

[0097] In constrained mode, the TPS creates a TIB with only the essential parts, consuming fewer system resources. The rest of the TIB is built or created on demand (i.e., when an operation is required). In addition, the RM scans existing TIBs that were created before the TPS entered constrained mode. If any on-demand parts of the existing TIB are not used, the TPS releases the storage associated with the existing TIB.

[0098] Additionally, the RM marks the TIB created for the new workload (the new TIB, which initially targets the failed / unavailable TPS) so that once the RM exits the constraint mode, i.e. once the RM enters the "return to ready" mode or "ready" mode, the processing TPS releases the new TIB first (before other TIBs).

[0099] (b) Activate flood monitors with weighting factors.

[0100] Since light TIBs are used when the TPS is in constrained mode, the RM considers smaller TIB sizes when calculating whether the flood limit has been reached for the number of all allocated TIBs.

[0101] For example, consider an optical TIB, lacking on-demand components (e.g., input and output components), which is 20% smaller than a conventional TIB. The resulting 20% ​​storage reduction factor, called a "weighting factor," is used to calculate an adjusted TIB flood limit X for flood control.

[0102] In some embodiments, the adjusted TIB flood limit X is determined using the following formula:

[0103] X=A+(0.2B)+(0.5C)+(0.7D)

[0104] in

[0105] A is the number of actual size TIBs

[0106] B is the number of TIBs with only base components (light TIBs)

[0107] C is the number of TIBs with base components and on-demand fields for input

[0108] D is the number of TIBs with base components and on-demand fields for output

[0109] 0.2, 0.5, and 0.7 (equal to 20%, 50%, and 70%, respectively) are example weighting factors applied to the numbers B, C, and D, respectively. These example weighting factors may vary from one embodiment to another.

[0110] (c) Activate transaction flow control with a delay time factor.

[0111] The RM activates a delay time factor feature for incoming transactions by adding delay time to incoming transactions in response to a high usage percentage of the TPS reaching a predefined flood threshold. The delay time factor helps reduce the impact on resource constraints caused by receiving increased incoming transactions from, for example, mobile devices. This allows the TPS to delay the creation of some TIBs, thereby consuming fewer system resources.

[0112] For example, if the TPS reaches 60% of the predefined flood threshold, the TPS assigns a delay time of 5 milliseconds to new input transactions before accepting the input transaction. If the TPS reaches 70% of the flood threshold, the TPS assigns a delay time of 10 milliseconds. In other words, the TPS dynamically increases or decreases the delay time as the flood situation worsens or improves, respectively. Note: The values ​​given here (60%, 70%, 5 milliseconds, and 10 milliseconds) are examples only. Some embodiments define different sets of flood threshold percentages and corresponding delay times. Some embodiments define a linear or nonlinear functional relationship between the fraction of the flood threshold reached (TPS load) and the response delay time. In addition, some embodiments establish a certain degree of hysteresis in the load / delay time relationship so that when the load changes in an increasing direction opposite to a decreasing direction, different delay times break in at different load breakpoints.

[0113] In addition to the above-mentioned “special” tasks (a), (b), and (c), the RM in constraint mode performs the following tasks: (iv) disables TPS diagnostic tracing to save storage space; (v) disables TPS dumps for non-termination issues; and (vi) cancels idle transactions with increased frequency to save storage space.

[0114] Figure 5A 、 5B , 5C, 5D and 5E, respectively, collectively describe a method according to an embodiment of the present invention for determining: (i) a source of a notification message related to a transaction processing system (TPS) failure; and (ii) a resource manager (RM 601; see also FIG. 6B ) in response to a notification related to a TPS failure. Figure 4 ) actions performed in the constraint mode. A TPS embodiment corresponding to the method described in the flowchart includes a TPS group 425 that includes at least a first and a second TPS member (respectively a first TPS 421 and a second TPS 422; see Figure 4 ).

[0115] The process begins at operation 501, where an instance of the RM corresponding to the first TPS 421 (hereinafter referred to as RM 601) receives a notification (TPS failure related notification) associated with an emergency situation involving a member of the TPS group 425. The notification may have been initiated from the first TPS 421 or from any other TPS member of the TPS group 425. For simplicity, the failed TPS will be referred to as the second TPS 422 (see Figure 4 ).

[0116] Processing continues at decision 503, where the RM 601 determines the source of the notification and the circumstances surrounding the notification. If the RM 601 determines that the notification is from the second TPS 422, and the second TPS 422 status has changed from available to unavailable (decision 503, "left" branch), processing continues at entry point A of flowchart 500B ( Figure 5B ).

[0117] If the RM 601 determines that the notification is from the second TPS 422 and the second TPS 422 status has changed from unavailable to available (decision 503, "right" branch), then processing continues at entry point B of flowchart 500C ( Figure 5C ).

[0118] If the RM 601 determines that the source of the notification is the first TPS 421 (decision 503 , “Center” branch), processing proceeds to selection operation 505 .

[0119] If the RM 601 determines that the first TPS 421 status changes from "available" to "unavailable" (select operation 505, "1" branch), the process proceeds to Figure 5D Flowchart 500D continues at entry point C.

[0120] If the RM 601 determines that the first TPS 421 status changes from "unavailable" to "available" (select operation 505, "2" branch), the process proceeds to Figure 5E Flowchart 500E continues at entry point D.

[0121] If the RM 601 determines that the first TPS 421 has entered a state of severe system resource shortage, but still remains "available" (selection operation 505, "3" branch), the process proceeds to Figure 5B Flowchart 500B continues at entry point A.

[0122] If the RM 601 determines that the first TPS 421 has recovered from a severe system resource shortage (the shortage has been alleviated) (operation 505 is selected, "4" branch), the process proceeds to Figure 5C Flowchart 500C continues at entry point B.

[0123] Now turn Figure 5B Flowchart 500B (entry point A, from Figure 5A 500A continued), processing continues at decision 520 where the RM 601 determines whether the first TPS 421 is in the restricted mode.

[0124] If the RM 601 determines that the first TPS 421 is in the restricted mode (decision 520, "yes" branch), the process proceeds to decision 521, where the RM 601 determines whether the notification was sent from the second TPS 422. If the RM 601 determines that the notification was sent from the second TPS 422 (decision 521, "yes" branch), the RM 601 ignores the notification.

[0125] If the notification was not sent from the second TPS 422 (decision 521, "no" branch), the RM 601 enables a maximum delay time factor for incoming (new) transactions and existing transactions (already received by the first TPS 421 and pending in the queue).

[0126] If the RM 601 determines that the first TPS 421 is not in the restricted mode (decision 520 , “no” branch), processing proceeds to operation 523 , where the RM 601 enters the restricted mode.

[0127] Processing continues at operation 524, where RM 601 performs the following tasks, not necessarily in the following order: (i) make the existing transaction instance block (TIB) lighter; (ii) create a new TIB with only basic components; (iii) activate flood monitoring with a TIB weighting factor; (iv) activate transaction flow control with a delay time factor; (v) disable TPS diagnostic tracing and TPS dumps; and (vi) start terminating idle transactions more frequently.

[0128] Now turn Figure 5C Flowchart 500C (entry point B, from Figure 5A 500A continued), processing continues at decision 530 where the RM 601 determines whether the first TPS 421 is in a “return to ready” mode.

[0129] If the RM 601 determines that the first TPS 421 is not in the "Return Ready" mode (decision 530, "No" branch), processing proceeds to decision 531 where the RM 601 determines whether the first TPS 421 is in the "Restricted" mode.

[0130] If the RM 601 determines that the first TPS 421 is in “restricted” mode (decision 531 , “yes” branch), processing proceeds to decision 532 , where the RM 601 determines whether a notification has been sent from the second TPS 422 .

[0131] If RM 601 determines that the notification was sent from second TPS 422 (decision 532 , “yes” branch), processing proceeds to decision 533 , where RM 601 determines whether second TPS 422 is the only unavailable TPS in TPS group 425 .

[0132] If RM 601 determines that second TPS 422 is not the only TPS in TPS group 425 that remains unavailable (decision 533, "no" branch), in other words, at least one other member of TPS group 425 is unavailable, the process ends.

[0133] Returning to decision 530 , if the RM 601 determines that the first TPS 421 is in “return ready” mode (decision 530 , “yes” branch), the RM 601 ignores the notification.

[0134] Returning to decision 531 , if the RM 601 determines that the first TPS 421 is not in “restricted” mode (decision decision 531 , “no” branch), the RM 601 ignores the notification.

[0135] Returning to decision 532 , if the RM 601 determines that the notification was not sent from the second TPS 422 (decision 532 , “no” branch), processing continues at operation 535 .

[0136] Returning to decision 533 , if RM 601 determines that the second TPS 422 is the only TPS in TPS group 425 that remains unavailable (decision 533 , “yes” branch), in other words, no other member of TPS group 425 is unavailable, processing continues at operation 535 .

[0137] At operation 535, RM 601 performs the following tasks, not necessarily in the following order: (i) deactivates the lightweight TIB functionality; cancels the tasks used to make the existing TIB lighter (consuming less system resources); (ii) disables flood monitoring using the TIB weighting factor; (iii) disables transaction flood control using the delay time factor; (iv) activates TPS diagnostic tracing and TPS dump; and (v) resumes terminating idle transactions at a normal frequency.

[0138] Now turn Figure 5D Flowchart 500D (entry point C, from Figure 5A 500A continues), processing continues at decision 540, where the RM 601 determines whether the first TPS 421 (the transaction processing system that initiated the failure notification) is a TPS group (e.g., TPS group 425; see Figure 4 ) part.

[0139] If the RM 601 determines that the first TPS 421 is part of a TPS group (decision 540 , “yes” branch), the RM 601 sends an “unavailable” notification regarding the first TPS 421 to the surviving TPS members of the TPS group 425 .

[0140] Now turn Figure 5E Flowchart 500E (entry point D, from Figure 5A 500A continues), processing continues at decision 550, where the RM 601 determines whether the first TPS 421 (the transaction processing system that initiated the failure notification) is a TPS group (e.g., TPS group 425; see Figure 4 ) part.

[0141] If the RM 601 determines that the first TPS 421 is part of the TPS group (decision 550 , “yes” branch), the RM 601 sends an “available” notification regarding the first TPS 421 to the surviving TPS members of the TPS group 425 .

[0142] The following paragraphs give examples of various use cases: (i) catastrophic system failure of one or more members of a TPS group; (ii) a single TPS system flooded with transactions; (iii) a TPS group with heavier than standard system resource usage; and (iv) a large transaction processing system with a single TPS or TPS group where it is desirable to save operating costs by reducing system resource usage.

[0143] (i) Catastrophic system failure of one or more members of the TPS group.

[0144] For a large banking client with multiple transaction processing systems (TPS) installed, one or more TPSs may stop due to reasons including transaction flooding, transaction routing issues, long I / O waits, or internal latching issues. Before the failed TPS completes the system restart, the surviving TPSs in the group may quickly run out of system resources due to the rerouted transaction workload from the failed TPS. In the case where a TPS Resource Manager (RM) is installed, the RM detects a catastrophic system failure. The RM performs internal actions on the surviving TPS group members to switch in and out of various RM processing modes, including "constraint mode", to save system resources, thereby avoiding further catastrophic outages of the remaining TPS group members. Therefore, the TPS group with the RM installed can avoid a series of catastrophic outages.

[0145] (ii) A single TPS system full of transactions .

[0146] For a small bank customer with a single TPS installed, millions of transactions from mobile devices may be submitted to the TPS in a short period of time. The TPS may then be in a severe flooding condition, and when the TPS runs out of system resources, the TPS may crash. In some embodiments of the present invention, the system automatically dynamically activates the RM "constraint mode" in response to reaching a defined flood threshold to reduce system resources. The RM actively acts to prevent catastrophic failure of any member of the TPS group. In some embodiments, once the transaction flood subsides, the system automatically dynamically deactivates the RM "constraint mode". In some embodiments, the RM provides a means for human intervention to activate and / or deactivate the RM or specific components thereof.

[0147] (iii) TPS groups with heavier than standard system resource usage .

[0148] In some embodiments of the present invention, for example, at a large banking client with multiple TPSs operating as a TPS group, a prioritized TPS group member receives the majority of the transaction workload from various gateways. If system resources run low on the prioritized TPS group member (e.g., resource usage levels exceed an action threshold level), the system automatically and dynamically activates RM "constraint mode" for some or all TPS group members to reduce system resource usage. If system resource usage levels fall below the action threshold (e.g., return to a standard level), the RM automatically deactivates "constraint mode." Through this functionality, the RM prevents members of the TPS group from unexpected catastrophic system failures.

[0149] (iv) Large transaction processing systems with a single TPS, or those that wish to save by reducing system resource usage TPS group of operating costs .

[0150] Large industrial TPSs (e.g., those focused primarily on customer analytics) can use the RM to run full-time in "constrained mode" to reduce overall system resource overhead. Running the RM in this manner can result in lower operating costs associated with the TPS. In some embodiments, the RM is automatically activated during system initialization or restart. Some embodiments provide a means for manually activating the RM.

[0151] The corresponding structures, materials, actions and equivalents of all parts or steps plus function elements in the following claims are intended to include any structure, material or action for performing a function in combination with other claimed elements as specifically claimed. The description of the present disclosure has been presented for the purpose of illustration and description, but it is not intended to be exhaustive or limited to the disclosure of the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the present disclosure. The embodiments are selected and described in order to best explain the principles and practical applications of the present disclosure and to enable others of ordinary skill in the art to understand the disclosure of various embodiments with various modifications suitable for the specific purposes envisioned.

[0152] IV. Definitions

[0153] The present invention: should not be taken as an absolute indication that the subject matter described by the term "the present invention" is covered by the claims as filed, or by claims that may eventually issue subsequent to the filing of a patent application; while the term "the present invention" is used to help the reader gain a general sense as to how the disclosure herein is believed to be potentially new, that understanding, as indicated by the use of the term "the present invention," is tentative and provisional and is subject to change during the course of patent prosecution as relevant information develops and the claims are potentially amended.

[0154] Examples: See definition of "the present invention" - similar caveats apply to the term "examples".

[0155] And / or: inclusive or; for example, A, B and / or C means that at least one of A or B or C is true and applicable.

[0156] Including / include / includes: Unless expressly stated otherwise, this means “including but not necessarily limited to”.

[0157] User / Subscriber: includes, but is not necessarily limited to: (i) a single individual person; (ii) an artificial intelligence entity with sufficient intelligence to act as a user or subscriber; and / or (iii) a group of related users or subscribers.

[0158] Data Communications: Any type of data communications scheme now known or developed in the future, including wireless communications, wired communications, and communications routes having both wireless and wired portions; data communications are not necessarily limited to: (i) direct data communications; (ii) indirect data communications; and / or (iii) data communications in which the format, packetization state, medium, encryption state, and / or protocol remain constant throughout the data communications.

[0159] Receive / Provide / Send / Input / Output / Report: Unless expressly specified otherwise, these words should not be taken to imply: (i) any particular degree of directness as to the relationship between their object and subject; and / or (ii) the absence of intervening components, actions and / or things between their object and subject.

[0160] No substantial human intervention: A process that occurs automatically (typically through the operation of machine logic, such as software) with little or no human input; some examples of "no substantial human intervention" include: (i) a computer is performing complex processing, and due to a power outage in the grid, a human switches the computer to an alternative power source, allowing processing to continue uninterrupted; (ii) a computer is performing resource-intensive processing, and a human confirms that the resource-intensive processing should indeed be performed (in this case, the confirmation process considered in isolation is with substantial human intervention, but the resource-intensive processing does not include any substantial human intervention, although a simple yes-no style confirmation by a human is required); and (iii) using machine logic, a computer has made a weighted decision (e.g., a decision to land all aircraft in anticipation of severe weather), but the computer must obtain a simple yes-no style confirmation from a human source before implementing the weighted decision.

[0161] Automatically: without any human intervention.

[0162] Module / Sub-module: Any collection of hardware, firmware, and / or software that works operatively to perform a function, regardless of whether the module is: (i) in a single localized proximity; (ii) distributed over a wide area; (iii) in a single proximity within a larger piece of software code; (iv) located within a single piece of software code; (v) located in a single storage device, memory, or media; (vi) mechanically connected; (vii) electrically connected; and / or (viii) connected in a data communication manner.

[0163] Computer: Any device with significant data processing and / or machine-readable instruction reading capabilities, including but not limited to desktop computers, mainframe computers, laptop computers, field programmable gate array (FPGA)-based devices, smart phones, personal digital assistants (PDAs), body-mounted or embedded computers, embedded device-type computers, and / or application-specific integrated circuit (ASIC)-based devices.

Claims

1. A computer-implemented method comprising: Determining a first unavailable state for a first TPS member in a group of transaction processing system TPS members; In response to determining the first state of the first TPS member, broadcasting a first message to a second TPS member in the group of TPS members, wherein the first message includes information about the first state of the first TPS member; and In response to receiving the first message, the second TPS member implements a resource usage reduction action, wherein the resource usage reduction action is selected from the group consisting of: creating a first transaction instance block (TIB) excluding an on-demand portion; establishing a flood monitor based on the number of TIBs that respectively exclude the corresponding on-demand portion; delaying creation of a second TIB based on a delay time factor; Disable creation of first diagnostic trace; Disabling the first system dump; and increasing the frequency with which the first plurality of idle transactions are canceled.

2. The method according to claim 1, further comprising: determining an available second status for the first TPS member; In response to determining the second state of the first TPS member, broadcasting a second message to the second TPS member, wherein the second message includes information about the second state of the first TPS member; and In response to receiving the second message, the resource usage reduction action is revoked by the second TPS member.

3. The method of claim 2, wherein undoing the resource usage reduction action is selected from the group consisting of: making a third TIB including an on-demand portion; creating a fourth TIB based on the corresponding transaction request; enabling creation of a second diagnostic trace; Enable secondary system dump; and reduce the frequency of canceling the second most idle transactions.

4. The method according to claim 1, further comprising: A plurality of instances of a resource manager are installed, including a first instance and a second instance, wherein each instance of the resource manager is associated with a TPS member corresponding to the TPS member group.

5. The method of claim 1 , wherein determining the first status of the first TPS member further comprises: Performance characteristics of the first TPS member are monitored.

6. The method of claim 5 , wherein the performance characteristic is selected from the group consisting of: rate of incoming transactions; time to process a transaction; clock frequency of a central processing unit (CPU); frequency of error occurrences; number of error occurrences; rate of error occurrences; CPU utilization metrics; memory access metrics; storage device access metrics; and network communication parameters.

7. A computer program product comprising: One or more computer-readable storage media, and program instructions collectively stored on the one or more computer-readable storage media, the program instructions comprising instructions programmed to execute the method according to any one of claims 1 to 6.

8. A computer system comprising: processor sets; as well as one or more computer-readable storage media; in: The processor set is constructed, located, connected and / or programmed to execute program instructions stored on the one or more computer-readable storage media; as well as The program instructions include instructions programmed to perform the method according to any one of claims 1 to 6.

Images