Intelligent storage of messages in page sets

Abstract

Intelligent storage of messages in page sets Intelligent storage of messages is provided. A spillover page set is selected for storing incoming messages corresponding to a predefined target page set associated with an application workload in response to the predefined target page set reaching a predefined lowest unused page threshold level. The spillover page set is utilized as a message storage destination for incoming messages corresponding to a predefined target page set associated with an application workload to expand message storage for the predefined target page set after the predefined target page set reaches a predefined lowest unused page threshold level.

Description

[Technical field]

[0001] The present disclosure relates generally to page sets, and more particularly to intelligent storage of a message in a spillover page set having available unused pages in response to the number of unused pages in the message's original predefined target page set reaching a predefined minimum unused page threshold level. [Background technology]

[0002] A page set is used to physically store messages such as data sets on a direct access storage device (DASD), for example, a disk drive. A page set is a virtual storage access method that is specially formatted for DASD data sets. Each page set is identified by a page set identifier. There are architectural limitations on page sets. For example, the page set data storage limit may be 64 gigabytes. Each page set is associated with a buffer pool.

[0003] Some current message engines have messages stored in page sets on disk. These current message engines can have an extensive area to store messages, but the input / output (I / O) cost to retrieve those messages is high. Other current message engines exist that can cache hot pages (e.g., pages that are often requested for a short amount of time) in memory, and messages are consumed without performing any I / O operations. These other current message engines store messages in multiple levels of message storage, depending on the performance and cost of storage. For example, short-lived messages (e.g., messages that are retrieved from the queue immediately after receiving the message) may be stored only in a buffer, while long-lived messages are stored in message storage formatted as a page set. However, these current message engines have problems filling certain page sets. Summary of the Invention

[0004] According to an exemplary embodiment, a computer-implemented method for intelligent storage of messages is provided. The computer selects a spillover page set for storing incoming messages corresponding to a predefined target page set associated with an application workload in response to the predefined target page set reaching a predefined lowest unused page threshold level. The computer utilizes the spillover page set as a message storage destination for incoming messages corresponding to a predefined target page set associated with an application workload, such that the computer expands message storage for the predefined target page set after the predefined target page set reaches a predefined lowest unused page threshold level. According to another exemplary embodiment, a computer system and computer program product for intelligent storage of messages are provided. As a result, the exemplary embodiment can automatically expand message storage to a spillover page set having available unused pages to maintain execution of an application workload when a predefined target page set corresponding to the application workload reaches a defined lowest unused page threshold level. Thus, by expanding the message storage capacity of a page set, an illustrative embodiment provides a scale-out page set process that protects the buffers, thereby ensuring that queues that receive sudden arrivals of many messages do not adversely affect the performance of priority queues.

[0005] The illustrative embodiments also optionally determine whether a scale-out bit is set for the predefined target page set. As a result, the illustrative embodiments by setting the scale-out bit for the page set can keep application workloads running without outages caused by a lack of message storage at the predefined target page set.

[0006] Preferred embodiments of the present invention will now be described, by way of example only, with reference to the following drawings, in which: [Brief description of the drawings]

[0007] [Figure 1] 1 is a pictorial representation of a network of data processing systems in which an illustrative embodiment may be implemented. [Diagram 2] 1 is an illustration of a data processing system in which an illustrative embodiment may be implemented; [Diagram 3] FIG. 1 illustrates an example of a page set management system according to an exemplary embodiment. [Figure 4] FIG. 2 illustrates an example of a page set management process according to an exemplary embodiment. [Diagram 5] FIG. 1 illustrates an example of a page set list according to an exemplary embodiment. [Figure 6] 1 is a flowchart illustrating a process for expanding page set storage of messages in accordance with an example embodiment. [Figure 7] 1 is a flowchart illustrating a process for intelligent storage of messages in a page set in accordance with an example embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] The present invention may be a system, method, and / or computer program product, in any possible level of technical detail integration, and may include one or more computer-readable storage media having computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0009] A computer readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer readable storage media includes the following: portable computer diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or grooved structures on which instructions are recorded, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, should not be construed as being ephemeral signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted over electrical wires.

[0010] The computer readable program instructions described herein may be downloaded from a computer readable storage medium to an individual computing / processing device or to an external computer or storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface of each computing / processing device receives the computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the individual computing / processing device.

[0011] The computer readable program instructions for carrying out the operations of the present invention may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state configuration data, configuration data for integrated circuits, or object oriented programming languages ​​such as Smalltalk®, C++, and procedural programming languages ​​such as the “C” programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer as a standalone software package, partly on the user's computer and partly on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be made to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer readable program instructions to individualize the electronic circuitry by utilizing state information of the computer readable program instructions to perform aspects of the invention.

[0012] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0013] These computer readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine, such that the instructions, which execute on the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer readable program instructions may also be stored on a computer readable storage medium having instructions stored thereon to instruct a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer readable storage medium having the instructions stored thereon comprises an article of manufacture including instructions for implementing aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0014] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause the computer, other programmable apparatus, or other device to perform a series of operable steps to create a computer-implemented process such that the instructions, which execute on the computer, other programmable apparatus, or other device, implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0015] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions shown in the blocks may occur out of the order shown in the figures. For example, two blocks shown in succession may actually be performed as one step, or may be executed simultaneously, substantially simultaneously, partially or fully in a time-overlapping manner, or the blocks may sometimes be executed in reverse order depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a special-purpose hardware-based system that performs the specified functions or actions or executes a combination of special-purpose hardware and computer instructions.

[0016] Referring now to the figures, and in particular to Figures 1-3, diagrams of data processing environments are provided in which example embodiments may be implemented. It should be understood that Figures 1-3 are intended as examples only and are not intended to assert or suggest any limitation with respect to the environments in which different embodiments may be implemented. Many modifications to the depicted environments may be made.

[0017] 1 illustrates a diagrammatic representation of a network of data processing systems in which exemplary embodiments may be implemented. Network data processing system 100 is a network of computers, data processing systems, and other devices in which exemplary embodiments may be implemented. Network data processing system 100 includes network 102, which is the medium used to provide communications links between the computers, data processing systems, and other connected devices in network data processing system 100. Network 102 may include connections such as, for example, wired communications links, wireless communications links, fiber optic cables, and the like.

[0018] In the illustrated example, server 104 and server 106 are connected to network 102 along with storage 108. Server 104 and server 106 may be, for example, server computers with high speed connections to network 102. Server 104 and server 106 may also each represent a cluster of servers in one or more data centers. Alternatively, server 104 and server 106 may each represent multiple computing nodes in one or more cloud environments.

[0019] Additionally, the servers 104 and 106 provide a set of services to users of client devices, such as, for example, event monitoring services, transaction services, financial services, banking services, government services, education services, reservation services, data services, or the like. Additionally, the servers 104 and 106 perform intelligent storage of messages corresponding to application workloads of the services provided by the servers 104 and 106 across page sets managed by the servers 104 and 106. For example, the servers 104 and 106 store messages corresponding to application workloads in a spillover page set having available unused pages in response to a number of unused pages of a predefined target page set reaching a predefined minimum unused page threshold level. The predefined minimum unused page threshold level may be, for example, 500, 1000, 1500, 2000, 2500, or a similar number of unused pages in a page set.

[0020] Client 110, client 112, and client 114 are also connected to network 102. Clients 110, 112, and 114 are client devices of server 104 and server 106. In this example, clients 110, 112, and 114 are shown as desktop or personal computers with wired communication links to network 102. However, it should be noted that clients 110, 112, and 114 are merely examples and may represent other types of data processing systems, such as, for example, network computers, laptop computers, handheld computers, smart phones, smart watches, smart televisions, smart vehicles, smart appliances, gaming devices, kiosks, and the like, having wired or wireless communication links to network 102. Users of clients 110, 112, and 114 may utilize clients 110, 112, and 114 to access and utilize the set of services provided by server 104 and server 106. Clients 110, 112, and 114 send messages to server 104 and server 106 that correspond to the application of a set of services.

[0021] The servers 104 and 106 store messages corresponding to the application of the set of services in page sets located on the servers 104 and 106 and / or the storage 108. The storage 108 is a network storage device that can store any type of data in a structured or unstructured format. In addition, the storage 108 may represent multiple network storage devices. Further, the storage 108 may store multiple different page sets corresponding to different application workloads associated with different services, identifiers of each respective page set, page set lists, and the like. Furthermore, the storage 108 may store other types of data, such as authentication data or sensitive data, which may include, for example, usernames, passwords, and the like associated with system administrators and users of client devices.

[0022] Additionally, it should be noted that network data processing system 100 may include any number of additional servers, clients, storage devices, and other devices not shown. The program code located in network data processing system 100 may be stored on a computer readable storage medium or set of computer readable storage media and downloaded to a computer or other data processing device for use. For example, the program code may be stored on a computer readable storage medium on server 104 and downloaded to client 110 over network 102 for use on client 110.

[0023] In the depicted example, network data processing system 100 may be implemented as a number of different types of communications networks, such as, for example, the Internet, an intranet, a wide area network, a local area network, a telecommunications network, or any combination thereof. Figure 1 is intended as an example only, and not as architectural limitations for the different illustrative embodiments.

[0024] As used herein, when used in reference to an element, "some" means one or more of the elements. For example, "several different types of communication networks" refers to one or more different types of communication networks. Similarly, "a set of" when used in reference to an element means one or more of the elements.

[0025] Furthermore, the term "at least one of," when used in reference to a list of elements, means one or more different combinations of the listed elements, and only one of each element in the list may be required. In other words, "at least one of" means that any combination of elements and any number of elements from the list may be used, but not all of the elements in the list are required. An element may be a particular object, thing, or category.

[0026] For example, without limitation, "at least one of element A, element B, or element C" may include element A, element A and element B, or element B. This example may also include element A, element B, and element C, or element B and element C. Of course, any combination of these elements may also be present. In some illustrative examples, "at least one of" may be, for example, without limitation, 2 elements A, 1 element B, and 10 elements C, 4 elements B, and 7 elements C, or other suitable combinations.

[0027] Referring to Figure 2, a diagram of a data processing system is shown in accordance with an exemplary embodiment. Data processing system 200 is an example of a computer in which computer readable program code or instructions implementing the intelligent message storage process of an exemplary embodiment may be located, such as server 104 of Figure 1. In this example, data processing system 200 includes a communications fabric 202 that provides communications between a processor unit 204, a memory 206, a persistent storage 208, a communications unit 210, an input / output (I / O) unit 212, and a display 214.

[0028] The processor unit 204 is responsible for executing instructions for software applications and programs that may be loaded into the memory 206. The processor unit 204 may be a set of one or more hardware processor devices or may be a multi-core processor, depending on the particular implementation.

[0029] Memory 206 and persistent storage 208 are examples of storage devices 216. As used herein, a computer-readable storage device or computer-readable storage medium is any hardware capable of storing information, such as, but not limited to, data, computer-readable program code in functional form, or other suitable information or combinations thereof, on either a temporary or persistent basis. Additionally, computer-readable storage device or computer-readable storage medium excludes propagating media, such as transitory signals. Additionally, a computer-readable storage device or computer-readable storage medium may represent a set of computer-readable storage devices or a set of computer-readable storage media. In these examples, memory 206 may be, for example, a random access memory (RAM), or any other suitable volatile or non-volatile storage device, such as a flash memory. Persistent storage 208 may take various forms, depending on the particular implementation. Persistent storage 208 may include one or more devices. For example, persistent storage 208 may be a disk drive, a solid state drive, a rewritable optical disk, a rewritable magnetic tape, or some combination of the above. The medium used by persistent storage 208 may be removable. For example, a removable hard drive may be used for persistent storage 208.

[0030] In this example, persistent storage 208 stores page set manager 218. However, it should be noted that even though page set manager 218 is illustrated as residing in persistent storage 208, in alternative exemplary embodiments, page set manager 218 may be a separate component of data processing system 200. For example, page set manager 218 may be a hardware component coupled to communications fabric 202, or a combination of hardware and software components. In another alternative exemplary embodiment, a first set of components of page set manager 218 may be located in data processing system 200, and a second set of components of page set manager 218 may be located in a second data processing system, such as, for example, server 106 of FIG. 1.

[0031] The page set manager 218 controls the process of intelligently storing messages across page sets corresponding to application workloads of services provided by the data processing system 200. The page set manager 218 monitors the status of each respective page set to detect when the number of unused pages of a predefined target page set for messages corresponding to an application workload reaches a predefined minimum unused page threshold level. In response to detecting, based on the monitoring, that the number of unused pages of a particular predefined target page set has reached a predefined minimum unused page threshold level, the page set manager 218 selects a first spillover page set listed in the list of available spillover page sets as a destination for messages corresponding to that particular predefined target page set that has reached the predefined minimum unused page threshold level. The page set manager 218 then stores messages corresponding to that particular predefined target page set in the selected spillover page set that has unused pages available for storing the messages.

[0032] As a result, data processing system 200 operates as a special purpose computer system where page set manager 218 within data processing system 200 enables intelligent storage of messages in page sets. In particular, page set manager 218 transforms data processing system 200 into a special purpose computer system as compared to currently available general purpose computer systems that do not have page set manager 218.

[0033] The communications unit 210 facilitates communication with other computers, data processing systems, and devices over a network, such as the network 102 of FIG. 1. The communications unit 210 may provide communication through the use of both physical and wireless communications links. The physical communications links may utilize wires, cables, universal serial buses, or any other physical technology for establishing a physical communications link to the data processing system 200. The wireless communications links may utilize short wave, high frequency, very high frequency, microwave, Wireless Fidelity (WiFi), Bluetooth® technology, Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), second generation (2G), third generation (3G), fourth generation (4G), 4G Long Term Evolution (LTE), LTE Advanced, fifth generation (5G), or any other wireless communications technology or standard for establishing a wireless communications link to the data processing system 200.

[0034] Input / output unit 212 allows for the input and output of data with other devices that may be connected to data processing system 200. For example, input / output unit 212 may provide a connection for user input through a keypad, keyboard, mouse, microphone, or other suitable input device, or combination thereof. Display 214 provides a mechanism for displaying information to a user and may include touch screen capability to allow the user to make on-screen selections, for example, through a user interface or input data.

[0035] Instructions for the operating system, applications, and / or programs may be located in storage devices 216 that communicate with processor unit 204 through communications fabric 202. In this illustrative example, the instructions have a functional form on persistent storage 208. These instructions may be loaded into memory 206 for execution by processor unit 204. The processes of the different embodiments may be performed by processor unit 204 using computer-implemented instructions that may be located in a memory, such as memory 206. These program instructions are referred to as program code, computer usable program code, or computer readable program code, which may be read and executed by a processor in processor unit 204. The program instructions may be embodied on different physical computer readable storage devices, such as memory 206 or persistent storage 208, in different embodiments.

[0036] The program code 220 is located in functional form on a selectively removable computer readable medium 222 and may be loaded or transferred to the data processing system 200 for execution by the processor unit 204. The program code 220 and the computer readable medium 222 form a computer program product 224. In one example, the computer readable medium 222 may be a computer readable storage medium 226 or a computer readable signal medium 228.

[0037] In these illustrative examples, computer readable storage medium 226 is a physical or tangible storage device used to store program code 220, rather than a medium that propagates or transmits program code 220. Computer readable storage medium 226 may include, for example, an optical or magnetic disk inserted or placed into a drive or other device that is part of persistent storage 208 for transfer to a storage device, such as a hard drive that is part of persistent storage 208. Computer readable storage medium 226 may also take the form of persistent storage, such as a hard drive, thumb drive, or flash memory connected to data processing system 200.

[0038] Alternatively, program code 220 may be transferred to data processing system 200 using computer readable signal media 228. Computer readable signal media 228 may be, for example, a propagated data signal containing program code 220. For example, computer readable signal media 228 may be an electromagnetic signal, an optical signal, or any other suitable type of signal. These signals may be transmitted over communications links, such as wireless communications links, fiber optic cable, coaxial cable, a wire, or any other suitable type of communications link.

[0039] Additionally, as used herein, "computer readable medium 222" may be singular or plural. For example, program code 220 may be located on computer readable medium 222 in the form of a single storage device or system. In another example, program code 220 may be located in computer readable medium 222 that is distributed across multiple data processing systems. In other words, some instructions in program code 220 may be located in one data processing system, while other instructions in program code 220 may be located in one or more other data processing systems. For example, a portion of program code 220 may be located on computer readable medium 222 in a server computer, while another portion of program code 220 may be located on computer readable medium 222 located in a set of client computers.

[0040] The different components illustrated for data processing system 200 are not meant to provide architectural limitations to the manner in which different embodiments may be implemented. In some illustrative examples, one or more of the components may be incorporated into or otherwise form a part of another component. For example, memory 206, or a portion thereof, may be incorporated into processor unit 204 in some illustrative examples. The different illustrative embodiments may be implemented in a data processing system including components in addition to or in place of the components illustrated for data processing system 200. Other components illustrated in FIG. 2 may vary from the illustrated illustrative example. The different embodiments may be implemented using any hardware device or system capable of executing program code 220.

[0041] In another example, a bus system may be used to implement communications fabric 202 and may be comprised of one or more buses, such as a system bus or an input / output bus. Of course, the bus system may be implemented using any suitable type of architecture that provides for a transfer of data between different components or devices attached to the bus system.

[0042] Currently, page sets have additional capacity to "scale out," but they are not intelligent enough to use existing free space (e.g., unused pages) in a page set. Current queue managers use buffers to perform message caching and can write updated pages from the buffer to other message storage, e.g., page sets, disk, or tape. These current queue managers benefit from buffers that are fast to write messages to and retrieve messages from, but also benefit from page sets that allow for larger storage capacity. However, the exemplary embodiment extends the storage capacity of page sets by providing a scale-out page set process, while protecting existing buffers, so that queues that receive sudden arrivals of many messages do not adversely affect the performance of "important" or "priority" queues, such as system queues or high priority application queues. The bottleneck is that the message retriever is far away from the message putter, or the message retriever fails when the message putter does not fail.

[0043] Exemplary embodiments continue to store messages when a predefined target page set corresponding to an application workload becomes full, preventing the application from crashing. Exemplary embodiments incorporate high performance message storage and can intelligently scale out across multiple levels of message storage. Exemplary embodiments utilize a page set manager to detect when a page set reaches a predefined minimum unused page threshold level, maintain a list of page sets with available unused pages from which the page set manager can select a particular page set to continue storing messages for the full or nearly full page set, and assign an extended storage class corresponding to the selected page set to the queue. Each page set has a bit that the page set manager can set to indicate whether the particular page set is available to be used as a message storage destination for other page sets that have reached a predefined minimum unused page threshold level for scaling out (e.g., spillover).

[0044] The exemplary embodiment also utilizes multi-dwell queues as a logical unit of message storage. Each queue in a multi-dwell queue has its own predefined storage class and extended storage class (e.g., multi-dimensional storage class), and the exemplary embodiment automatically updates the message storage destination for that particular queue when it reaches a predefined maximum message storage threshold level. Each storage class maps to one page set that stores messages. In other words, queues refer to storage classes, storage classes refer to page sets, and page sets store messages.

[0045] Each queue has a bit that the page set manager of an exemplary embodiment can set to indicate whether that particular queue is capable of being scaled out to other page sets. The page set manager utilizes spillover page sets to utilize available storage space (e.g., unused pages in the spillover page set). Each spillover page set contains a message location index that includes an identifier of the spillover page set to read messages or return messages to a buffer when messages corresponding to an entire page set are stored across multiple spillover page sets. The page set manager utilizes the index to locate, read, and return a particular message to a buffer when the message may be stored in one of many spillover page sets.

[0046] It should be noted that the exemplary embodiments can keep the allocation or selection of buffer pools under user control to ensure that high priority queues requiring optimal response time are separated from queues where capacity is important when reclaiming buffer pool space is not desired. Additionally, the exemplary embodiments can utilize "real storage" as needed. For example, upon entering buffer scale-out mode, the page set manager of the exemplary embodiments can query the operating system to determine the amount of unused real storage available and dynamically allocate that amount of unused real storage to the scale-out buffers while maintaining some predefined percentage of unused real storage. The exemplary embodiments can then free that allocated amount of real storage when it is no longer needed. As a result, the exemplary embodiments allow for increased performance from the buffer pools without impacting existing buffer pool utilization and multi-level message storage, and without causing application corruption by stealing real storage, which can cause paging.

[0047] The page set manager of the exemplary embodiment can dynamically expand the spillover page set depending on the historical peak workload of the service application, and then revert the queue to the original target page set when the peak workload period ends. Each application workload has a defined page set limit. The page set manager sets a scale-out bit for the page set that is enabled to expand message storage into the spillover page set. In this manner, the exemplary embodiment by setting the scale-out bit for the page set can keep the application workload running without outages caused by a lack of message storage at the original predefined target page set.

[0048] As a result, even if the number of page sets and buffer pools are predefined, the exemplary embodiments can automatically expand message storage to a spillover page set that has available unused pages to keep an application workload running when the original predefined target page set corresponding to that application workload reaches a defined minimum unused page threshold level. The exemplary embodiments can then automatically revert to the original target page set corresponding to the application workload when the extended message storage on the spillover page set is no longer needed (e.g., after a peak workload period ends, the number of unused pages in the original predefined target page set is greater than the predefined unused page threshold level by a defined percentage amount, etc.). In this way, the exemplary embodiments achieve scaling across multiple levels of message storage (e.g., buffer pools for performance and page sets for capacity) and balance storage efficiency with storage capacity.

[0049] Accordingly, the illustrative embodiments provide one or more technical solutions that overcome technical problems related to storage capacity utilization in a page set to prevent application workload crashes. As a result, these one or more technical solutions provide technical effects and practical applications in the field of data storage.

[0050]

[0023] Referring now to Figure 3, a diagram illustrating an example of a page set management system is depicted in accordance with an exemplary embodiment. Page set management system 300 may be implemented in a network of data processing systems, such as network data processing system 100 of Figure 1. Page set management system 300 is a system of hardware and software components for intelligent storage of messages corresponding to application workloads in page sets.

[0051] In this example, page set management system 300 includes buffer pool 302 and page set 304. Buffer pool 302 and page set 304 may be implemented in a computer such as, for example, server 104 of Figure 1 or data processing system 200 of Figure 2. Buffer pool 302 may be implemented in a memory such as, for example, memory 206 of Figure 2. Page set 304 may be implemented in persistent storage such as, for example, persistent storage 208 of Figure 2.

[0052] Buffer pool 302 includes multiple individual buffer pools, such as buffer pool 306. Buffer pool 306 temporarily stores messages 312 in a buffer, such as buffer 308 or buffer 310, before messages 312 are stored in page set 304. Messages 312 are received from client devices, such as clients 110, 112, and 114 of FIG. 1, over a network, such as network 102 of FIG. 1. Messages 312 correspond to a set of application workloads associated with a set of services provided by the computer. Messages may include, for example, message descriptors, application data, and the like, corresponding to a particular application workload of a given service. Each application workload has a predefined target page set as a message storage destination.

[0053] Buffer 308 and buffer 310 may be, for example, 4 kilobyte buffers. Each buffer pool in buffer pools 302 is associated with (e.g., mapped to) a particular page set, such as page set X 314, page set Y 316, or page set Z 318, of page sets 304. Alternatively, a set of buffer pools may be mapped to a single page set.

[0054] For example, a page set manager, such as page set manager 218 of Figure 2, may scale out using a spillover page set (e.g., page set Z 318) as a message storage destination for messages originally mapped to a predefined target page set (e.g., page set Y 316) until the original predefined target page set reaches or falls below a predefined minimum unused page threshold level and has its scale-out bit set. The page set scale-out bit, when set, allows that particular page set to scale out or expand its message storage to a spillover page set that has available unused pages.

[0055] The page set manager of an example embodiment can utilize an addressing rule, such as pageset.page.record, for page set 304. Currently, for a single queue, the page set portion of the addressing rule remains the same. However, if there is not enough storage space in the original predefined target page set for a particular application workload, the page set manager can create a different page set portion of the addressing rule for messages on a single queue or for a single message. The page set manager automatically updates the page set portion of the "pageset.page.record" addressing rule as needed (e.g., when the original predefined target page set reaches a predefined minimum unused page threshold level).

[0056] In this way, the exemplary embodiments ensure that the sudden arrival of a large number of messages does not adversely affect the performance of critical or high priority queues, such as, for example, system queues or high priority application queues. The exemplary embodiments do not reclaim buffer pool space because they keep the allocation or selection of buffer pools under user control, ensuring that high priority queues requiring the shortest possible response times are separated from queues where capacity is needed.

[0057] 4, a diagram illustrating an example of a page set management process is shown, in accordance with an exemplary embodiment. The page set management process 400 may be implemented in a computer, such as, for example, the server 104 of FIG. 1 or the data processing system 200 of FIG. 2.

[0058] In this example, page set management process 400 includes a page set manager 402, a buffer pool 404, and a page set 406. Page set manager 402 may be, for example, page set manager 218 of FIG. 2. Buffer pool 404 and page set 406 may be, for example, buffer pool 306 and page set X 314 of FIG. 3. However, it should be noted that page set management process 400 is intended as an example only and not as a limitation on the exemplary embodiment. In other words, page set management process 400 may include any number of page set managers, buffer pools, page sets, and other components.

[0059] The computer receives a message 408, such as from a client device, such as client 110 of FIG. 1. The computer temporarily places the message 408 in a buffer pool 404 before loading the message 408 into a page set 406. The page set 406 includes a predefined buffer pool identifier 410 and a scale-out bit 412. The predefined buffer pool identifier 410 represents a unique identifier corresponding to the buffer pool 404. In other words, the buffer pool 404 is a predefined buffer pool for the page set 406. The page set 406 is an original predefined target page set for the message 408. The scale-out bit 412 indicates whether the page set 406 can be scaled out to a spillover page set if the number of unused pages in the page set 406 reaches a predefined minimum unused page threshold level.

[0060] At 414, the page set manager 402 performs a scale-out process to detect whether the scale-out bit 412 is set in the page set 406 in response to the number of unused pages in the page set 406 reaching a predefined minimum unused page threshold level. If the page set manager 402 detects that the scale-out bit 412 is set in the page set 406, the page set manager 402 searches the page set list 416 for a spillover page set that has unused pages as a scale-out message destination for the page set 406.

[0061] For example, at 418, a message 408 arrives corresponding to a predefined target page set that is an original target. At 420, the page set manager 402 determines whether a predefined lowest unused page threshold level has been reached for the original predefined target page set. If the page set manager 402 determines that the predefined lowest unused page threshold level has not been reached for the original predefined target page set, the page set manager 402 stores the message 408 in the original predefined target page set 422 (e.g., page set 406) via a predefined storage class 424. If the page set manager 402 determines that the lowest unused page threshold level has been reached for the original predefined target page set, then at 426, the page set manager 402 selects the first spillover page set at the top of the page set list 416. Additionally, at 428 , the page set manager 402 assigns the extended storage class 430 of the selected spillover page set to a queue 432 in a multiple dwell queue 434 .

[0062] A queue is a logical unit of message storage. The page set manager 402 opens the queue and puts a message on it. The page set manager 402 knows which queue it has put the message on. Queues are mapped to page sets via storage classes. In other words, the storage class controls which page sets are used by which queues. A storage class maps one or more queues to a page set. This means that messages for that queue are stored on that page set. Storage classes allow users to control where messages are stored for application workload management.

[0063] In this example, the scale-out bit for queue 432 is set, allowing queue 432 to be scaled out to other page sets. As a result, page set manager 402 stores message 408 in selected spillover page set 436 via extended storage class 430. Selected spillover page set 436 includes index 438, which is a message location index that includes, for example, a page set identifier. Page set manager 402 utilizes index 438 to read messages back into the buffer when messages corresponding to full or nearly full page sets 406 are stored across multiple spillover page sets.

[0064] 5, a diagram illustrating an example of a page set list is shown, according to an example embodiment. Page set list 500 may be implemented by a page set manager, such as, for example, page set manager 402 of FIG. 4. Page set list 500 may be, for example, page set list 416 of FIG.

[0065] In this example, the page set list 500 includes columns for a page set identifier (ID) 502, a buffer pool ID 504, a nearly full occurrence frequency 506, was the nearly full bit set recently 508, total pages 510, unused pages 512, a pointer to next scale-out page set 514, and assigned storage class 516. The page set ID 502 column provides a unique identifier for each respective page set managed by the page set manager. The buffer pool ID 504 column provides a unique identifier for the predefined buffer pool for each corresponding page set. The nearly full occurrence frequency 506 column provides an indication of the number of times a particular page set was close to full (e.g., reached a predefined minimum unused page threshold level). The was the nearly full bit set recently 508 column indicates with a yes (Y) or no (N) whether the bit indicating that a particular page set was close to full recently was set. The total pages 510 column indicates the total number of pages in each respective page set. The unused pages 512 column indicates the number of unused pages in each respective page set. The pointer to next scale-out page set 514 column provides a pointer to the next spillover page set for each respective page set. The assigned storage class 516 column indicates which storage class is assigned to each respective page set.

[0066] When the number of unused pages for a particular page set reaches a defined minimum number of unused page threshold levels, the page set manager searches the page set list 500 to identify another page set (e.g., a spillover page set) as a destination for messages corresponding to the original predefined target page set that reached that threshold level. Note that, to ensure that a queue receiving a sudden influx of a large number of messages does not adversely affect the performance of high priority queues, such as system queues or high priority application queues, each page set includes a bit that can be set to indicate whether the particular page set is available as a message destination for other page sets to scale out. In other words, the page set manager protects the performance of these high priority queues. Additionally, each respective queue includes a bit that the page set manager can set to indicate whether that particular queue is available to scale out to other page sets. In other words, this bit ensures that high priority queues that require optimal response time are separated from queues for which capacity is more critical. Additionally, each respective spillover page set contains a message location index, including a page set identifier, for retrieving or reading back into the buffer messages corresponding to the full original predefined target page set if those messages are stored across multiple spillover page sets. This index links messages for a queue that are stored in different page sets for that queue, so that the messages can be efficiently loaded back into the buffer.

[0067] The page set manager selects the page set listed first in the page set list 500 (e.g., the page set with page set ID 0 listed first in the page set list 500) as the storage destination for messages corresponding to the original predefined target page set that has reached the predefined lowest unused page threshold level. In addition, the page set manager assigns the extended storage class corresponding to the selected page set to a specific queue of the multiplexed dwell queue. Messages sent to that specific queue are stored in the newly selected page set in the queue via the extended storage class corresponding to the selected page set after the original predefined target page set has reached its threshold level. The page set manager checks the unused page status of each respective page set continuously or based on a defined time interval. In addition, the page set manager not only knows the current real-time utilization of each respective page set, but also knows the historical utilization of each respective page set per application workload for faster spillover when required.

[0068] The page set manager maintains the page set list 500 using a set of rules. For example, the page set list 500 includes only page sets available for spillover (e.g., page sets with a scale-out bit set). The page set manager sorts the page sets by the number of unused pages. For example, the page set manager places the identifier of the page set with the greatest number of unused pages at the top of the page set list 500. Furthermore, the page set manager moves the identifier of a particular page set to the bottom of the page set list 500 when the page set reaches a predefined minimum unused page threshold level. In addition, the page set manager marks the particular page set that is moved to the bottom of the list with a "Y" in the Recently Near Full Bit Set 508 column, and also increments the Near Full Occurrence 506 column by one for the particular page set each time the particular page set is moved to the bottom of the page set list 500. The page set manager also selects the page set that has the lowest number in the Near Full Occurrence column 506 when the number of unused pages is the same for each candidate spillover page set. Additionally, the page set manager can determine which page set to utilize by combining the real-time data in the page set list 500 with the historical utilization data corresponding to that page set.

[0069]

[0023] Referring now to Figure 6, a flow chart illustrating a process for expanding page set storage of messages is shown, according to an example embodiment. The process illustrated in Figure 6 may be implemented in a computer, such as, for example, server 104 of Figure 1 or data processing system 200 of Figure 2. For example, the process illustrated in Figure 6 may be implemented in page set manager 218 of Figure 2.

[0070] The process begins when the computer receives a message corresponding to a predefined target page set associated with an application workload of a service (step 602). In response to receiving the message, the computer identifies a number of unused pages in the predefined target page set based on information contained in a page set list (step 604).

[0071] The computer then makes a determination as to whether the number of unused pages in the predefined target page set is less than or equal to a predefined lowest unused page threshold level (step 606). If the computer determines that the number of unused pages in the predefined target page set is not less than or equal to the predefined lowest unused page threshold level, a "No" is output in step 606 and the computer stores the message in the predefined target page set of the predefined storage class that corresponds to the predefined target page set (step 608). The process then returns to step 602 and the computer waits to receive another message.

[0072] Returning again to step 606, if the computer determines that the number of unused pages in the predefined target page set is equal to or less than the predefined minimum unused page threshold level, a "yes" is output in step 606 and the computer selects a spillover page set from the top of the page set list (step 610). In addition, the computer assigns an extended storage class corresponding to the selected spillover page set to a queue in the multiple dwell queue (step 612). Furthermore, the computer stores the message in the selected spillover page set via the extended storage class in the queue (step 614). The process then returns to step 602 and the computer waits to receive another message.

[0073] Referring now to Figure 7, a flow chart illustrating a process for intelligent storage of messages in a page set is shown, according to an example embodiment. The process illustrated in Figure 7 may be implemented in a computer, such as, for example, server 104 of Figure 1 or data processing system 200 of Figure 2. For example, the process illustrated in Figure 7 may be implemented in page set manager 218 of Figure 2.

[0074] The process begins when a computer monitors the unused page status of each respective page set of a plurality of page sets managed by the computer (step 702). The computer may perform the monitoring of the unused page status of each respective page set, for example, on a continuous basis, on a predefined time interval basis, or on demand. The computer detects when a predefined target page set associated with an application workload of the service has reached a predefined minimum unused page threshold level based on the monitoring of the unused page status of each respective page set (step 704).

[0075] In response to detecting that the predefined target page set associated with the application workload has reached a predefined lowest unused page threshold level, the computer determines whether a scale-out bit is set for the predefined target page set (step 706). If the computer determines that the scale-out bit is not set for the predefined target page set, a "no" is output in step 706 and the process proceeds to step 712. If the computer determines that the scale-out bit is set for the predefined target page set, a "yes" is output in step 706 and the computer selects a spillover page set for storing an incoming message corresponding to the predefined target page set associated with the application workload in response to the predefined target page set reaching a predefined lowest unused page threshold level (step 708).

[0076] Additionally, the computer utilizes the spillover page set as a message storage destination for incoming messages corresponding to the predefined target page set associated with the application workload, so as to expand message storage for the predefined target page set after the predefined target page set reaches a predefined minimum unused page threshold level (step 710). The computer then detects that the number of unused pages of the predefined target page set associated with the application workload is greater than the predefined minimum unused page threshold level by a defined percentage amount based on monitoring the unused page status of each respective page set (step 712). The defined percentage amount may be, for example, 5%, 10%, 15%, 20%, 25%, etc., of the total number of pages of the predefined target page set. The computer then utilizes the predefined target page set as a message storage destination in response to detecting that the number of unused pages of the predefined target page set associated with the application workload is greater than the predefined minimum unused page threshold level by a defined percentage amount (step 714). The process then returns to step 702 to continue monitoring the unused page status of each respective page set.

[0077] Thus, exemplary embodiments of the present invention provide a computer-implemented method, computer system, and computer program product for intelligent storage of a message in a spillover page set having available unused pages when the number of unused pages in an original predefined target page set for that message reaches a predefined minimum unused page threshold level. The description of various embodiments of the present invention is presented for illustrative purposes, but is not intended to be exhaustive and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used herein have been selected to best explain the principles of the embodiments, practical applications or technical improvements over techniques found in the marketplace, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A computer-implemented method for intelligent storage of messages, comprising: selecting, by the computer, a spillover page set for storing incoming messages corresponding to a predefined target page set associated with an application workload in response to the predefined target page set reaching a predefined lowest unused page threshold level; 22. The computer-implemented method of claim 21, further comprising: utilizing, by the computer, the spillover pageset as a message storage destination for the received messages corresponding to a predefined target pageset associated with the application workload, such that the computer expands message storage for the predefined target pageset after the predefined target pageset reaches the predefined lowest unused page threshold level.

2. monitoring, by the computer, an unused page status of each page set within the plurality of page sets; 2. The computer-implemented method of claim 1, further comprising detecting, by the computer, that the predefined target page sets associated with the application workload have reached the predefined minimum unused page threshold level based on the monitoring of the unused page status of each respective page set.

3. 3. The computer-implemented method of claim 1, further comprising determining, by the computer, whether a scale-out bit is set for the predefined target pageset.

4. detecting, by the computer, that a number of unused pages of the predefined target page set associated with the application workload is greater than the predefined lowest unused page threshold level by a defined percentage amount; 4. The computer-implemented method of claim 1, further comprising: in response to detecting, by the computer, that the number of unused pages of the predefined target page set associated with the application workload is greater than the predefined lowest unused page threshold level by the defined percentage amount, utilizing the predefined target page set as the message storage destination.

5. receiving, by the computer, a message corresponding to the predefined target pageset associated with the application workload of a service; 5. The computer-implemented method of claim 1, further comprising: identifying, by the computer, a number of unused pages in the predefined target pageset based on information contained in a pageset list.

6. determining, by the computer, whether the number of unused pages in the predefined target page set is less than or equal to the predefined lowest unused page threshold level; 6. The computer-implemented method of claim 5, further comprising, in response to the computer determining that the number of unused pages in the predefined target page set is not less than or equal to the predefined lowest unused page threshold level, storing the message in the predefined target page set of a predefined storage class corresponding to the predefined target page set.

7. 7. The computer-implemented method of claim 6, further comprising selecting, by the computer, the spillover page set from the top of the page set list in response to determining that the number of unused pages in the predefined target page set is less than or equal to the predefined lowest unused page threshold level.

8. assigning, by the computer, an expanded storage class corresponding to the spillover pageset to a queue within a multiple-dwell queue; 8. The computer-implemented method of claim 7, further comprising storing, by the computer, the message in the spillover pageset via the extended storage class in the queue.

9. 9. The computer-implemented method of claim 1, wherein when a message corresponding to an entire page set is stored across multiple spillover page sets, the spillover page set includes a message location index that includes an identifier of the spillover page set for reading the message back into the buffer.

10. 10. The computer-implemented method of claim 1, wherein a queue is automatically updated in response to a predefined storage class and an extended storage class of the queue reaching a predefined maximum storage threshold level.

11. 1. A computer system for intelligent storage of messages, comprising: A bus system; a storage device connected to the bus system, the storage device storing program instructions; A processor connected to the bus system, the processor comprising: selecting a spillover page set for storing incoming messages corresponding to a predefined target page set associated with an application workload in response to the predefined target page set reaching a predefined lowest unused page threshold level; and utilizing the spillover pageset as a message storage destination for the received messages corresponding to the predefined target pageset associated with the application workload to expand message storage for the predefined target pageset after the predefined target pageset reaches the predefined lowest unused page threshold level.

12. The processor, monitoring an unused page status of each respective page set within the plurality of page sets; 12. The computer system of claim 11, further comprising: detecting when the predefined target page sets associated with the application workload have reached the predefined minimum unused page threshold level based on the monitoring of the unused page status of each respective page set.

13. The processor, 13. The computer system of claim 11 or 12, further executing the program instructions to determine whether a scale-out bit is set for the predefined target pageset.

14. The processor, Detecting that a number of unused pages of the predefined target page set associated with the application workload is greater than the predefined lowest unused page threshold level by a defined percentage amount; 14. The computer system of claim 11, 12, or 13, further executing the program instructions to: utilize the predefined target pageset as the message storage destination in response to detecting that the number of unused pages of the predefined target pageset associated with the application workload is greater than the predefined lowest unused page threshold level by the defined percentage amount.

15. The processor, receiving a message corresponding to the predefined target pageset associated with the application workload of a service; 15. The computer system of claim 11, further comprising: a program instruction set for executing the program instructions to: identify a number of unused pages in the predefined target pageset based on information contained in a pageset list.

16. A computer program product for intelligent storage of messages, the computer program product comprising a computer readable storage medium having program instructions embodied by the computer readable storage medium, the program instructions causing a computer to: selecting, by the computer, a spillover page set for storing received messages corresponding to a predefined target page set associated with an application workload in response to the predefined target page set reaching a predefined lowest unused page threshold level; and utilizing the spillover pageset as a message storage destination for the received messages corresponding to the predefined target pageset associated with the application workload.

17. monitoring, by the computer, an unused page status of each respective page set within the plurality of page sets; 17. The computer program product of claim 16, further comprising detecting, by the computer, that the pre-target page sets associated with the application workload have reached the pre-defined minimum unused page threshold level based on the monitoring of the unused page status of each respective page set.

18. 18. The computer program product of claim 16 or 17, further comprising determining, by the computer, whether a scale-out bit is set for the predefined target pageset.

19. detecting, by the computer, that a number of unused pages of the predefined target page set associated with the application workload is greater than the predefined lowest unused page threshold level by a defined percentage amount; 20. The computer program product of claim 16, 17, or 18, further comprising: in response to detecting, by the computer, that the number of unused pages of the predefined target pageset associated with the application workload is greater than the predefined lowest unused page threshold level by the defined percentage amount, utilizing the predefined target pageset as the message storage destination.

20. receiving, by the computer, a message corresponding to the predefined target pageset associated with the application workload of a service; 20. The computer program product of claim 16, further comprising: identifying, by the computer, a number of unused pages in the predefined target pageset based on information contained in a pageset list.

21. A computer program comprising program code means adapted to perform the method according to any of claims 1 to 10 when the computer program is run on a computer.

Images