Container data sharing method and system

Abstract

A method and system for sharing container data is provided. In response to detecting a failure of a first container processing a service request, a second container in a cluster of containers is initiated to process the service request. The service request and data generated by the failed first container stored on a physical external memory device are accessed. The service request and data generated by the failed first container are then loaded from the physical external memory device onto the second container via a dedicated hardware link for fast container failure recovery.

Description

[Technical Field]

[0001] The present disclosure relates generally to containers, and more particularly to enabling containers running on operating systems with container extensions to utilize physical external memory devices for container data sharing and fast container failure recovery via dedicated hardware links. [Background technology]

[0002] A container is the lowest layer of a service (e.g., a microservice) that holds running applications, libraries, and their dependencies. Containers can be exposed using an external IP address. Containers are typically used in various cloud environments and bare-metal data centers. Currently, when a service request is handled by a specific container in a cluster of containers, and that specific container encounters a problem that causes the container to crash or fail, the service request is interrupted, and the client device requesting the service receives a timeout response after a defined period of time. As a result, after receiving the timeout response, the client device must resend the service request over a standard network for another container in the cluster to process the service request, causing further delays and increased network traffic. However, no solution currently exists that enables containers to share data between containers using a physical external memory device and enable fast container failure recovery via a dedicated hardware link. Summary of the Invention

[0003] According to one exemplary embodiment, a computer-implemented method for container data sharing is provided. In response to detecting a failure of a first container processing a service request, a second container of a cluster of containers is initiated to process the service request. The service request and data generated by the failed first container stored on a physical external memory device are accessed. The service request and data generated by the failed first container are loaded from the physical external memory device onto the second container via a dedicated hardware link for fast container failure recovery. According to other exemplary embodiments, a computer system and computer program product for container data sharing are provided. As a result, the exemplary embodiment enables containers to achieve higher data availability for fast container failure recovery by utilizing a physical external memory device that communicates via a dedicated hardware link, compared to traditional distributed solutions that communicate over a standard network. [Brief explanation of the drawings]

[0004] [Figure 1] 1 is a pictorial representation of a computing environment in which an illustrative embodiment may be implemented.

[0005] [Figure 2] FIG. 1 illustrates an example of a container data sharing architecture according to an exemplary embodiment.

[0006] [Figure 3] FIG. 2 illustrates an example of a container file according to an exemplary embodiment.

[0007] [Figure 4] FIG. 1 illustrates an example of a memory data sharing process according to an exemplary embodiment.

[0008] [Figure 5]FIG. 2 illustrates an example memory structure according to an exemplary embodiment.

[0009] [Figure 6] FIG. 1 illustrates an example of a workflow according to an exemplary embodiment.

[0010] [Figure 7] FIG. 10 illustrates an example of a shared queue data sharing process according to an exemplary embodiment.

[0011] [Figure 8] 1 is a flowchart illustrating a process for enabling container data sharing according to an exemplary embodiment.

[0012] [Figure 9] 1 is a flowchart illustrating a process for fast container failure recovery using a physical external memory device according to an exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Various aspects of the present disclosure are described through text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in computer program product (CPP) embodiments. For any flowchart, depending on the technology involved, operations may be performed in an order different from that shown in a given flowchart. For example, again depending on the technology involved, two operations shown in successive flowchart blocks may be performed in the reverse order, as a single integrated step, simultaneously, or in an at least partially overlapping manner.

[0014] A computer program product embodiment ("CPP embodiment" or "CPP") is a term used in this disclosure to describe any set of one or more storage media (also referred to as "media") collectively included in a set of one or more storage devices that collectively contain machine-readable code corresponding to instructions and / or data for performing the computer operations specified in a given CPP claim. A "storage device" is any tangible device that can hold and store instructions for use by a computer processor. The computer-readable storage medium may be, but is not limited to, an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media include diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices (such as punch cards or pits / lands formed on a major surface of a disk), or any suitable combination of the foregoing. Computer-readable storage media, as the term is used in this disclosure, is not to be construed as storage in the form of a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through fiber optic cables, electrical signals communicated through wires, and / or other transmission media.As will be appreciated by those skilled in the art, data is typically moved at some infrequent time during the normal operation of a storage device, such as during access, defragmentation, or garbage collection, but the above does not make the storage device temporary, as the data is not temporary while it is stored.

[0015] Referring now to the figures, and in particular to Figures 1-2, diagrams of data processing environments in which illustrative embodiments may be implemented are provided. It should be understood that Figures 1-2 are intended as examples only and are not intended to assert or imply any limitation with respect to the environments in which different embodiments may be implemented. Many modifications to the depicted environments may be made.

[0016] 1 illustrates a pictorial representation of a computing environment in which an exemplary embodiment may be implemented. Computing environment 100 includes an example environment for execution of at least a portion of computer code involved in performing the methodology of the present invention, such as container data sharing code 200. Container data sharing code 200 enables containers running on operating systems with container extensions to utilize physical external memory devices for container data sharing and achieve fast container failure recovery via dedicated hardware links. In addition to container data sharing code block 200, computing environment 100 includes, for example, computer 101, wide area network (WAN) 102, end user device (EUD) 103, remote server 104, public cloud 105, and private cloud 106. In this embodiment, computer 101 includes a set of processors 110 (including processing circuitry 120 and cache 121), a communications fabric 111, volatile memory 112, persistent storage 113 (including operating system 122 and container data sharing code block 200 as identified above), a set of peripheral devices 114 (including a set of user interface (UI) devices 123, storage 124, and an Internet of Things (IoT) sensor set 125), and a network module 115. Remote server 104 includes a remote database 130. Public cloud 105 includes a gateway 140, a cloud orchestration module 141, a set of host physical machines 142, a set of virtual machines 143, and a set of containers 144.

[0017] Computer 101 may take the form of a desktop computer, a laptop computer, a tablet computer, a smartphone, a smartwatch or other wearable computer, a mainframe computer, a quantum computer, or any other form of computer or mobile device now known or later developed that is capable of executing programs, accessing a network, or querying a database, such as remote database 130. As is well understood in the art of computer technology, and depending on the technology, execution of a computer-implemented method may be distributed among multiple computers and / or among multiple locations. While in this presentation of computing environment 100, to keep the presentation as concise as possible, the detailed discussion focuses on a single computer, specifically computer 101. Although computer 101 is not depicted in the cloud in FIG. 1 , it may be located in a cloud. However, computer 101 is not required to reside within a cloud except to any extent expressly indicated.

[0018] Processor set 110 includes one or more computer processors of any type now known or later developed. Processing circuitry 120 may be distributed across multiple packages, e.g., multiple coordinated integrated circuit chips. Processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. Cache 121 is memory located within the processor chip package and is typically used for data or code that should be available for fast access by threads or cores executing on processor set 110. Cache memory is typically organized into multiple levels depending on relative proximity to the processing circuitry. Alternatively, some or all caches for a processor set may be located “off-chip.” In some computing environments, processor set 110 may be designed to operate with qubits and perform quantum computing.

[0019] Computer-readable program instructions are typically loaded onto computer 101 to cause processor set 110 of computer 101 to perform a series of operational steps, thereby realizing a computer-implemented method, whereby the instructions so executed instantiate the method specified in the flowcharts and / or descriptions of the computer-implemented method contained herein (collectively referred to as the "methods of the present invention"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by processor set 110 to control and direct the execution of the methods of the present invention. In computing environment 100, at least some of the instructions for executing the methods of the present invention may be stored in block 200 within persistent storage 113.

[0020] Communications fabric 111 is the signal-conducting pathway that allows various components of computer 101 to communicate with one another. Typically, this fabric is made up of switches and conductive pathways, such as those that make up buses, bridges, physical input / output ports, and the like. Other types of signal communication pathways may be used, such as fiber optic and / or wireless communication pathways.

[0021] Volatile memory 112 may be any type of volatile memory now known or later developed. Examples include dynamic type random access memory (RAM) or static type RAM. Typically, volatile memory is characterized by random access, although this is not required unless expressly stated. In computer 101, volatile memory 112 is located in a single package and is internal to computer 101; however, alternatively or additionally, volatile memory may be distributed across multiple packages and / or located external to computer 101.

[0022] Persistent storage 113 is any form of non-volatile storage for a computer, now known or later developed. The non-volatility of this storage means that stored data remains whether or not power is supplied to computer 101 and / or to persistent storage 113 directly. Persistent storage 113 may be read-only memory (ROM), but typically at least a portion of persistent storage allows data to be written, data to be deleted, and data to be rewritten. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. Operating system 122 may take several forms, such as various known proprietary operating systems or open-source Portable Operating System Interface-type operating systems that employ a kernel. The code contained in block 200 typically includes at least a portion of the computer code involved in performing the methods of the present invention.

[0023] The peripheral device set 114 includes a set of peripheral devices of the computer 101. Data communication connections between the peripheral devices and other components of the computer 101 may be implemented in various ways, such as Bluetooth connections, Near-Field Communication (NFC) connections, connections made by cable (such as a universal serial bus (USB)-type cable), insertion-type connections (e.g., a secure digital (SD) card), connections made over a local area communication network, and even connections made over a wide area network such as the Internet. In various embodiments, the UI device set 123 may include components such as a display screen, speakers, microphones, wearable devices (such as goggles and smartwatches), keyboards, mice, printers, touchpads, game controllers, and haptic devices. The storage 124 may be external storage, such as an external hard drive, or insertable storage, such as an SD card. The storage 124 may be persistent and / or volatile. In some embodiments, the storage 124 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer 101 is required to have a large amount of storage (e.g., computer 101 stores and manages large databases locally), this storage may be provided by peripheral storage devices designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. IoT sensor set 125 consists of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer and another sensor may be a motion detector.

[0024] Network module 115 is a collection of computer software, hardware, and firmware that enables computer 101 to communicate with other computers over WAN 102. Network module 115 may include hardware such as a modem or Wi-Fi signal transceiver, software for packetizing and / or depacketizing data for communication network transmission, and / or web browser software for communicating data over the Internet. In some embodiments, the network control and network forwarding functions of network module 115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN)), the control and forwarding functions of network module 115 are performed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for implementing the methods of the present invention can be downloaded to computer 101 from an external computer or external storage device, typically through a network adapter card or network interface included in network module 115.

[0025] WAN 102 is any wide area network (e.g., the Internet) capable of communicating computer data over non-local distances by any technology for communicating computer data, now known or later developed. In some embodiments, a WAN may be replaced and / or supplemented by a local area network (LAN) designed to communicate data between devices located in a local area, such as a Wi-Fi network. WANs and / or LANs typically include copper transmission cables, optical fiber transmissions, wireless transmissions, and computer hardware such as routers, firewalls, switches, gateway computers, and edge servers.

[0026] End-user device (EUD) 103 is any computer system used and controlled by an end user (e.g., a customer of the enterprise operating computer 101) and may take any of the forms discussed above in connection with computer 101. EUD 103 typically receives useful and useful data from the operation of computer 101. For example, in the hypothetical case where computer 101 is designed to provide recommendations to the end user, the recommendations would typically be communicated from network module 115 of computer 101 over WAN 102 to EUD 103. In this manner, EUD 103 can display or otherwise present the recommendations to the end user. In some embodiments, EUD 103 may be a client device such as a thin client, a heavy client, a mainframe computer, a desktop computer, etc.

[0027] Remote server 104 is any computer system that provides at least some data and / or functionality to computer 101. Remote server 104 may be controlled and used by the same entity that operates computer 101. Remote server 104 represents a machine that collects and stores useful and useful data for use by other computers, such as computer 101. For example, in the hypothetical case where computer 101 is designed and programmed to provide recommendations based on historical data, then this historical data may be provided to computer 101 from remote database 130 of remote server 104.

[0028] A public cloud 105 is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and / or other computer functionality, particularly data storage (cloud storage) and computing power, without direct active management by users. Cloud computing typically leverages resource sharing to achieve coherence and economies of scale. Direct active management of public cloud 105's computing resources is performed by computer hardware and / or software in cloud orchestration module 141. The computing resources provided by public cloud 105 are typically implemented by virtual computing environments (VCEs) running on various computers that comprise host physical machine set 142, the universe of physical computers in and / or available to public cloud 105. Virtual computing environments (VCEs) typically take the form of virtual machines from virtual machine set 143 and / or containers from container set 144. It is understood that these VCEs may be stored as images and transferred among and between various physical machine hosts, either as images or after VCE instantiation. Cloud orchestration module 141 manages the transfer and storage of images, deploys new instantiations of VCE, and manages active instantiations of VCE deployments. Gateway 140 is a collection of computer software, hardware, and firmware that enables public cloud 105 to communicate over WAN 102.

[0029] Some further description of a virtualized computing environment (VCE) is now provided. A VCE can be stored as an "image." A new, active instance of a VCE can be instantiated from the image. Two well-known types of VCE are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature where the kernel allows for the existence of multiple isolated user space instances called containers. These isolated user space instances typically behave as actual computers from the perspective of programs running in them. A computer program running on a normal operating system can utilize all of the computer's resources, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running inside a container can only use the contents of the container and of the devices assigned to the container; this feature is known as containerization.

[0030] A private cloud 106 is similar to a public cloud 105, except that the computing resources are available only for use by a single enterprise. While the private cloud 106 is shown as communicating with the WAN 102, in other embodiments, the private cloud may be completely disconnected from the Internet and accessible only through a local / private network. A hybrid cloud is a composite of multiple clouds of different types (e.g., private, community, or public cloud types), often each implemented by a different vendor. While each of the multiple clouds remains a separate, discrete entity, the larger hybrid cloud architecture is bound together by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability between the constituent clouds. In this embodiment, both the public cloud 105 and the private cloud 106 are part of a larger hybrid cloud.

[0031] As used herein, when used in reference to an item, a "set of" means one or more of that item. For example, a set of clouds is one or more different types of cloud environments. Similarly, when used in reference to an item, a "number of" means one or more of that item.

[0032] Furthermore, the term "at least one of," when used in conjunction with a list of items, means that different combinations of one or more of the listed items may be used, and that only one of each item in the list is required. In other words, "at least one of" means that any combination and number of items from the list may be used, but not all items in the list are required. An item may be a specific object, thing, or category.

[0033] For example, "at least one of item A, item B, or item C" may include, but is not limited to, item A, item A and item B, or item B. This example may also include item A, item B, and item C, or item B and item C. Of course, any combination of these items may be present. In some illustrative examples, "at least one of" may be, for example, but is not limited to, two items A; one item B; and ten items C; four items B and seven items C; or other suitable combinations.

[0034] A container can run on an operating system that can utilize a physical external memory device, such as a coupling facility. A coupling facility is a mainframe processor that runs in its own logical partition (LPAR) defined via a hardware management console. It includes a dedicated physical central processor, memory, special hardware communication channels (e.g., physical coupling facility links) dedicated to transferring data between the container's shared data queues, and a special operating system (e.g., coupling facility control code). A coupling facility has no I / O devices other than the physical coupling facility links. Data stored in a coupling facility resides entirely in storage (e.g., memory) because the coupling facility control code is not a virtual memory operating system. Typically, a coupling facility has large storage (e.g., on the order of tens of gigabytes). Also, a coupling facility does not run application software.

[0035] Currently, applications and middleware running on certain operating systems (e.g., z / OS®) can utilize physical external memory devices for data sharing and high availability. z / OS is a registered trademark of International Business Machines, Corp., Armonk, New York, USA. Entities, such as enterprises, business units, companies, organizations, institutions, agencies, and the like, are increasingly adopting the use of hybrid workloads. As a result, these operating systems include container extensions to enable cloud-native workloads as well as allow containers to run on them. However, no solution currently exists that allows these containers to use physical external memory devices to achieve data sharing and high availability.

[0036] Exemplary embodiments enable containers running on these operating systems with container extensions to utilize physical external memory devices for container data sharing and fast container failure recovery via a dedicated hardware link. For example, exemplary embodiments provide a plug-in container external memory application programming interface (API) to enable containers to call the container external memory API to perform operations such as create, delete, clean, lock, unlock, and the like on physical external memory devices according to a service request the container is processing. Illustrative examples of plug-in container external memory APIs are as follows: [Table 1] Exemplary embodiments may create a new external memory driver function for each respective service request received to perform the corresponding operation on that particular physical external memory device.

[0037] Additionally, the exemplary embodiments add an external memory manager to the operating system kernel. The external memory manager enables the operating system kernel to create a dedicated field (e.g., an external memory data field) in virtual memory and copy data stored in the external memory data field to virtual external memory. Furthermore, the exemplary embodiments add an external memory exploiter under a container-extended virtualization layer. The container-extended virtualization layer is, for example, a partition manager such as a hypervisor. The container-extended virtualization layer virtualizes resources of the container data sharing architecture into multiple LPARs. For example, each respective container corresponds to a specific LPAR among the multiple LPARs in the container data sharing architecture. Each LPAR shares physical resources such as processing capacity, memory, storage, network devices, and the like.

[0038] An exemplary embodiment utilizes an external memory exploiter to connect to a cross-system extension services component of an operating system to transfer data from the virtual external memory of a container extension virtualization layer to a physical external memory device. The cross-system extension services component of the operating system includes a section for an external device driver program. The cross-system extension services component utilizes the external device driver program to operate the physical external memory device. The container extension virtualization layer, when instructed by an external memory manager, virtualizes shared container data stored on the physical external memory device into the virtual external memory.

[0039] An exemplary embodiment utilizes an external memory manager to register and transmit data changes (e.g., write changes) being generated in a container (e.g., a writable container) to a physical external memory device. The external memory manager registers data structures (i.e., data storage units, memory segments, or the like) in the physical external memory device to store data to be shared among the containers in response to a cluster of containers starting. Additionally, in response to a new service request arriving at the cluster of containers, the external memory manager stores the new service request in the external memory device. While the containers in the cluster process the new service request, the external memory manager retrieves each respective data change generated by the container and stores each of these data changes in the registered data structure in the physical external memory device in real time.

[0040] Furthermore, in response to a container processing a new service request encountering a problem that causes the container to crash or fail, the external memory manager selects another container in the cluster to take over processing of the service request. For example, the external memory manager recovers the processing point of the crashed container in the other container by retrieving the service request that was being processed by the crashed container and the corresponding data changes made by the crashed container that are stored in a registered data structure in the external memory device. The external memory manager then transmits the retrieved service request and the corresponding data changes to the other container in the cluster using a dedicated hardware external memory device link or communication channel for fast container failure recovery. Furthermore, the external memory manager synchronizes the service request and the corresponding data changes stored in the external memory device with the other container that takes over processing of the crashed container.

[0041] As a result, the exemplary embodiments enable containers managed by an operating system with a container extension to utilize physical external memory devices that communicate over dedicated hardware links, thereby enabling these containers to achieve high data availability for fast container failure recovery compared to traditional distributed solutions that communicate over standard networks. Thus, the exemplary embodiments provide one or more technical solutions that overcome technical problems by enabling containers to share data via physical external memory devices for high data availability and achieve fast container failure recovery. As a result, these one or more technical solutions provide technical effects and practical applications in the field of containers.

[0042] Referring now to Figure 2, a diagram illustrating an example of a container data sharing architecture according to an exemplary embodiment is shown. Container data sharing architecture 201 may be implemented in a computing environment, such as computing environment 100 in Figure 1. Container data sharing architecture 201 is a system of hardware and software components for enabling containers to share data via physical external memory devices for high data availability and to achieve fast container failure recovery.

[0043] In this example, the container data sharing architecture 201 includes a normal operating system (OS) address space 202 of an operating system such as z / OS, and a container extension (CX) virtual container server address space 204. The normal operating system address space 202 and the container extension virtual container server address space 204 represent ranges of virtual addresses available for executing instructions and storing data.

[0044] The container-extended virtual container server address space 204 includes an operating system kernel 206, which stores an external memory manager (EMM) 208, and an operating system container engine 210. The operating system container engine 210 includes a standard container API 212 and a plug-in container external memory API 214. Containers 216, 218, 220, and 222 call the plug-in container external memory API 214 to instruct the external memory manager 208 to create virtual external memory 224. Note that containers 216, 218, 220, and 222 are intended merely as examples and not as limitations on the exemplary embodiment. In other words, the container-extended virtual container server address space 204 can include any number of containers, and these containers can be included in a cluster of containers. Additionally, the containers can process any type and number of service requests.

[0045] In response to receiving an instruction from one of the plug-in container external memory APIs 214, the external memory manager 208 of the operating system kernel 206 creates a virtual external memory 224. In addition, the external memory manager 208 copies data generated by the container while processing a service request in an external memory data (EMDATA) field 226 of the virtual memory 228 to the virtual external memory 224. Furthermore, the external memory manager 208 continuously monitors for data changes in the external memory data field 226 and updates the virtual external memory 224 with those data changes in the external memory data field 226.

[0046] The container-extended virtual container server address space 204 also includes an external memory (EM) exploiter 230. The external memory exploiter 230 connects to a physical external memory device 232 via an external memory driver 234 using a dedicated hardware link 236 to store data changes from the virtual external memory 224 to the physical external memory device 232 for container data sharing and to achieve fast container failure recovery in case of a container failure while processing a service request. The external memory driver 234 controls operations executed on the physical external memory device 232.

[0047] The external memory exploiter 230 typically generates inter-system extension services 238 in the operating system address space 202 to connect to the physical external memory device 232 through the operating system. Note that the inter-system extension services 238 include an external memory driver 234. When connecting to a specific physical external memory device, the external memory driver 234 selects a corresponding code paragraph for the type (e.g., coupling facility) of the specific external memory device and connects to the specific external memory device to obtain shared data resources stored on the specific external memory device. The container extension (CX) virtualization layer 240 virtualizes the shared data resources stored on the specific external memory device into virtual external memory 224 for container data sharing.

[0048] Also, note that physical memory 242 represents physical memory corresponding to container-extended virtual container server address space 204. Furthermore, a particular address space in physical memory 242 corresponds to a particular address space in virtual memory 228, specifically the address space associated with external memory data field 226. This represents data that would be shared between containers, such as container 216 and container 218, via physical external memory device 232 in the event of a failure of container 216 while processing a service request.

[0049] Referring now to FIG. 3, a diagram illustrating an example of a container file according to an exemplary embodiment is shown. Container file 300 may be implemented in a plug-in container external memory API, such as one of plug-in container external memory APIs 214 in FIG. 2. Also, in this example, container file 300 is a YAML file. YAML is a human-readable data serialization language for files in which data is to be stored or transmitted. However, it should be noted that container file 300 is intended merely as an example and not as a limitation on the exemplary embodiment.

[0050] The container file 300 includes an external memory section 302 in a request segment 304. The external memory section 302 specifies, for example, the type, location, and size of a physical external memory device. The physical external memory device may be, for example, physical external memory device 232 in FIG. 2. In addition, the container file 300 also includes an external memory location section 306 in a volume segment 308. The external memory location section 306 specifies, for example, an external memory location name and the corresponding type, name, and data structure of the external memory device that stores the shared container data.

[0051] 4, a diagram illustrating an example of a memory data sharing process according to an example embodiment is shown. Memory data sharing process 400 can be implemented in a container data sharing architecture, such as, for example, container data sharing architecture 201 in FIG. 2.

[0052] The memory data sharing process 400 includes a physical memory 402, a virtual memory 404, a virtual external memory 406, and an external memory device 408. The physical memory 402, the virtual memory 404, the virtual external memory 406, and the external memory device 408 may be, for example, the physical memory 242, the virtual memory 228, the virtual external memory 224, and the physical external memory device 232 in FIG.

[0053] Memory data sharing process 400 illustrates the correspondence between physical memory 402 and virtual memory 404. Virtual memory 404 includes an external memory data (EMDATA) field 410, such as external memory data field 226 in FIG. 2. An external memory manager 412, such as external memory manager 208 in FIG. 2, adds external memory data field 410 to virtual memory 404. External memory manager 412 stores data that service applications want to share between containers (i.e., partitions corresponding to containers such as containers 216 and 218 in FIG. 2). In addition, external memory manager 412 monitors external memory data field 410 for data changes made by the containers and synchronizes the data changes in external memory data field 410 with virtual external memory 406. For example, an external memory exploiter 414, such as external memory exploiter 230 in FIG. 2, stores data to be shared between containers and cached in virtual external memory 406 in external memory device 408 using an external memory driver program, such as external memory driver 234 in FIG. 2.

[0054] 5, a diagram illustrating an example of a memory structure according to an illustrative embodiment is shown. Memory structure 500 includes a virtual memory area structure (VM_AREA_STRUC) 502, a virtual external memory area structure (EXMEM_AREA_STRUC) 504, and an external memory device data structure 506. Virtual memory area structure 502, virtual external memory area structure 504, and external memory device data structure 506 are implemented in virtual memory 508, virtual external memory 510, and external memory device 512, respectively. Virtual memory 508, virtual external memory 510, and external memory device 512 may be, for example, virtual memory 404, virtual external memory 406, and external memory device 408 in FIG. 4.

[0055] Operating system kernel 514 creates a virtual memory region structure 502 for virtual memory 508. Note that in this example, VM_EXMEM_FLAG 518 is set to YES. Because VM_EXMEM_FLAG 518 is set to YES, external memory manager 516 creates a virtual external memory region structure 504 for virtual external memory 510. Virtual external memory region structure 504 contains information about the location of an external memory device data structure 506 in external memory device 512 for storing data to be shared between containers.

[0056] 6, a diagram illustrating an example of a workflow according to an exemplary embodiment is shown. Workflow 600 can be implemented in a container data sharing architecture, such as container data sharing architecture 201 in FIG. 2. Workflow 600 includes a container file 602, a container 604, an external memory manager 606, an operating system kernel 608, virtual memory 610, an external memory data field 612, a virtual external memory 614, an external memory exploiter 616, a container extension virtualization layer 618, a cross-system extension service 620, and a physical external memory device 622.

[0057] The container file 602 may be, for example, the container file 300 in FIG. 3 . The container file 602 specifies, for example, an external memory device type and a memory space size. The container 604 uses the container file 602 to call a plug-in container external memory API, causing the external memory manager 606 to perform a series of actions. The series of actions may include, for example, the external memory manager 606 setting a virtual memory flag to yes, creating an external memory data field 612 in a virtual memory 610 created by the operating system kernel 608, and notifying the container extension virtualization layer 618 of the data to be shared that is stored in the data structure of the physical external memory device 622. When requested by an upper-layer service application, the container extension virtualization layer 618 virtualizes the data to be shared that is stored in the data structure of the physical external memory device 622 into the virtual external memory 614 for container data sharing. The set of actions may also include the external memory manager 606 instructing the external memory exploiter 616 to generate a new external memory driver in the cross-system extension service 620 that corresponds to the service request being processed by the container 604.

[0058] The external memory exploiter 616 uses an external memory driver, such as the external memory driver 234 in Figure 2, to connect to the physical external memory device 622 via the inter-system extension service 620. When connecting to the physical external memory device 622, the external memory driver of the inter-system extension service 620 selects a corresponding code paragraph in the library of the external memory driver according to the type (e.g., coupling facility) of the physical external memory device 622, connects to the physical external memory device 622, and obtains the data to be shared, which is stored in the data structure of the physical external memory device 622.

[0059] 7, a diagram illustrating an example of a shared queue data sharing process according to an illustrative embodiment is shown. Shared queue data sharing process 700 can be implemented in a container data sharing architecture, such as, for example, container data sharing architecture 201 in FIG. 2.

[0060] In this example, shared queue data sharing process 700 includes operating system LPAR1 702 and operating system LPAR2 704. However, it should be noted that shared queue data sharing process 700 is intended merely as an example and not as a limitation of the illustrative embodiment. In other words, shared queue data sharing process 700 can include any number of operating system LPARs.

[0061] The operating system LPAR1 702 includes a container extension LPAR1 706. The container extension LPAR1 706 is included in a container-extended virtual container server address space, such as, for example, the container-extended virtual container server address space 204 in FIG. 2. Furthermore, the container extension LPAR1 706 corresponds to a container, such as, for example, the container 216 in FIG. 2, for sharing data generated by the container while processing a service request. The external memory manager 708 stores data generated by the container while processing a service request in a shared data queue 710 in an external memory data field 712. Additionally, the external memory manager 708 copies the data stored in the shared data queue 710 in the external memory data field 712 to a shared data queue 714 in a virtual external memory 716. Further, the external memory manager 708 instructs the external memory exploiter 718 to send data generated by the container while processing the service request stored in the shared data queue 714 of the virtual external memory 716 to the shared queue 720 of the physical external memory device 722 via a dedicated hardware link 724 for container data sharing.

[0062] Operating system LPAR2 704 includes container extension LPAR2 726. Container extension LPAR2 726 is also included in the container-extended virtual container server address space and corresponds to another container, such as container 218 in FIG. 2. If the container corresponding to container extension LPAR1 706 fails, external memory manager 728 instructs external memory exploiter 730 to retrieve data generated by the failed container while processing service requests from shared queue 720 in physical external memory device 722 and continue processing service requests for the failed container for fast container failure recovery. External memory exploiter 730 retrieves the data via dedicated hardware link 732 and places the retrieved data from shared queue 720 in physical external memory device 722 in shared data queue 734 in virtual external memory 736 for processing of the service request by the container taking over for the failed container.

[0063] 8, a flowchart illustrating a process for enabling container data sharing is shown, according to an exemplary embodiment. The process shown in FIG. 8 may be implemented in a computer, such as computer 101 in FIG. 1. For example, the process shown in FIG. 8 may be implemented in container data sharing code 200 in FIG. 1.

[0064] The process begins when a computer adds a container external memory API to an operating system container engine to enable one container of a cluster of containers running on the computer to call the container external memory API to perform a set of operations on data stored on a data structure in a physical external memory device according to a service request being processed by the container (step 802). In addition, the computer adds an external memory manager to a kernel of the computer's operating system to enable the external memory manager to create a dedicated external memory data field in the computer's virtual memory and to copy data stored in the dedicated external memory data field to the computer's virtual external memory (step 804).

[0065] The computer further adds an external memory exploiter under the computer's container-extended virtualization layer, which connects with the operating system's inter-system extension service to transfer data from the virtual external memory to a physical external memory device via a dedicated hardware link (step 806). The inter-system extension service includes an external memory device driver for operating the physical external memory device. The container-extended virtualization layer virtualizes data stored on the physical external memory device to the virtual external memory when instructed by the external memory manager. The computer further uses the external memory exploiter to enable the containers to achieve data sharing and high availability between containers in a cluster of containers using the physical external memory device (step 808). The process then terminates.

[0066] 9, a flowchart illustrating a process for fast container failure recovery using a physical external memory device is shown, according to an exemplary embodiment. The process shown in FIG. 9 may be implemented in a computer or set of computers, such as, for example, computer 101 in FIG. 1. For example, the process shown in FIG. 9 may be implemented in container data sharing code 200 in FIG. 1.

[0067] The process begins when a computer receives a service request from a client device over a network to execute a service corresponding to a service application (step 902). In response to receiving the service request, the computer starts one container of a cluster of containers on the computer to process the service request (step 904). In addition, the computer registers a data structure in a physical external memory device for storing data generated by the container corresponding to the service request (step 906). Furthermore, the computer stores the service request on the physical external memory device using an external memory manager of an operating system on the computer (step 908).

[0068] The computer uses an external memory manager to retrieve data generated by the container corresponding to the service request while the container is processing the service request (step 910). The computer uses an external memory exploiter on the computer to store data generated by the container corresponding to the service request while the container is processing the service request in a data structure on the physical external memory device via a dedicated hardware link (step 912).

[0069] The computer uses the external memory manager to detect a failure of the container that is processing the service request (step 914). In response to detecting the failure of the container using the external memory manager, the computer starts another container of the cluster of containers to process the service request (step 916). Note that the other container can be located on a different computer or on the same computer as the failed container.

[0070] Additionally, the other container accesses the service request and data generated by the failed container stored on the data structure of the physical external memory device (step 918). The other container loads the service request and data generated by the failed container from the data structure of the physical external memory device via a dedicated hardware link for fast container failure recovery (step 920). The process then ends.

[0071] Thus, exemplary embodiments of the present invention provide a computer-implemented method, computer system, and computer program product for enabling a container running on an operating system with a container extension to utilize a physical external memory device for container data sharing and fast container failure recovery via a dedicated hardware link. The description of various embodiments of the present invention has been presented for purposes of illustration and is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been selected to best explain the principles of the embodiments, practical applications, or technical improvements over technologies found in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A computer-implemented method for container data sharing, the computer-implemented method comprising: initiating a second container of the cluster of containers to process the service request in response to detecting a failure of a first container processing the service request; accessing the service request and data generated by the failed first container stored on a physical external memory device; and loading the service request and the data generated by the failed first container from the physical external memory device onto the second container via a dedicated hardware link for fast container failure recovery. A computer-implemented method comprising:

2. receiving, via a network, from a client device, a service request to execute a service corresponding to a service application; starting the first container of the cluster of containers to process the service request; registering a data structure in the physical external memory device for storing the data generated by the first container; The computer-implemented method of claim 1 further comprising:

3. storing the service request on the physical external memory device; Retrieving the data generated by the first container corresponding to the service request while the first container is processing the service request.

10. The computer-implemented method of any one of the preceding claims, further comprising:

4. storing the data generated by the first container in the physical external memory device via the dedicated hardware link while the first container processes the service request; 10. The computer-implemented method of any one of the preceding claims, further comprising:

5. adding a container external memory application programming interface (API) to an operating system container engine, enabling the first container of the cluster of containers to invoke the container external memory API to perform a set of operations on the data stored on the physical external memory device in accordance with the service request being processed by the first container; adding an external memory manager to the operating system kernel, the external memory manager enabling the external memory manager to create a dedicated external memory data field in a virtual memory and to copy the data stored in the dedicated external memory data field to a virtual external memory; adding an external memory exploiter under a container-extended virtualization layer, wherein the external memory exploiter interfaces with an inter-system extension service of the operating system to enable the data to be transferred from the virtual external memory to the physical external memory device via the dedicated hardware link; 10. The computer-implemented method of any one of the preceding claims, further comprising:

6. The computer-implemented method of claim 5 , wherein the inter-system extension services include an external memory device driver for operating the physical external memory device.

7. The computer-implemented method of claim 5 , wherein the container extension virtualization layer, when instructed by the external memory manager, virtualizes the data stored on the physical external memory device into the virtual external memory.

8. enabling the first container to use the physical external memory device to achieve data sharing with the second container in the cluster of containers using the external memory exploiter; The computer-implemented method of claim 5 further comprising:

9. 1. A computer system for container data sharing, the computer system comprising: communications fabric; a storage device coupled to said communications fabric, wherein said storage device stores program instructions; a set of processors connected to said communications fabric, said set of processors comprising: In response to detecting a failure of a first container processing a service request, initiating a second container of the cluster of containers to process the service request; accessing the service request and data generated by the failed first container stored on a physical external memory device; Loading the service request and the data generated by the failed first container from the physical external memory device onto the second container via a dedicated hardware link for fast container failure recovery. Executing the program instructions; A computer system comprising:

10. The set of processors receiving the service request from a client device over a network to execute a service corresponding to a service application; starting the first container of the cluster of containers to process the service request; registering a data structure in the physical external memory device for storing the data generated by the first container; 10. The computer system of claim 9 further configured to execute said program instructions.

11. The set of processors storing the service request on the physical external memory device; Retrieving the data generated by the first container corresponding to the service request while the first container is processing the service request. A computer system according to any one of the preceding claims 9 to 10, further executing said program instructions.

12. The set of processors storing the data generated by the first container in the physical external memory device via the dedicated hardware link while the first container processes the service request; A computer system according to any one of the preceding claims 9 to 11, further executing said program instructions.

13. The set of processors adding a container external memory application programming interface (API) to an operating system container engine, enabling a first container of the cluster of containers executing on the computer system to invoke the container external memory API to perform a set of operations on the data stored on the physical external memory device in accordance with the service request being processed by the first container; adding an external memory manager to an operating system kernel of the computer system, the external memory manager enabling the external memory manager to create a dedicated external memory data field in a virtual memory of the computer system and to copy the data stored in the dedicated external memory data field to a virtual external memory of the computer system; Adding an external memory exploiter under a container-extended virtualization layer of the computer system, wherein the external memory exploiter interfaces with an inter-system extension service of the operating system to enable the data to be transferred from the virtual external memory to the physical external memory device via the dedicated hardware link. A computer system according to any one of the preceding claims 9 to 12, further executing said program instructions.

14. 1. A computer program product for container data sharing, the computer program product comprising a computer-readable storage medium having program instructions embodied thereon, the program instructions causing a set of processors to: initiating a second container of the cluster of containers to process the service request in response to detecting a failure of a first container processing the service request; accessing the service request and data generated by the failed first container stored on a physical external memory device; and loading the service request and the data generated by the failed first container from the physical external memory device onto the second container via a dedicated hardware link for fast container failure recovery. a computer program product executable by said set of processors to cause said set of processors to perform the method of

15. receiving, via a network, from a client device, a service request to execute a service corresponding to a service application; starting the first container of the cluster of containers to process the service request; registering a data structure in the physical external memory device for storing the data generated by the first container; The computer program product of claim 14 further comprising:

16. storing the service request on the physical external memory device; Retrieving the data generated by the first container corresponding to the service request while the first container is processing the service request.

16. A computer program product according to any one of the preceding claims 14 to 15, further comprising:

17. storing the data generated by the first container in the physical external memory device via the dedicated hardware link while the first container processes the service request; 17. A computer program product according to any one of the preceding claims 14 to 16, further comprising:

18. adding a container external memory application programming interface (API) to an operating system container engine, enabling the first container of the cluster of containers to invoke the container external memory API to perform a set of operations on the data stored on the physical external memory device in accordance with the service request being processed by the first container; adding an external memory manager to the operating system kernel, the external memory manager enabling the external memory manager to create a dedicated external memory data field in a virtual memory and to copy the data stored in the dedicated external memory data field to a virtual external memory; adding an external memory exploiter under a container-extended virtualization layer, wherein the external memory exploiter interfaces with an inter-system extension service of the operating system to enable the data to be transferred from the virtual external memory to the physical external memory device via the dedicated hardware link; 18. A computer program product according to any one of the preceding claims 14 to 17, further comprising:

19. 20. The computer program product of claim 18, wherein the cross-system extension services include an external memory device driver for operating the physical external memory device.

20. 20. The computer program product of claim 18, wherein the container extension virtualization layer, when directed by the external memory manager, virtualizes the data stored on the physical external memory device into the virtual external memory.

Images