Cluster operating system implementation method and apparatus, device, and storage medium

Abstract

The present disclosure provides a cluster operating system implementation method and apparatus, a device, and a storage medium. The method comprises: mapping the physical memories of computing nodes in a cluster operating system into a unified virtual memory space of a full cluster operating system; mapping sections in the virtual memory space to the devices of the computing nodes; and an application process directly executing an operation on the devices of the computing nodes by means of the virtual memory space. When different or identical application processes run on different computing nodes, cross-node memory data access, synchronization or scheduling, and cross-node file read-write and device access do not need to be carried out manually or explicitly, thereby improving the performance of an application or a computing task. Moreover, the working efficiency of the process or the computing task can be improved while distributed computing / application development is simplified, and the complexity of development and maintenance of the cluster operating system is reduced.

Description

Implementation method, device, equipment and storage medium of cluster operating system Technical Field

[0001] The present disclosure relates to the field of cloud computing cluster control technology, and in particular to an implementation method, apparatus, device, and storage medium of a cluster operating system. Background Art

[0002] A cluster operating system (COS) refers to a server-level operating system. Server operating systems primarily rely on various Linux kernel-based distributions, which cover the majority of cloud computing instances. A single server is insufficient for the development and deployment of application services in an application environment. Distributed clusters utilize multiple servers in a unified manner, minimizing the role of the operating system on each server. Application components and middleware become common in server application development and deployment scenarios. In this scenario, the management, provisioning, orchestration, and monitoring system integration of cluster resources serve as an operating system for the entire data / computing center. This system can also be called a "cluster operating system for cloud computing." A powerful cluster operating system can address the management and utilization of large numbers of servers and their computing, storage, and other resources. Container technologies (such as Docker) and microservices architecture are mainstream trends in cloud computing. Cluster operating systems are also increasingly integrated with container orchestration tools (such as Kubernetes) to support automated container deployment, scaling, and management.

[0003] However, an application process can only use the resources of the computing node where it is located. For application processes or computing tasks that require a large amount of resources to complete, this not only increases the complexity of development and maintenance, but also limits the performance of the application process or computing task.

[0004] Summary of the Invention

[0005] The present disclosure aims to solve one of the technical problems in the related art at least to a certain extent.

[0006] To this end, the first purpose of the present disclosure is to propose an implementation method of a cluster operating system to solve the technical problems existing in the prior art, namely, that the application process can only use the resources of the computing node where it is located, which increases the complexity of development and maintenance and limits the performance of the application process or computing task.

[0007] The second objective of the present disclosure is to provide an implementation device of a cluster operating system.

[0008] A third objective of the present disclosure is to provide a computing and processing device.

[0009] To achieve the above objectives, the first embodiment of the present disclosure provides a method for implementing a cluster operating system, including:

[0010] Mapping the physical memory provided by each computing node in the cluster operating system to obtain a virtual memory space, wherein the virtual memory space is composed of segments of multiple types of virtual spaces;

[0011] Mapping the plurality of segments in the virtual memory space to devices on each of the computing nodes;

[0012] The operation of the application process on the segment in the virtual memory space is executed on the device of the computing node mapped by the segment, wherein the application process runs on each of the computing nodes in the cluster operating system.

[0013] According to one embodiment of the present disclosure, the multi-type virtual space includes: a virtual process address space, a virtual file system space, and a virtual device space, and mapping the multiple segments in the virtual memory space to the devices on each computing node includes:

[0014] Mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to the physical memory on each of the computing nodes respectively;

[0015] The local physical device of each computing node is mapped to a segment corresponding to the virtual device space to obtain a virtual device corresponding to the local physical device.

[0016] According to one embodiment of the present disclosure, the virtual process address space includes multiple process spaces, and mapping the segments corresponding to the virtual process address space to the physical memory on each of the computing nodes includes:

[0017] Determining a process space corresponding to an application process in the cluster operating system from a plurality of process spaces in the virtual process address space, wherein the process space of each application process in the virtual process address space is fixed;

[0018] The segments of the process space corresponding to the application process are respectively mapped to the physical memory on each of the computing nodes.

[0019] According to one embodiment of the present disclosure, the application process running on each of the computing nodes performs an operation on mapping the segment to a device in the computing node by operating the segment in the virtual process address space, including:

[0020] Loading the non-running period data of the application process into the process space corresponding to the application process according to the address of the process space;

[0021] When each computing node runs the application process, allocating runtime data corresponding to the application in the process space corresponding to the application;

[0022] The program counter pointer of the computing node running the application program is pointed to the address of the process space corresponding to the application process, so as to run the application program in the physical memory of the computing node corresponding to the process space.

[0023] According to one embodiment of the present disclosure, the method further includes:

[0024] In the case that any of the application processes has a data transmission demand, the object pointer of the data corresponding to the data transmission demand in the virtual process address space is transferred to the target application process;

[0025] The target application process accesses the virtual process address space through the object pointer to complete data transmission.

[0026] According to one embodiment of the present disclosure, mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to physical memory on each of the computing nodes includes:

[0027] Registering the physical memory provided by each of the computing nodes into the virtual memory space;

[0028] Dividing the physical memory registered to the virtual memory space into physical memory segments having the same size as the segment;

[0029] The segments corresponding to the virtual process address space and the virtual file system space are respectively mapped to the physical memory segments of each of the computing nodes.

[0030] According to one embodiment of the present disclosure, mapping the segments corresponding to the virtual file system space to the physical memory on each of the computing nodes includes:

[0031] According to a predetermined strategy, mapping the segments corresponding to the virtual file system to the physical memory segments of the storage devices in each of the computing nodes;

[0032] The operation of the application process on the segment in the virtual memory space is performed on the device of the computing node mapped by the segment, wherein the application process runs on each computing node in the cluster operating system, including:

[0033] The application process running on each of the computing nodes reads and writes data in the virtual file system space and is executed in the physical memory segment of the storage device of the computing node.

[0034] According to one embodiment of the present disclosure, mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to physical memory on each of the computing nodes includes:

[0035] The segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space are respectively mapped to the physical memory on each of the computing nodes through redundant mapping and / or distributed consistency.

[0036] According to one embodiment of the present disclosure, mapping the local physical device of each computing node to a segment corresponding to the virtual device space to obtain a virtual device corresponding to the local physical device includes:

[0037] Mapping the local physical device of each computing node to the virtual device space through the PVM_GDM of each computing node;

[0038] In the segment corresponding to the virtual device space, a virtual device corresponding to the local physical device is created.

[0039] According to one embodiment of the present disclosure, the operation of the application process on the segment in the virtual memory space is performed on the device of the computing node to which the segment is mapped, wherein the application process runs on each computing node in the cluster operating system, including:

[0040] The application process running on each of the computing nodes performs read and write operations on the virtual device on the local physical device of the computing node mapped to the segment.

[0041] This disclosure provides a method for implementing a cluster operating system. By mapping the physical memory of each compute node into a unified virtual memory space for the entire cluster operating system and mapping segments in the virtual memory space to the compute node's devices, application processes can directly execute operations on the compute node's devices through the virtual memory space. This eliminates the need for manual or explicit cross-node memory data access, synchronization, or scheduling, as well as cross-node file reading and writing and device access, while enabling application processes to improve the performance of applications or computing tasks. Furthermore, this method can simplify distributed computing / application development while improving the efficiency of processes or computing tasks and reducing the complexity of cluster operating system development and maintenance.

[0042] To achieve the above objectives, a second embodiment of the present disclosure provides an implementation device for a cluster operating system, including:

[0043] a first mapping module configured to map the physical memory provided by each computing node in the cluster operating system to obtain a virtual memory space, wherein the virtual memory space is composed of segments of multiple types of virtual spaces;

[0044] a second mapping module, configured to map the plurality of segments in the virtual memory space to devices on each of the computing nodes;

[0045] An execution module is configured to execute the operation of the application process on the segment in the virtual memory space on the device of the computing node mapped by the segment, wherein the application process runs on each computing node in the cluster operating system.

[0046] According to an embodiment of the present disclosure, the multi-type virtual space includes: a virtual process address space, a virtual file system space, and a virtual device space, and the second mapping module is further configured to:

[0047] Mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to the physical memory on each of the computing nodes respectively;

[0048] The local physical device of each computing node is mapped to a segment corresponding to the virtual device space to obtain a virtual device corresponding to the local physical device.

[0049] According to one embodiment of the present disclosure, the virtual process address space includes multiple process spaces, and the second mapping module is further configured to:

[0050] Determining a process space corresponding to an application process in the cluster operating system from a plurality of process spaces in the virtual process address space, wherein the process space of each application process in the virtual process address space is fixed;

[0051] The segments of the process space corresponding to the application process are respectively mapped to the physical memory on each of the computing nodes.

[0052] According to one embodiment of the present disclosure, the execution module is further configured to:

[0053] Loading the non-running period data of the application process into the process space corresponding to the application process according to the address of the process space;

[0054] When each computing node runs the application process, allocating runtime data corresponding to the application in the process space corresponding to the application;

[0055] The program counter pointer of the computing node running the application program is pointed to the address of the process space corresponding to the application process, so as to run the application program in the physical memory of the computing node corresponding to the process space.

[0056] According to one embodiment of the present disclosure, the apparatus further includes: a transmission module configured to:

[0057] In the case that any of the application processes has a data transmission demand, the object pointer of the data corresponding to the data transmission demand in the virtual process address space is transferred to the target application process;

[0058] The target application process accesses the virtual process address space through the object pointer to complete data transmission.

[0059] According to one embodiment of the present disclosure, the second mapping module is further configured to:

[0060] Registering the physical memory provided by each of the computing nodes into the virtual memory space;

[0061] Dividing the physical memory registered to the virtual memory space into physical memory segments having the same size as the segment;

[0062] The segments corresponding to the virtual process address space and the virtual file system space are respectively mapped to the physical memory segments of each of the computing nodes.

[0063] According to one embodiment of the present disclosure, the second mapping module is further configured to:

[0064] According to a predetermined strategy, mapping the segments corresponding to the virtual file system to the physical memory segments of the storage devices in each of the computing nodes;

[0065] The execution module is further configured to:

[0066] The application process running on each of the computing nodes reads and writes data in the virtual file system space and is executed in the physical memory segment of the storage device of the computing node.

[0067] According to one embodiment of the present disclosure, the second mapping module is further configured to:

[0068] The segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space are respectively mapped to the physical memory on each of the computing nodes through redundant mapping and / or distributed consistency.

[0069] According to one embodiment of the present disclosure, the second mapping module is further configured to:

[0070] Mapping the local physical device of each computing node to the virtual device space through the PVM_GDM of each computing node;

[0071] In the segment corresponding to the virtual device space, a virtual device corresponding to the local physical device is created.

[0072] According to one embodiment of the present disclosure, the execution module is further configured to:

[0073] The application process running on each of the computing nodes performs read and write operations on the virtual device on the local physical device of the computing node mapped to the segment.

[0074] To achieve the above-mentioned objectives, a third embodiment of the present disclosure provides a computing and processing device, including:

[0075] a memory having computer readable code stored therein; and

[0076] One or more processors, when the computer-readable code is executed by the one or more processors, the computing processing device executes the implementation method of the cluster operating system proposed in the embodiment of the first aspect of the present disclosure.

[0077] To achieve the above-mentioned purpose, the fourth embodiment of the present disclosure proposes a computer program, including computer-readable code. When the computer-readable code runs on a computing processing device, it causes the computing processing device to execute the implementation method of the cluster operating system proposed in the first embodiment of the present disclosure.

[0078] To achieve the above-mentioned objectives, the fifth embodiment of the present disclosure proposes a computer-readable storage medium, in which the computer program proposed in the sixth embodiment of the present disclosure is stored.

[0079] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] The above and / or additional aspects and advantages of the present disclosure will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0081] FIG1 is a flowchart of a method for implementing a cluster operating system according to an embodiment of the present disclosure;

[0082] FIG2 is a schematic diagram of the architecture of a cluster operating system provided by an embodiment of the present disclosure;

[0083] FIG3 is a schematic diagram of another architecture of a cluster operating system according to an embodiment of the present disclosure;

[0084] FIG4 is a flowchart of a method for implementing step S12 in FIG1 according to an embodiment of the present disclosure;

[0085] FIG5 is a flowchart of a method for implementing step S121 in FIG4 according to an embodiment of the present disclosure;

[0086] FIG6 is a flowchart of a method for implementing step S13 in FIG1 according to an embodiment of the present disclosure;

[0087] FIG7 is a flowchart of a method for implementing step S122 in FIG4 according to an embodiment of the present disclosure;

[0088] FIG8 is a schematic structural diagram of an implementation device of a cluster operating system provided by an embodiment of the present disclosure;

[0089] FIG9 is a schematic structural diagram of a server provided in an embodiment of the present disclosure;

[0090] FIG10 is a schematic diagram of the structure of a cloud server provided in an embodiment of the present disclosure;

[0091] FIG11 is a schematic structural diagram of a computing and processing device provided in an embodiment of the present disclosure.

[0092] FIG12 is a schematic diagram of a storage unit for portable or fixed program code implementation of the method according to the present invention, provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0093] The following describes in detail embodiments of the present disclosure, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0094] Before introducing the implementation method, apparatus, device, and storage medium of a cluster operating system provided by the present disclosure, we will discuss the technical issues encountered in related scenarios. The current state of cluster operating systems indicates that cluster operating systems and their related tools in cloud computing primarily address the automated scheduling and resource allocation of processes / applications on cluster computing nodes. Application development requires the use of specialized techniques or methods to manually or explicitly access other nodes. Otherwise, processes / applications can only use the resources of their own settlement node and cannot utilize the computing and storage resources of other computing nodes. This increases the complexity of development and maintenance for application processes or computing tasks that require significant resources, and also limits the performance of these processes or computing tasks.

[0095] In light of this, the present disclosure provides a cluster operating system implementation method designed to improve the performance of applications or computing tasks by eliminating the need for manual or explicit cross-node memory data access, synchronization, or scheduling, as well as cross-node file reading and writing and device access, during process execution. Furthermore, this method can simplify distributed computing / application development, improve the efficiency of processes or computing tasks, and reduce the complexity of cluster operating system development and maintenance.

[0096] The following describes the implementation method, apparatus, device, and storage medium of the cluster operating system according to the embodiments of the present disclosure with reference to the accompanying drawings.

[0097] FIG1 is a flow chart of a method for implementing a cluster operating system according to an embodiment of the present disclosure. As shown in FIG1 , the method for implementing a cluster operating system according to an embodiment of the present disclosure includes the following steps:

[0098] S11. Mapping the physical memory provided by each computing node in the cluster operating system to obtain a virtual memory space, wherein the virtual memory space is composed of segments of multiple types of virtual spaces.

[0099] As shown in Figure 2, at the Cloud Cluster Operating System (CCOS) level in a cloud / cluster computing scenario, the compute nodes in the cluster operating system use an independent module, PVMM (Permanent Virtual Memory Manager), to uniformly map the provided physical memory to form a globally unified and unique virtual memory space, PVM (Permanent Virtual Memory). This maps the physical memory of each compute node in the cluster operating system into a unified virtual memory space for the entire cluster operating system.

[0100] Each type of virtual space can include at least one segment, and each segment has a corresponding address. Under the unified management and scheduling of PVMM, the physical memory in each computing node is divided into a certain space to form a complete virtual memory space PVM.

[0101] In the disclosed embodiment, as shown in Figure 2, each compute node in the cluster operating system runs a PVMM (Permanent Virtual Memory Manager) operating system. When a compute node boots up, PVMM allocates a certain percentage of the compute node's physical memory and maps it to the virtual memory space (PVM). The remaining physical memory not mapped to the PVM is retained locally.

[0102] Continuing with Figure 2, each compute node includes at least one CPU (Central Processing Unit) node. Each CPU node includes a processor, locally reserved physical memory, and physical memory mapped to a virtual memory space (PVM). Compute nodes can transfer data between each other using RDMA (Remote Direct Memory Access).

[0103] S12. Map the multiple segments in the virtual memory space to devices on each of the computing nodes.

[0104] In an embodiment of the present disclosure, the physical memory used for mapping on the computing node can be mapped to multiple segments in the virtual memory space, and the physical memory not mapped to the virtual memory space PVM is locally retained and will not be mapped to multiple segments in the virtual memory space.

[0105] A computing node may map one or more segments to its physical memory. Furthermore, when mapping the physical memory in the computing node to the segments in the virtual memory space PVM, a mapping relationship may be established with the segments in the virtual memory space PVM, for example, in a one-to-one correspondence manner.

[0106] S13. The operation of the application process on the segment in the virtual memory space is executed on the device of the computing node mapped to the segment, wherein the application process runs on each computing node in the cluster operating system.

[0107] 3 , the application process running on each of the computing nodes can be loaded and run into the virtual memory space, and can also perform file access and device access in the virtual memory space. Specific loading and running, file access, and device access will be described in detail in subsequent embodiments.

[0108] 64-bit processors have become the absolute mainstream, and network speeds are now faster than storage systems. Therefore, high-speed networks can handle the massive memory of numerous compute nodes in cluster operating systems. By uniformly managing the memory resources of compute nodes in cluster operating systems at the operating system level and building a globally unified virtual memory space based on this unified memory management, we can address the shortcomings of cluster operating systems and their tools in terms of usability, maintenance, and performance in cloud computing scenarios.

[0109] This disclosure provides a method for implementing a cluster operating system. By mapping the physical memory of each compute node into a unified virtual memory space for the entire cluster operating system and mapping segments in the virtual memory space to the compute node's devices, application processes can directly execute operations on the compute node's devices through the virtual memory space. This eliminates the need for manual or explicit cross-node memory data access, synchronization, or scheduling, as well as cross-node file reading and writing and device access, while enabling application processes to improve the performance of applications or computing tasks. Furthermore, this method can simplify distributed computing / application development while improving the efficiency of processes or computing tasks and reducing the complexity of cluster operating system development and maintenance.

[0110] According to one embodiment of the present disclosure, the multi-type virtual space includes: a virtual process address space, a virtual file system space, and a virtual device space. A portion of the virtual memory space (PVM) serves as a shared virtual file system space within the cluster operating system. One or more memory-based virtual file systems can be created within this virtual file system space, and the virtual file system space is accessible to all application processes. A portion of the virtual memory space (PVM) serves as the virtual process address space for application processes. Based on preset empirical values, this virtual process address space is further divided into multiple independent process spaces of a certain size.

[0111] It can be understood that the virtual process address space, the virtual file system space and the virtual device space are all composed of at least one segment.

[0112] As shown in FIG4 , in S12 , mapping the multiple segments in the virtual memory space to the devices on each computing node includes:

[0113] S121. Mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to the physical memory on each of the computing nodes respectively;

[0114] It is understood that the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space are mapped to the physical memory registered with the virtual memory space on each computing node, and are not mapped to the physical memory locally reserved in the computing node. The physical memory can be a storage device such as a physical memory stick or a non-volatile storage device.

[0115] S122. Map the local physical device of each computing node to a segment corresponding to the virtual device space to obtain a virtual device corresponding to the local physical device.

[0116] The local physical device may be a hardware device on a computing node, such as a graphics card, a computing accelerator card, or the like.

[0117] Among them, the application process can run in the virtual process address space, and then access the virtual devices in the virtual device space, and then the corresponding read and write operations are performed by the local physical device in the computing node corresponding to the virtual device. Similarly, the application process running in the virtual process address space can access the virtual system file space, and then the corresponding read and write operations are performed by the storage device of the computing node corresponding to the accessed virtual system file space.

[0118] According to one embodiment of the present disclosure, the virtual process address space includes multiple process spaces. Referring to FIG. 5 , in S121, mapping a segment corresponding to the virtual process address space to a physical memory on each of the computing nodes includes:

[0119] S1211. Determine a process space corresponding to an application process in the cluster operating system from the multiple process spaces in the virtual process address space, wherein the process space of each application process in the virtual process address space is fixed.

[0120] In the embodiment of the present disclosure, the address range of each process space in the virtual memory space PVM is permanently fixed, and the address range of the process space in the virtual memory space PVM can be continuous or discrete.

[0121] In the disclosed embodiments, the address space of a specific application process running on a compute node in the PVM is permanently fixed, and process data can be transferred. This allows for even if the application process is destroyed and then reloaded, its various segment data is allocated within the same process space. This allows the application process to be retrieved from a fixed address when activated on different compute nodes, eliminating the need to re-determine the process space. This improves the efficiency of application process computing and eliminates the need to load the same application process into multiple process spaces.

[0122] S1212. Map the segments of the process space corresponding to the application process to the physical memory on each of the computing nodes.

[0123] In the embodiment of the present disclosure, the process space corresponding to the same application process may be composed of one or more segments. Therefore, the same process space may be mapped to the physical memory of one or more different computing nodes.

[0124] In the embodiment of the present disclosure, before the application process runs, one or more segments of the process space corresponding to the application process are respectively mapped to the physical memory on each of the computing nodes. That is to say, a process space can be composed of one or more segments. Therefore, when an application process is running, if the process space corresponding to the application process is composed of one segment, the segment mapped on one computing node is one, and the application process runs in the process space of one computing node; if the process space corresponding to the application process is composed of multiple segments, the segments corresponding to the process space are mapped on multiple computing nodes, and the application process runs in the process space of multiple computing nodes.

[0125] According to one embodiment of the present disclosure, referring to FIG. 6 , based on FIG. 5 , in S13, the application process running on each of the computing nodes performs an operation on mapping the segment to a device in the computing node by operating the segment in the virtual process address space, including:

[0126] S131. Load the non-running time period data of the application process into the process space corresponding to the application process according to the address of the process space.

[0127] In the disclosed embodiment, when an application process is loaded into the virtual memory space PVM, it can first be determined which specific process space in the virtual memory space PVM the application process should be loaded into based on a specific algorithm or mechanism (such as Hash or signature).

[0128] S132. When each computing node runs the application process, allocate runtime time period data corresponding to the application in the process space corresponding to the application;

[0129] In the disclosed embodiment, after determining which specific process space in the virtual memory space PVM the application process should be loaded into, the code segment and data segment, and other non-runtime data of the application process, are loaded into the specific process space. Thus, the virtual address of the application process in the virtual memory space PVM is fixed and permanent.

[0130] S133. Point the program counter pointer of the computing node running the application to the address of the process space corresponding to the application process, so as to run the application in the physical memory of the computing node corresponding to the process space.

[0131] In the embodiment of the present disclosure, when a computing node runs an already loaded application process, the computing node first allocates and initializes the required BSS segment, heap, stack and other runtime data for the application process in the process space of the application process in the virtual memory space PVM through its own PVMM, and then the computing node uses its own PVMM to point the CPU's PC (Program Counter) pointer to the fixed process space address of the process in the above-mentioned PVM. Among them, the BSS segment is a memory area used to store global variables and static variables that are not initialized or initialized to 0 in the application corresponding to the application process. It is readable and writable. The BSS segment will be automatically cleared to 0 before the program is executed. The program counter is a register in the CPU used to store the address of the next instruction.

[0132] According to one embodiment of the present disclosure, the method further includes:

[0133] In the case that any of the application processes has a data transmission requirement, an object pointer of the data corresponding to the data transmission requirement in the virtual process address space is transferred to the target application process.

[0134] When data transmission is required between application processes, data transmission can be understood as an application process requiring other application processes to communicate data to itself or to share its own data with other application processes. In CCOS, this is no longer necessary to be completed through traditional network communication methods. The application process only needs to pass the address (i.e., object pointer) of the data in the virtual memory PVM to another target application process.

[0135] The target application process accesses the virtual process address space through the object pointer to complete data transmission.

[0136] Following the above embodiment, the target application process can directly access the communicated or shared data through the address of the communicated or shared data in the virtual memory space PVM. When the address in the virtual memory space PVM that any computing node needs to access is mapped to the physical memory of another computing node, it will be implemented through the PVM_GMMU (Permanent Virtual Memory Global Memory Management Unit) in the computing node in a remote direct storage access manner, wherein the remote direct storage access manner can be an access manner based on, for example, the RDMA protocol.

[0137] According to one embodiment of the present disclosure, in S121, mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to the physical memory on each of the computing nodes respectively includes:

[0138] Register the physical memory provided by each of the computing nodes in the virtual memory space.

[0139] In the embodiment of the present disclosure, only the physical memory mapped to form the virtual memory space is registered in the virtual memory space, and the locally reserved physical memory is not registered in the virtual memory space.

[0140] The size of the physical memory provided by each computing node may be different. The specific size is determined by the PVMM in the computing node. The PVMM provides the size of the physical memory and the corresponding physical address to the virtual memory space.

[0141] The physical memory registered to the virtual memory space is divided into physical memory segments having the same size as the segment.

[0142] It is understandable that the physical memory divided in the computing node is further divided into physical memory segments of the same size as the segments of the virtual memory space PVM.

[0143] The segments corresponding to the virtual process address space and the virtual file system space are respectively mapped to the physical memory segments of each of the computing nodes.

[0144] In the disclosed embodiment, the physical memory segments of a computing node correspond to segments in the virtual memory space. That is, a physical memory segment in a computing node corresponds to only one segment in the virtual memory space. Of course, segments in the virtual memory space are redundantly mapped to the physical memory segments of multiple computing nodes.

[0145] According to one embodiment of the present disclosure, mapping a segment corresponding to the virtual file system space to a physical memory on each of the computing nodes includes:

[0146] According to a predetermined strategy, the segments corresponding to the virtual file system are mapped to the physical memory segments of the storage devices in each of the computing nodes.

[0147] In the disclosed embodiments, the Permanent Virtual Memory Global File System Manager (PVM_GFSM) in the PVMM running on each compute node can persistently map the physical memory segments mapped by the virtual file system to a local non-volatile storage device on the compute node, according to a predetermined policy, such as synchronous or asynchronous. The non-volatile storage device can be, for example, a hard disk or other storage device.

[0148] The operation of the segment in the virtual memory space by the application process is performed on the device of the computing node to which the segment is mapped, including:

[0149] The application process running on each of the computing nodes reads and writes data in the virtual file system space and is executed in the physical memory segment of the storage device of the computing node.

[0150] In the disclosed embodiment, an application program running on each computing node can directly read and write data from the virtual file system space through a pointer, thereby performing data read and write operations in the physical memory segment of the storage device of the computing node.

[0151] It can be explained that, unlike the non-runtime data and runtime data of the application process loaded or allocated in the process space, the data in the virtual file system space are system files, program files and other types.

[0152] Among them, the data of each segment in the virtual file system space can be stored on non-volatile storage devices of different computing nodes through redundant mapping and / or distributed consistency in subsequent embodiments, thereby ensuring data security and consistency.

[0153] According to one embodiment of the present disclosure, mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to physical memory on each of the computing nodes includes:

[0154] The segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space are respectively mapped to the physical memory on each of the computing nodes through redundant mapping and / or distributed consistency.

[0155] In the embodiment of the present disclosure, the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space can be respectively mapped to the physical memory on each of the computing nodes through a 1:N multi-copy method, an erasure code method or other redundant methods.

[0156] Among them, through the 1:N multi-copy method, the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space are respectively mapped to the physical memory on each of the computing nodes. This can be understood as copying each segment in the virtual memory space into N copies, obtaining the original segment and N copy segments, and then mapping the original segment and copy segment of each segment to the physical memory of different computing nodes. Of course, each time the data is stored in the segment, N copies will also be copied, so that the data can be stored on different computing nodes. Therefore, if the data on the segment of any computing node is lost, it can be restored through the data on other computing nodes.

[0157] Among them, mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to the physical memory on each of the computing nodes by means of erasure codes can be understood as taking multiple segments as the main body of an erasure code, generating erasure codes corresponding to the main body, and then mapping each of the multiple segments to a computing node, and each erasure code is also mapped to a computing node. When data in any segment is lost, the lost data can be obtained by reverse calculation through the erasure code.

[0158] Redundant backup of data in the middle segment of the virtual memory space PVM is performed through redundant mapping and / or distributed consistency. This not only ensures data security and prevents data loss, but also prevents a computing node from being offline, resulting in the inability to read and write data in the physical memory of the computing node, thereby ensuring the stability of the application process operation. In addition, it prevents a computing node from being offline, resulting in the application process having to wait until the computing node comes online, thereby improving the efficiency of the application process operation.

[0159] The distributed consensus approach can be, for example, one based on the RAFT algorithm. This approach achieves consistency by selecting a leader compute node from each compute node and assigning it full responsibility for managing the replicated log. The leader compute node receives log entries from each compute node, replicates them to the segments of other compute nodes, and, while ensuring security, instructs other compute nodes to apply the log entries to their respective state machines.

[0160] It's understandable that the leader compute node can determine which segment of the virtual file system space new log entries should be placed in, without consulting other compute nodes. If the leader compute node fails, a new leader compute node can be selected. This ensures that as compute nodes are added or removed from the CCOS cluster, each compute node maintains a consistent record of the mapping between segments in the virtual memory space (PVM) and the physical memory segments within the compute node. This mapping is accomplished by the PVM_GMMU of the PVMM in the compute node, which is responsible for maintaining and recording it.

[0161] According to one embodiment of the present disclosure, as shown in FIG7 , in S122 , mapping the local physical device of each computing node to a segment corresponding to the virtual device space to obtain a virtual device corresponding to the local physical device includes:

[0162] S1221. Map the local physical device of each computing node to the virtual device space through the PVM_GDM of each computing node.

[0163] In the embodiment of the present disclosure, the local physical device may include a graphics card, a computing accelerator card, and other physical devices on the computing node.

[0164] S1222. Create a virtual device corresponding to the local physical device in a segment corresponding to the virtual device space.

[0165] In the disclosed embodiment, a portion of the virtual memory space PVM is divided out as a virtual device space shared by the cluster operating system. Different computing nodes can selectively map the local physical devices of the computing nodes to the virtual device space through the PVM_GDM (Permanent Virtual Memory Global Device Manager) in their respective PVMMs, and create virtual devices corresponding to the local physical devices in the virtual device space.

[0166] In this way, the local physical devices of the computing nodes can be globally mapped to the virtual memory space, so that any application process can directly call the local physical devices of any computing node from the virtual memory space, thereby making full use of the local physical device resources on the computing nodes and solving the problem of low computing node resource utilization of cluster operating systems in cloud computing scenarios.

[0167] According to one embodiment of the present disclosure, based on FIG. 7 , in S13 , the operation of the application process on the segment in the virtual memory space is performed on the device of the computing node mapped to the segment, including:

[0168] S134. The application process running on each of the computing nodes performs read and write operations on the virtual device on the local physical device of the computing node mapped to the segment.

[0169] In the disclosed embodiment, each application process running on the virtual memory space PVM can access the virtual devices in the virtual device space. The application process's read and write operations on the virtual devices in the virtual device space can be executed on the local physical device of the corresponding computing node in a synchronous or asynchronous manner through the PVM_GDM of PVMM.

[0170] In this way, the read and write operations of the virtual devices in the virtual device space can be executed on the local physical devices of the corresponding computing nodes. The application process does not need to manually or explicitly access the computing nodes when using the local physical devices on other computing nodes. The local physical devices on other computing nodes can be used seamlessly and transparently, thereby greatly simplifying distributed computing / application development while greatly improving the work efficiency of application processes or computing tasks, and improving the convenience and efficiency of cross-node device access.

[0171] The embodiment of the present disclosure further proposes an implementation device of a cluster operating system. As shown in FIG8 , the implementation device of the cluster operating system includes: a first mapping module 810 , a second mapping module 820 and an execution module 830 .

[0172] A first mapping module 810 is configured to map the physical memory provided by each computing node in the cluster operating system to obtain a virtual memory space, wherein the virtual memory space is composed of segments of multiple types of virtual spaces;

[0173] A second mapping module 820 is configured to map the plurality of segments in the virtual memory space to devices on each of the computing nodes;

[0174] The execution module 830 is configured to execute the operation of the application process on the segment in the virtual memory space on the device of the computing node mapped by the segment, wherein the application process runs on each computing node in the cluster operating system.

[0175] According to an embodiment of the present disclosure, the multi-type virtual space includes: a virtual process address space, a virtual file system space, and a virtual device space. The second mapping module 820 is further configured to:

[0176] Mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to the physical memory on each of the computing nodes respectively;

[0177] The local physical device of each computing node is mapped to a segment corresponding to the virtual device space to obtain a virtual device corresponding to the local physical device.

[0178] According to an embodiment of the present disclosure, the virtual process address space includes multiple process spaces, and the second mapping module 820 is further configured to:

[0179] Determining a process space corresponding to an application process in the cluster operating system from a plurality of process spaces in the virtual process address space, wherein the process space of each application process in the virtual process address space is fixed;

[0180] The segments of the process space corresponding to the application process are respectively mapped to the physical memory on each of the computing nodes.

[0181] According to one embodiment of the present disclosure, the execution module 830 is further configured to:

[0182] Loading the non-running period data of the application process into the process space corresponding to the application process according to the address of the process space;

[0183] When each computing node runs the application process, allocating runtime data corresponding to the application in the process space corresponding to the application;

[0184] The program counter pointer of the computing node running the application program is pointed to the address of the process space corresponding to the application process, so as to run the application program in the physical memory of the computing node corresponding to the process space.

[0185] According to one embodiment of the present disclosure, the apparatus further includes: a transmission module configured to:

[0186] In the case that any of the application processes has a data transmission demand, the object pointer of the data corresponding to the data transmission demand in the virtual process address space is transferred to the target application process;

[0187] The target application process accesses the virtual process address space through the object pointer to complete data transmission.

[0188] According to an embodiment of the present disclosure, the second mapping module 820 is further configured to:

[0189] Registering the physical memory provided by each of the computing nodes into the virtual memory space;

[0190] Dividing the physical memory registered to the virtual memory space into physical memory segments having the same size as the segment;

[0191] The segments corresponding to the virtual process address space and the virtual file system space are respectively mapped to the physical memory segments of each of the computing nodes.

[0192] According to an embodiment of the present disclosure, the second mapping module 820 is further configured to:

[0193] According to a predetermined strategy, mapping the segments corresponding to the virtual file system to the physical memory segments of the storage devices in each of the computing nodes;

[0194] The execution module 830 is further configured to:

[0195] The application process running on each of the computing nodes reads and writes data in the virtual file system space and is executed in the physical memory segment of the storage device of the computing node.

[0196] According to an embodiment of the present disclosure, the second mapping module 820 is further configured to:

[0197] The segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space are respectively mapped to the physical memory on each of the computing nodes through redundant mapping and / or distributed consistency.

[0198] According to an embodiment of the present disclosure, the second mapping module 820 is further configured to:

[0199] Mapping the local physical device of each computing node to the virtual device space through the PVM_GDM of each computing node;

[0200] In the segment corresponding to the virtual device space, a virtual device corresponding to the local physical device is created.

[0201] According to one embodiment of the present disclosure, the execution module 830 is further configured to:

[0202] The application process running on each of the computing nodes performs read and write operations on the virtual device on the local physical device of the computing node mapped to the segment.

[0203] It should be noted that the above explanations of the embodiment of the implementation method of the cluster operating system are also applicable to the implementation device of the cluster operating system of this embodiment, and will not be repeated here.

[0204] In order to implement the above embodiment, the present disclosure further proposes a server 900, as shown in FIG9 , which includes a cluster operating system implementation device 800 to implement the above cluster operating system implementation method.

[0205] FIG10 is a block diagram of an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a cloud server, and a plurality of cloud servers may constitute a cluster operating system in the disclosed embodiment, for executing the steps of the implementation method of the cluster operating system and performing the operations of the computing nodes in the aforementioned cluster operating system. Referring to FIG10 , the electronic device 1900 includes a processor 1922, which may be one or more, and a memory 1932 for storing a computer program executable by the processor 1922. The computer program stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processor 1922 may be configured to execute the computer program to execute the steps of the implementation method of the cluster operating system and perform the operations of the computing nodes in the aforementioned cluster operating system.

[0206] In addition, the electronic device 1900 may further include a power supply component 1926 and a communication component 1950. The power supply component 1926 may be configured to perform power management of the electronic device 1900, and the communication component 1950 may be configured to implement communication of the electronic device 1900, for example, wired or wireless communication. In addition, the electronic device 1900 may further include an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , Mac OS XTM , Unix TM , Linux TM etc.

[0207] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, the program instructions implement the steps of the aforementioned method for implementing the cluster operating system and execute the operations of the computing nodes in the aforementioned cluster operating system. For example, the computer-readable storage medium may be the aforementioned memory 1932 including the program instructions. The program instructions may be executed by the processor 1922 of the electronic device 1900 to complete the steps of the aforementioned method for implementing the cluster operating system and execute the operations performed by the computing nodes in the aforementioned cluster operating system.

[0208] In another exemplary embodiment, a computer program product is also provided, which includes a computer program that can be executed by a programmable device, and the computer program has a code portion for executing the implementation method of the above-mentioned cluster operating system and executing the operation of the computing node in the above-mentioned cluster operating system when executed by the programmable device.

[0209] In order to implement the above embodiments, the present disclosure further proposes a computing and processing device, including:

[0210] a memory having computer readable code stored therein; and

[0211] One or more processors, when the computer-readable code is executed by the one or more processors, the computing processing device executes the aforementioned implementation method of the cluster operating system.

[0212] To implement the above embodiments, the present disclosure further proposes a computer program, including computer-readable codes. When the computer-readable codes are executed on a computing processing device, the computing processing device is caused to execute the aforementioned cluster operating system implementation method.

[0213] In order to implement the above embodiments, the present disclosure also proposes a computer-readable storage medium in which the above-mentioned computer program is stored.

[0214] FIG11 is a schematic diagram of the structure of a computing and processing device according to an embodiment of the present disclosure. The computing and processing device generally includes a processor 1110 and a computer program product or computer-readable medium in the form of a memory 1130. The memory 1130 can be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. The memory 1130 has a storage space 1150 for program code 1151 for executing any of the method steps described above. For example, the storage space 1150 for program code can include individual program codes 1151 for implementing various steps in the method described above. These program codes can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards, or floppy disks. Such computer program products are typically portable or fixed storage units, as shown in FIG12. The storage unit can have storage segments and storage space arranged similarly to the memory 1130 in the server of FIG11. The program code can be compressed, for example, in a suitable form. Typically, the storage unit includes computer-readable code 1151 ′, ie, code that can be read by a processor such as 1110 , which, when executed by a server, causes the server to perform the steps of the method described above.

[0215] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and features of different embodiments or examples, unless they are mutually inconsistent.

[0216] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0217] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0218] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0219] It should be understood that various parts of the present disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0220] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0221] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0222] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. A person of ordinary skill in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A method for implementing a cluster operating system, characterized in that, it includes: Mapping the physical memory provided by each computing node in the cluster operating system to obtain a virtual memory space, where the virtual memory space is composed of segments of multiple types of virtual spaces; Mapping multiple segments in the virtual memory space to devices on each computing node; Operations of an application process on segments in the virtual memory space are executed on the devices of the computing node mapped by the segment, where the application process runs on each computing node in the cluster operating system.

2. The method according to claim 1, characterized in that, The multiple types of virtual spaces include: virtual process address space, virtual file system space, and virtual device space. The mapping of multiple segments in the virtual memory space to devices on each computing node includes: Mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to the physical memory on each computing node respectively; Mapping the local physical devices of each computing node to the segments corresponding to the virtual device space to obtain virtual devices corresponding to the local physical devices.

3. The method according to claim 2, characterized in that, The virtual process address space includes multiple process spaces. The mapping of the segments corresponding to the virtual process address space to the physical memory on each computing node includes: Determining, from the multiple process spaces of the virtual process address space, the process space corresponding to the application process in the cluster operating system, and the process space of each application process in the virtual process address space is fixed; Mapping the segments of the process space corresponding to the application process to the physical memory on each computing node respectively.

4. The method according to claim 3, characterized in that, The application process running on each computing node executes operations on the devices in the computing node mapped by the segment by operating on the segments in the virtual process address space, including: Loading the non-running period data of the application process into the process space corresponding to the application process according to the address of the process space; Allocating the running period data corresponding to the application program in the process space corresponding to the application program when the application process runs on each computing node; Pointing the program counter pointer of the application program running on this computing node to the address of the process space corresponding to the application process to run the application program in the physical memory of the computing node corresponding to the process space.

5. The method according to any one of claims 2-4, characterized in that, The method further includes: In the case that any application process has a data transmission requirement, passing the object pointer of the data corresponding to the data transmission requirement in the virtual process address space to the target application process; The target application process accesses the virtual process address space through the object pointer to complete data transmission.

6. The method according to any one of claims 2-4, characterized in that, Mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to the physical memory on each of the computing nodes respectively includes: Registering the physical memory provided by each of the computing nodes to the virtual memory space; Dividing the physical memory registered to the virtual memory space into physical memory segments having the same size as the segments; Mapping the segments corresponding to the virtual process address space and the virtual file system space to the physical memory segments on each of the computing nodes respectively.

7. The method according to claim 6, wherein, Mapping the segments corresponding to the virtual file system space to the physical memory on each of the computing nodes includes: Mapping the segments corresponding to the virtual file system to the physical memory segments of the storage devices in each of the computing nodes according to a predetermined policy; The operations of the application process on the segments in the virtual memory space are executed on the devices of the computing nodes where the segments are mapped, including: The application processes running on each of the computing nodes read and write data in the virtual file system space and execute in the physical memory segments of the storage devices of the computing nodes.

8. The method according to any one of claims 2-4, wherein, Mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to the physical memory on each of the computing nodes respectively includes: Mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to the physical memory on each of the computing nodes respectively by means of redundant mapping and / or distributed consistency.

9. The method according to any one of claims 2-4, wherein, Mapping the local physical devices of each of the computing nodes to the segments corresponding to the virtual device space to obtain virtual devices corresponding to the local physical devices includes: Mapping the local physical devices of each of the computing nodes to the virtual device space through the PVM_GDM of each of the computing nodes; Creating virtual devices corresponding to the local physical devices in the segments corresponding to the virtual device space.

10. The method according to claim 9, wherein, The operations of the application process on the segments in the virtual memory space are executed on the devices of the computing nodes where the segments are mapped, including: The read and write operations of the application processes running on each of the computing nodes on the virtual devices are executed on the local physical devices of the computing nodes where the segments are mapped.

11. An implementation device of a cluster operating system, wherein, comprising: A first mapping module configured to map the physical memory provided by each computing node in the cluster operating system to obtain a virtual memory space, wherein the virtual memory space is composed of segments of multiple space types; A second mapping module configured to map multiple segments in the virtual memory space to devices on each of the computing nodes; An execution module, configured to perform operations on segments in the virtual memory space of an application process, and execute on a device of the computing node mapped by the segment, wherein the application process runs on each of the computing nodes in the cluster operating system.

12. A computing processing device, characterized in that it comprises: a memory storing computer-readable code; and one or more processors, when the computer-readable code is executed by the one or more processors, the computing processing device executes the implementation method of the cluster operating system according to any one of claims 1-10.

13. A computer program, comprising computer-readable code, when the computer-readable code runs on a computing processing device, causing the computing processing device to execute the implementation method of the cluster operating system according to any one of claims 1-10.

14. A computer-readable storage medium storing the computer program according to claim 13.

Images