Method, device, equipment and storage medium for realizing a cluster operating system

Abstract

The present disclosure proposes a method, an apparatus, a device, and a storage medium for implementing a cluster operating system. By mapping the physical memory of each computing node in the cluster operating system into a unified virtual memory space of the entire cluster operating system, and mapping segments in the virtual memory space to devices in the computing nodes, application processes can directly perform operations on devices in the computing nodes through the virtual memory space. When different or the same application processes are executed on different computing nodes, there is no need to manually or explicitly perform memory data access, synchronization or scheduling between nodes, and file read / write and device access between nodes, thereby improving the performance of applications or computing tasks.

Description

[Technical Field]

[0001] The present disclosure relates to the field of cloud computing cluster control technology, and in particular to a method, apparatus, device and storage medium for implementing a cluster operating system. [Background technology]

[0002] A cluster operating system is an operating system at the server level. Server operating systems primarily rely on various release versions based on the Linux kernel, and various Linux release versions cover most cloud computing scenarios. A single server is difficult to meet the application environment and application service development and deployment needs. Distributed clusters use multiple servers in an integrated scheduling manner, minimizing the role of the operating system on each server. Various application components and middleware are common server application development and deployment scenarios. In such scenarios, the integration of cluster resource management, scheduling, orchestration, and monitoring systems creates an operating system that covers the entire data center or computing center, functioning as a "cloud computing cluster operating system." A powerful cluster operating system can solve problems in managing and utilizing a large number of servers and their computing and storage resources. Container technology (e.g., Docker) and microservices architectures are becoming mainstream trends in cloud computing. Cluster operating systems are increasingly integrated with container orchestration tools (e.g., Kubernetes) to support automated container deployment, scaling, and management.

[0003] However, in conventional technologies, an application process can only utilize the resources of the computing node on which it is located, which not only increases the complexity of development and maintenance for application processes or computing tasks that require large amounts of resources, but also limits the performance of the application processes or computing tasks. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure aims to solve, at least to some extent, one of the technical problems in the related art.

[0005] Therefore, the first objective of the present disclosure is to provide a method for realizing a cluster operating system to solve the technical problem of the prior art that an application process can only use the resources of the computing node on which it is located, which increases the complexity of development and maintenance and limits the performance of the application process or computing task.

[0006] A second object of the present disclosure is to provide an apparatus for realizing a cluster operating system.

[0007] A third object of the present disclosure is to provide a computing processing device. [Means for solving the problem]

[0008] To achieve the above object, an embodiment of a first aspect of the present disclosure is a method for implementing a cluster operating system, comprising: Mapping physical memory provided by each computing node in the cluster operating system to obtain a virtual memory space, the virtual memory space consisting of multi-type virtual space segments; mapping a plurality of said segments in said virtual memory space to devices on each of said computing nodes; Executing an operation on a segment in the virtual memory space of an application process on a device in the computing node to which the segment is mapped, the application process being executed on each of the computing nodes in the cluster operating system; A method for realizing a cluster operating system is proposed.

[0009] According to an embodiment of the present disclosure, the multi-type virtual space includes a virtual process address space, a virtual file system space, and a virtual device space, and the step of mapping the plurality of segments in the virtual memory space to devices in each of the computing nodes includes: mapping a segment corresponding to the virtual process address space and a segment corresponding to the virtual file system space to physical memory on each of the computing nodes; Mapping a local physical device at each of the computing nodes to a corresponding segment in the virtual device space to obtain a virtual device corresponding to the local physical device.

[0010] According to an embodiment of the present disclosure, the virtual process address space includes a plurality of process spaces, and the step of mapping segments corresponding to the virtual process address space to physical memory in each of the computing nodes includes: determining a process space corresponding to an application process in the cluster operating system from a plurality of process spaces in the virtual process address space, wherein the process space in the virtual process address space of each of the application processes is fixed; and mapping a segment of process space corresponding to the application process to physical memory on each of the computing nodes, respectively.

[0011] According to an embodiment of the present disclosure, the step of an application process executed on each of the computing nodes performing an operation on a device on the computing node to which the segment is mapped by performing an operation on a segment in the virtual process address space includes: loading non-execution period data of the application process into the process space corresponding to the application process based on the address of the process space; When each of the computing nodes executes the application process, Application Process In the process space corresponding to Application Process assigning execution period data corresponding to The computing node Application Process to an address in a process space corresponding to the application process, in a physical memory in a computing node corresponding to the process space. Application Process and performing the steps of:

[0012] According to one embodiment of the present disclosure, the method comprises: When a data transmission demand exists in any one of the application processes, transferring a target pointer in the virtual process address space of data corresponding to the data transmission demand to a target application process; The method further includes the step of the target application process accessing the virtual process address space through the object pointer to complete the data transfer.

[0013] According to an embodiment of the present disclosure, the step of mapping the segment corresponding to the virtual process address space and the segment corresponding to the virtual file system space to physical memory in each of the computing nodes includes: registering physical memory provided by each of said computing nodes in said virtual memory space; Dividing the physical memory registered in the virtual memory space into physical memory segments having the same size as the segments; and mapping a segment corresponding to the virtual process address space and a segment corresponding to the virtual file system space to the physical memory segments in each of the computing nodes, respectively.

[0014] According to one embodiment of the present disclosure, the step of mapping segments corresponding to the virtual file system space to physical memory in each of the computing nodes includes: The virtual file system is created according to a predetermined policy. space to physical memory segments of a storage device in each of said computing nodes; Executing an operation of an application process on a segment in the virtual memory space on a device in the computing node to which the segment is mapped, the application process being executed on each of the computing nodes in the cluster operating system, comprises: The method includes a step in which an application process running on each of the computing nodes reads and writes data from a physical memory segment of a storage device on the computing node, thereby causing the application process running on each of the computing nodes to read and write data from and to the virtual file system space.

[0015] According to an embodiment of the present disclosure, the step of mapping the segment corresponding to the virtual process address space and the segment corresponding to the virtual file system space to physical memory in each of the computing nodes includes: The method includes a step of respectively mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to physical memory in each of the computing nodes using a redundant mapping scheme and / or a distributed consistency scheme.

[0016] According to an embodiment of the present disclosure, the step of mapping a local physical device in each computing node to a corresponding segment in the virtual device space and obtaining a virtual device corresponding to the local physical device includes: mapping local physical devices on each of said computing nodes to said virtual device space by a PVM_GDM on each of said computing nodes; creating a virtual device corresponding to the local physical device in a segment corresponding to the virtual device space.

[0017] According to one embodiment of the present disclosure, the step of executing an operation on a segment in the virtual memory space of an application process on a device in the computing node to which the segment is mapped, wherein the application process is executed on each of the computing nodes in the cluster operating system, includes: The method includes a step of performing read and write operations on the virtual device of an application process running on each of the computing nodes to a local physical device on the computing node to which the segment is mapped.

[0018] In the method for realizing a cluster operating system according to the present disclosure, the physical memory of each computing node is mapped into a unified virtual memory space of the entire cluster operating system, and segments in the virtual memory space are mapped to devices in the computing nodes, so that application processes can directly perform operations on devices in the computing nodes through the virtual memory space. When an application process is executed, there is no need to manually or explicitly perform memory data access, synchronization, or scheduling between nodes, or file read / write and device access between nodes, thereby improving the performance of applications or computing tasks. This also simplifies distributed computing / application development, improves the operating efficiency of processes or computing tasks, and reduces the complexity of cluster operating system development and maintenance.

[0019] To achieve the above object, an embodiment of a second aspect of the present disclosure is an apparatus for implementing a cluster operating system, comprising: a first mapping module configured to map a physical memory provided by each computing node in the cluster operating system to obtain a virtual memory space consisting of segments of multiple space types; a second mapping module configured to map the plurality of segments in the virtual memory space to devices in each of the computing nodes; an execution module configured to execute operations on a segment in the virtual memory space of an application process on a device in the computing node to which the segment is mapped, the application process executing on each of the computing nodes in the cluster operating system; The present invention proposes an apparatus for implementing a cluster operating system, including:

[0020] According to an embodiment of the present disclosure, the multi-type virtual space includes a virtual process address space, a virtual file system space, and a virtual device space, and the second mapping module further comprises: mapping a segment corresponding to the virtual process address space and a segment corresponding to the virtual file system space to physical memory in each of the computing nodes; The method is configured to map a local physical device at each of the computing nodes to a corresponding segment in the virtual device space to obtain a virtual device corresponding to the local physical device.

[0021] According to an embodiment of the present disclosure, the virtual process address space includes a plurality of process spaces, and the second mapping module further comprises: determining a process space corresponding to an application process in the cluster operating system from the plurality of process spaces in the virtual process address space, the process space in the virtual process address space of each of the application processes being fixed; A segment of process space corresponding to the application process is configured to be mapped to physical memory on each of the computing nodes, respectively.

[0022] According to an embodiment of the present disclosure, the execution module further comprises: loading non-execution period data of the application process into the process space corresponding to the application process based on the address of the process space; When each of the computing nodes executes the application process, Application Process In the process space corresponding to Application Process assigning execution period data corresponding to The computing node Application Processto an address in a process space corresponding to the application process, in a physical memory in a computing node corresponding to the process space. Application Process is configured to execute

[0023] According to one embodiment of the present disclosure, the device is a transmission module, If any one of the application processes has a data transmission demand, transfer a target pointer in the virtual process address space of data corresponding to the data transmission demand to a target application process; The method further includes a transmission module configured to cause the target application process to access the virtual process address space through the object pointer to complete a data transmission.

[0024] According to an embodiment of the present disclosure, the second mapping module further comprises: registering physical memory provided by each of said computing nodes in said virtual memory space; Dividing the physical memory registered in the virtual memory space into physical memory segments having the same size as the segments; The virtual process address space and the virtual file system space are configured to be mapped to the physical memory segments in each of the computing nodes, respectively.

[0025] According to an embodiment of the present disclosure, the second mapping module further comprises: The virtual file system is created according to a predetermined policy. space to physical memory segments of a storage device in each of the computing nodes; The execution module further comprises: An application process running on each of the computing nodes is configured to read and write data in the physical memory segments of the storage device on the computing node, thereby causing the application process running on each of the computing nodes to read and write data in the virtual file system space.

[0026] According to an embodiment of the present disclosure, the second mapping module further comprises: The segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space are configured to be mapped to physical memory in each of the computing nodes using a redundant mapping scheme and / or a distributed consistency scheme.

[0027] According to an embodiment of the present disclosure, the second mapping module further comprises: mapping local physical devices on each of said computing nodes to said virtual device space by a PVM_GDM on each of said computing nodes; A virtual device corresponding to the local physical device is created in a segment corresponding to the virtual device space.

[0028] According to an embodiment of the present disclosure, the execution module further comprises: The segment is configured to perform read and write operations on the virtual device of an application process running on each of the computing nodes to a local physical device on the computing node to which the segment is mapped.

[0029] To achieve the above object, an embodiment of a third aspect of the present disclosure is a computing processing device, comprising: a storage device in which computer readable code is stored; one or more processors, wherein the computer readable code, when executed by the one or more processors, causes the computing processing device to perform a method for implementing a cluster operating system according to an embodiment of the first aspect of the present disclosure; The present invention proposes a computing processing device including:

[0030] To achieve the above object, an embodiment of a fourth aspect of the present disclosure proposes a computer program comprising computer readable code, which, when executed on a computing processing device, causes the computing processing device to perform a method for implementing a cluster operating system according to an embodiment of the first aspect of the present disclosure.

[0031] In order to achieve the above object, the fifth embodiment of the present disclosure is 4 A computer readable storage medium is proposed on which a computer program according to an embodiment of the aspect is stored.

[0032] Additional aspects and advantages of the disclosure will be set forth in part in the description that follows, and in part will be obvious from the description, or may be learned through practice of the disclosure. [Brief explanation of the drawings]

[0033] The above and / or additional aspects and advantages of the present disclosure will be apparent and readily understood from the following description of the embodiments taken in conjunction with the drawings.

[0034] [Figure 1] 1 is a flowchart of a method for implementing a cluster operating system according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a schematic architecture diagram of a cluster operating system according to an embodiment of the present disclosure. [Figure 3] FIG. 2 is a schematic architecture diagram of another cluster operating system according to an embodiment of the present disclosure. [Figure 4] 2 is a flowchart of a method for implementing step S12 in FIG. 1 according to an embodiment of the present disclosure. [Figure 5] 5 is a flowchart of a method for implementing step S121 in FIG. 4 according to an embodiment of the present disclosure. [Figure 6] 2 is a flowchart of a method for implementing step S13 in FIG. 1 according to an embodiment of the present disclosure. [Figure 7] 5 is a flowchart of a method for implementing step S122 in FIG. 4 according to an embodiment of the present disclosure. [Figure 8] 1 is a structural schematic diagram of an apparatus for implementing a cluster operating system according to an embodiment of the present disclosure; [Figure 9] FIG. 2 is a structural schematic diagram of a server according to an embodiment of the present disclosure; [Figure 10] FIG. 2 is a structural schematic diagram of a cloud server according to an embodiment of the present disclosure; [Figure 11] FIG. 1 is a structural schematic diagram of a computing processing device according to an embodiment of the present disclosure. [Figure 12] 1 is a schematic diagram of a portable or fixed storage unit for implementing program code based on the method of the present disclosure according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0035] Hereinafter, the embodiments of the present disclosure will be described in detail. The examples of the embodiments are shown in the drawings, and the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the drawings are examples for explaining the present disclosure and should not be understood as limitations on the present disclosure.

[0036] Before introducing the method, device, equipment, and storage medium for implementing the cluster operating system according to the present disclosure, technical problems existing in related scenarios will be introduced. From the current state of cluster operating systems, it can be seen that cluster operating systems and their related tools in traditional cloud computing mainly solve the problem of automated scheduling and resource allocation for processes / applications on cluster computing nodes. During application development, specialized techniques and methods are used to manually or explicitly access other nodes. Otherwise, a process / application cannot use the computing and storage resources of other computing nodes, and can only use the resources of the computing node where it is located. This not only increases the complexity of development and maintenance for application processes or computing tasks that require a large amount of resources, but also limits the performance of the application processes or computing tasks.

[0037] In view of this, the method for realizing a cluster operating system according to the present disclosure aims to improve the performance of applications or computing tasks by eliminating the need to manually or explicitly perform inter-node memory data access, synchronization or scheduling, and inter-node file read / write and device access when a process is executed, thereby simplifying distributed computing / application development, improving the operating efficiency of processes or computing tasks, and reducing the complexity of cluster operating system development and maintenance.

[0038] Hereinafter, a method, an apparatus, a device, and a storage medium for implementing a cluster operating system according to an embodiment of the present disclosure will be described with reference to the accompanying drawings.

[0039] 1 is a flowchart of a method for implementing a cluster operating system according to an embodiment of the present disclosure. As shown in FIG. 1, the method for implementing a cluster operating system according to an embodiment of the present disclosure includes the following steps:

[0040] In step S11, the physical memory provided by each computing node in the cluster operating system is mapped to obtain a virtual memory space consisting of segments of a multi-type virtual space.

[0041] As shown in Figure 2, at the operating system CCOS (Cloud Cluster Operating System) level in a cloud / cluster computing scenario, the computing nodes in the cluster operating system use an independent module, Permanent Virtual Memory Manager (PVMM), to unify the physical memory provided to them and form a globally unified unique virtual memory space, Permanent Virtual Memory (PVM). That is, the physical memory of each computing node in the cluster operating system is mapped to the unified virtual memory space of the entire cluster operating system.

[0042] Each type of virtual space may include at least one segment, and each segment has a corresponding address. The physical memory in each computing node is divided into certain spaces under the unified management and scheduling of the PVMM, which together constitute the complete virtual memory space PVM.

[0043] In an embodiment of the present disclosure, referring to Fig. 2, each computing node in the cluster operating system runs a PVMM (Permanent Virtual Memory Manager) operating system. When a computing node starts up, the PVMM divides a certain percentage of the computing node's physical memory and maps it to a virtual memory space PVM, and keeps the physical memory that is not mapped to the virtual memory space PVM locally.

[0044] 2, each computing node may include at least one CPU (Central Processing Unit) node, and each CPU node includes a processor, a locally stored physical memory, and a physical memory mapped to a virtual memory space PVM. Data transfer between computing nodes can be performed using RDMA (Remote Direct Memory Access).

[0045] In step S12, the segments in the virtual memory space are mapped to devices in each of the computing nodes.

[0046] In an embodiment of the present disclosure, physical memory for mapping in a computing node can be mapped to the plurality of segments in the virtual memory space, and physical memory that is not mapped to the virtual memory space PVM is kept locally without being mapped to the plurality of segments in the virtual memory space.

[0047] A computing node can map one or more segments to its own physical memory, and when mapping the physical memory of a computing node to a segment in the virtual memory space PVM, a mapping relationship may be created to the segment in the virtual memory space PVM, for example, using a one-to-one correspondence method.

[0048] In step S13, an operation of an application process on a segment in the virtual memory space is executed on a device in the computing node to which the segment is mapped. The application process is executed on each of the computing nodes in the cluster operating system.

[0049] 3, an application process executed on each computing node may be loaded into a virtual memory space and executed therein, and may access files and devices in the virtual memory space. Specific examples of loading, file access, and device access will be described in detail in subsequent embodiments.

[0050] 64-bit processors have become the absolute mainstream, and network speeds are faster than storage system speeds, so high-speed networks can handle large amounts of memory in a large number of computing nodes in a cluster operating system. At the operating system level, the memory resources of the computing nodes in the cluster operating system are managed in an integrated manner, and based on this integrated memory management, a globally integrated virtual memory space for the cluster is created, thereby resolving the shortcomings of cluster operating systems and their tools in terms of use, maintenance, performance, etc. in cloud computing scenarios.

[0051] In the method for realizing a cluster operating system according to the present disclosure, the physical memory of each computing node is mapped into a unified virtual memory space of the entire cluster operating system, and segments in the virtual memory space are mapped to devices in the computing nodes, so that application processes can directly perform operations on devices in the computing nodes through the virtual memory space. When an application process is executed, there is no need to manually or explicitly perform memory data access, synchronization, or scheduling between nodes, or file read / write and device access between nodes, thereby improving the performance of applications or computing tasks. This also simplifies distributed computing / application development, improves the operating efficiency of processes or computing tasks, and reduces the complexity of cluster operating system development and maintenance.

[0052] According to one embodiment of the present disclosure, the multi-type virtual space includes a virtual process address space, a virtual file system space, and a virtual device space. A portion of the virtual memory space PVM is a virtual file system space shared within the cluster operating system. One or more memory-based virtual file systems can be created in this virtual file system space, and the virtual file system space can be accessed by all application processes. A portion of the virtual memory space PVM is a virtual process address space for application processes. Based on a preset experience value, this virtual process address space is further divided into multiple mutually independent process spaces of a certain size.

[0053] As can be seen, the virtual process address space, the virtual file system space, and the virtual device space all consist of at least one segment.

[0054] Referring to FIG. 4, the step S12 of mapping the segments in the virtual memory space to devices in each of the computing nodes includes the following steps:

[0055] In step S121, the segment corresponding to the virtual process address space and the segment corresponding to the virtual file system space are respectively mapped to the physical memory in each of the computing nodes.

[0056] As can be understood, each of the segments corresponding to the virtual process address space and the virtual file system space is mapped to physical memory registered in the virtual memory space of each computing node, without being mapped to physical memory held locally in the computing node, which may be a storage device such as a physical memory bar, or may be a non-volatile storage device.

[0057] In step S122, a local physical device in each computing node is mapped to a corresponding segment in the virtual device space to obtain a virtual device corresponding to the local physical device.

[0058] The local physical device may be a hardware device in a computing node, such as a graphics card or a computing accelerator card.

[0059] An application process can run in a virtual process address space and access a virtual device in the virtual device space, and a local physical device in the computing node corresponding to the virtual device performs a corresponding read / write operation. Similarly, an application process can run in a virtual process address space and access a virtual system file space, and a storage device in the computing node corresponding to the accessed virtual system file space performs a corresponding read / write operation.

[0060] According to an embodiment of the present disclosure, the virtual process address space includes a plurality of process spaces. Referring to Figure 5, in step S121, mapping segments corresponding to the virtual process address space to physical memory in each of the computing nodes includes the following steps:

[0061] In step S1211, a process space corresponding to an application process in the cluster operating system is determined from the plurality of process spaces in the virtual process address space. The process space in the virtual process address space of each of the application processes is fixed.

[0062] In the embodiment of the present disclosure, the address range of each process space in the virtual memory space PVM is permanently fixed, and the address range of each process space in the virtual memory space PVM may be contiguous or discrete.

[0063] In an embodiment of the present disclosure, the address space in the PVM of a specific application process running on a computing node is permanently fixed, and the process data is transferable. Thus, even if the application process is destroyed and then reloaded and executed, various segment data of the application process are allocated in the same process space as before. In this way, when the application process is enabled on a different computing node, the process space can be queried by the fixed address without the need to re-determine the process space, thereby improving the computing efficiency of the application process and eliminating the need to load the same application process in multiple process spaces.

[0064] In step S1212, the process space segments corresponding to the application processes are mapped to physical memory in each of the computing nodes, respectively.

[0065] In embodiments of the present disclosure, the process space corresponding to the same application process may consist of one or more segments, and therefore the same process space may be mapped to physical memory on one or more different computing nodes.

[0066] In an embodiment of the present disclosure, before an application process is executed, one or more segments of a process space corresponding to the application process are respectively mapped to the physical memory of each computing node. That is, one process space may consist of one or more segments. Therefore, when one application process is executed, if the process space corresponding to the application process consists of one segment, only one segment is mapped to one computing node, and the application process is executed in the process space of the one computing node. If the process space corresponding to the application process consists of multiple segments, the segments corresponding to the process space are mapped to multiple computing nodes, and the application process is executed in the process spaces of the multiple computing nodes.

[0067] According to one embodiment of the present disclosure, referring to FIG. 6, based on FIG. 5, in step S13, the step in which an application process running on each of the computing nodes performs an operation on a device on the computing node to which the segment is mapped by operating on a segment in the virtual process address space includes the following steps:

[0068] In step S131, non-execution period data of the application process is loaded into the process space corresponding to the application process based on the address of the process space.

[0069] In an embodiment of the present disclosure, when an application process is loaded into the virtual memory space PVM, it is first determined based on a specific algorithm or mechanism (e.g., a hash or signature) into which specific process space in the virtual memory space PVM the application process should be loaded.

[0070] In step S132, when each of the computing nodes executes the application process, Application Process In the process space corresponding to Application Process Assign the corresponding execution period data.

[0071] In the embodiment of the present disclosure, after determining which specific process space in the virtual memory space PVM the application process should be loaded into, non-execution period data such as code segments and data segments of the application process are loaded into the specific process space, thereby ensuring that the virtual address of the application process in the virtual memory space PVM is always determined and remains permanently unchanged.

[0072] In step S133, this computing node Application Process to an address in a process space corresponding to the application process, in a physical memory in a computing node corresponding to the process space. Application Process Execute.

[0073] In an embodiment of the present disclosure, when a computing node executes a pre-loaded application process, the computing node first allocates and initializes the necessary execution-period data for the application process, such as a BSS segment, heap, and stack, in the process space of the virtual memory space PVM for the application process using its own PVMM. Then, the computing node uses its own PVMM to point the CPU's PC (Program Counter) pointer to the process's fixed process space address in the PVM. The BSS segment is a memory area for storing global variables and static variables that are uninitialized or initialized to 0 in the application program corresponding to the application process. The BSS segment is readable and writable, and is automatically cleared to 0 before the program is executed. The program counter is a register for storing the next instruction address in the CPU.

[0074] According to an embodiment of the present disclosure, the method further includes the following steps:

[0075] If any one of the application processes has a data transmission demand, the target pointer in the virtual process address space of the data corresponding to the data transmission demand is transferred to the target application process.

[0076] When data transmission between application processes is required, the data transmission may be understood as an application process needing other application processes to communicate data to it and share its own data with other application processes. CCOS does not require a conventional network communication method, and an application process simply transfers the address (i.e., target pointer) in virtual memory PVM of the communicated (requesting other application processes to transmit to it) or shared (sharing its own data with other application processes) data to another target application process.

[0077] The target application process accesses the virtual process address space through the object pointer to complete the data transfer.

[0078] According to the above embodiment, the target application process can directly access the communicated or shared data by the address of the communicated or shared data in the virtual memory space PVM. When an address in the virtual memory space PVM that a computing node needs to access is mapped to physical memory in another computing node, it is realized by a PVM_GMMU (Permanent Virtual Memory Global Memory Management Unit) in the computing node in a remote direct memory access manner. The remote direct memory access manner may be, for example, an access manner based on the RDMA protocol.

[0079] According to one embodiment of the present disclosure, the step S121 of respectively mapping the segment corresponding to the virtual process address space and the segment corresponding to the virtual file system space to physical memory in each of the computing nodes includes the following steps:

[0080] The physical memory provided by each of the computing nodes is registered in the virtual memory space.

[0081] In the embodiment of the present disclosure, only physical memory that is mapped to constitute a virtual memory space is registered in the virtual memory space, and locally held physical memory is not registered in the virtual memory space.

[0082] The size of the physical memory provided by each computing node may be different, and the specific size is determined by the PVMM in the computing node, which provides both the size of the physical memory and the corresponding physical address in the virtual memory space.

[0083] The physical memory registered in the virtual memory space is divided into physical memory segments having the same size as the segments.

[0084] As can be seen, the partitioned physical memory in the computing node is subdivided into physical memory segments of the same size as the segments of the virtual memory space PVM.

[0085] A segment corresponding to the virtual process address space and a segment corresponding to the virtual file system space are respectively mapped to the physical memory segments in each of the computing nodes.

[0086] In the embodiment of the present disclosure, the physical memory segments in a computing node and the segments in the virtual memory space have a corresponding relationship, that is, a physical memory segment in a computing node corresponds to only one segment in the virtual memory space, and of course, a segment in the virtual memory space is redundantly mapped to physical memory segments in multiple computing nodes.

[0087] According to an embodiment of the present disclosure, the step of mapping a segment corresponding to the virtual file system space to a physical memory in each of the computing nodes includes: The virtual file system is created according to a predetermined policy. space to physical memory segments of a storage device in each of said computing nodes.

[0088] In the embodiment of the present disclosure, a Permanent Virtual Memory Global File system Manager (PVM_GFSM) in the PVMM executed by each computing node manages the virtual file system. space The mapped physical memory segments may be persistently mapped to a non-volatile storage device local to the computing node according to a predetermined policy, for example, synchronous or asynchronous. The non-volatile storage device may be, for example, a storage device such as a hard disk.

[0089] The step of executing an operation on a segment in the virtual memory space of an application process on a device in the computing node to which the segment is mapped includes: The method includes a step in which an application process running on each of the computing nodes reads and writes data in the virtual file system space by the application process running on each of the computing nodes reading and writing data in a physical memory segment of a storage device on the computing node.

[0090] In the embodiment of the present disclosure, each computing node executes Application Process can read and write data directly from the virtual file system space through the pointer, thereby performing data read and write operations in the physical memory segments of the storage device in the computing node.

[0091] However, the data in the virtual file system space is of the type of system files, program files, etc., different from the non-execution period data and execution period data of the loaded or allocated application processes in the process space.

[0092] The data of each segment in the virtual file system space may be stored in non-volatile storage devices in different computing nodes in a redundant mapping manner and / or a distributed consistency manner in subsequent embodiments, thereby ensuring data security and consistency.

[0093] According to an embodiment of the present disclosure, the step of mapping the segment corresponding to the virtual process address space and the segment corresponding to the virtual file system space to physical memory in each of the computing nodes includes: The method includes a step of respectively mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to physical memory in each of the computing nodes using a redundant mapping scheme and / or a distributed consistency scheme.

[0094] In an embodiment of the present disclosure, the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space may be respectively mapped to physical memory in each of the computing nodes using a 1:N multi-copy scheme, an erasure code scheme, or other redundancy scheme.

[0095] Mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to the physical memory of each of the computing nodes in a 1:N multi-copy manner can be understood as follows: Each segment in the virtual memory space is copied N times to obtain the original segment and N copy segments, and the original segment and copy segment of each segment are then mapped to the physical memory of different computing nodes. Of course, each piece of data stored in a segment can be copied N times and stored in different computing nodes. Furthermore, if data in a segment of one computing node is lost, it can be restored using data in other computing nodes.

[0096] The use of erasure codes to respectively map the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to the physical memory of each computing node can be understood as follows: A plurality of segments are used as the body of an erasure code, an erasure code corresponding to this body is generated, and each of the plurality of segments is mapped to a computing node, and each erasure code is also mapped to a computing node, so that if data in any one of the segments is lost, the lost data can be obtained by erasure code reverse computing.

[0097] By using the redundant mapping method and / or the distributed consistency method to perform redundant backup of data within segments in the virtual memory space PVM, not only can data security be ensured and data loss be prevented, but also a situation can be avoided in which a computing node goes offline and data in the physical memory of that computing node cannot be read or written, thereby ensuring the stability of application process execution and avoiding the need for application processes to wait for a computing node to come online when a computing node goes offline, thereby improving the execution efficiency of application processes.

[0098] The distributed consistency scheme may be, for example, a distributed consistency scheme based on the RAFT algorithm, that is, consistency is achieved by selecting a leader computing node from each computing node and giving the leader computing node all responsibility for managing the copy log. The leader computing node receives log entries from each computing node, copies the log entries to segments in other computing nodes, and instructs other computing nodes to apply the log entries to their respective state machines in a secure manner.

[0099] As can be seen, the leader computing node may determine which segment of the virtual file system space a new log entry should be placed in without consulting other computing nodes. If the leader computing node goes down, a new leader computing node may be selected, thereby ensuring that the mapping relationship between the segments in the virtual memory space PVM and the physical memory segments on the computing node is always consistent when there are additional or additional computing nodes in the CCOS cluster. The above mapping may be completed in the PVM_GMMU of the PVMM on the computing node and may be maintained and recorded by the PVMM on the computing node.

[0100] According to one embodiment of the present disclosure, referring to FIG. 7, in step S122, the step of mapping local physical devices in each computing node to segments corresponding to the virtual device space and obtaining virtual devices corresponding to the local physical devices includes the following steps:

[0101] In step S1221, the local physical devices on each of the computing nodes are mapped to the virtual device space by the PVM_GDM on each of the computing nodes.

[0102] In embodiments of the present disclosure, local physical equipment may include physical equipment such as a graphics card or a computing accelerator card in a computing node.

[0103] In step S1222, a virtual device corresponding to the local physical device is created in the segment corresponding to the virtual device space.

[0104] In an embodiment of the present disclosure, a portion of the virtual memory space PVM is partitioned as a virtual device space shared by the cluster operating system, and different computing nodes can selectively map local physical devices in the computing nodes to the virtual device space through PVM_GDM (Permanent Virtual Memory Global Device Manager) in each PVMM, and create virtual devices in this virtual device space that correspond to the local physical devices.

[0105] In this way, the local physical devices in the computing nodes can be globally mapped into the virtual memory space, so that any application process can directly call the local physical devices in any computing node from the virtual memory space, thereby making full use of the local physical device resources in the computing nodes and solving the problem of low computing node resource utilization in cluster operating systems in cloud computing scenarios.

[0106] According to one embodiment of the present disclosure, based on FIG. 7, in step S13, the step of executing an operation on a segment in the virtual memory space of an application process on a device in the computing node to which the segment is mapped includes the following steps:

[0107] In step S134, the read / write operations for the virtual device of the application process running on each of the computing nodes are performed on the local physical device of the computing node to which the segment is mapped.

[0108] In an embodiment of the present disclosure, each application process running in the virtual memory space PVM can access a virtual device in the virtual device space, and the read and write operations of the application process to the virtual device in the virtual device space can be executed synchronously or asynchronously by the PVM_GDM of the PVMM on the local physical device of the corresponding computing node.

[0109] In this way, read and write operations on virtual devices in the virtual device space can be performed on local physical devices on the corresponding computing nodes, and application processes can use local physical devices on other computing nodes without manually or explicitly accessing the computing nodes, thereby realizing the insensitive and transparent use of local physical devices on other computing nodes, which greatly simplifies the development of distributed computing / applications, greatly improves the operating efficiency of application processes or computing tasks, and improves the convenience and efficiency of device access between nodes.

[0110] 8, the apparatus for implementing the cluster operating system includes a first mapping module 810, a second mapping module 820, and an execution module 830.

[0111] The first mapping module 810 is configured to map the physical memory provided by each computing node in the cluster operating system to obtain a virtual memory space, where the virtual memory space consists of multi-type virtual space segments.

[0112] A second mapping module 820 is configured to map the segments in the virtual memory space to devices in each of the computing nodes.

[0113] The execution module 830 is configured to execute operations on segments in the virtual memory space of an application process on a device in the computing node to which the segments are mapped, and the application process runs on each of the computing nodes in the cluster operating system.

[0114] According to an embodiment of the present disclosure, the multi-type virtual space includes a virtual process address space, a virtual file system space, and a virtual device space, and the second mapping module 820 further comprises: mapping a segment corresponding to the virtual process address space and a segment corresponding to the virtual file system space to physical memory in each of the computing nodes; The method is configured to map a local physical device at each of the computing nodes to a corresponding segment in the virtual device space to obtain a virtual device corresponding to the local physical device.

[0115] According to an embodiment of the present disclosure, the virtual process address space includes a plurality of process spaces, and the second mapping module 820 further comprises: determining a process space corresponding to an application process in the cluster operating system from the plurality of process spaces in the virtual process address space, the process space in the virtual process address space of each of the application processes being fixed; A segment of process space corresponding to the application process is configured to be mapped to physical memory on each of the computing nodes, respectively.

[0116] According to an embodiment of the present disclosure, the execution module 830 further comprises: loading non-execution period data of the application process into the process space corresponding to the application process based on the address of the process space; When each of the computing nodes executes the application process, Application Process In the process space corresponding to Application Process assigning execution period data corresponding to The computing node Application Process to an address in a process space corresponding to the application process, in a physical memory in a computing node corresponding to the process space. Application Process is configured to execute

[0117] According to one embodiment of the present disclosure, the device is a transmission module, If any one of the application processes has a data transmission demand, transfer a target pointer in the virtual process address space of data corresponding to the data transmission demand to a target application process; The method further includes a transmission module configured to cause the target application process to access the virtual process address space through the object pointer to complete a data transmission.

[0118] According to an embodiment of the present disclosure, the second mapping module 820 further comprises: registering physical memory provided by each of said computing nodes in said virtual memory space; Dividing the physical memory registered in the virtual memory space into physical memory segments having the same size as the segments; The virtual process address space and the virtual file system space are configured to be mapped to the physical memory segments in each of the computing nodes, respectively.

[0119] According to an embodiment of the present disclosure, the second mapping module 820 further comprises: The virtual file system is created according to a predetermined policy. space to physical memory segments of a storage device in each of the computing nodes; The execution module 830 further The application process running on each of the computing nodes is configured to read and write data in the virtual file system space by performing data reads and writes in the physical memory segments of the storage device on the computing node.

[0120] According to an embodiment of the present disclosure, the second mapping module 820 further comprises: The segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space are configured to be mapped to physical memory in each of the computing nodes using a redundant mapping scheme and / or a distributed consistency scheme.

[0121] According to an embodiment of the present disclosure, the second mapping module 820 further comprises: mapping local physical devices on each of said computing nodes to said virtual device space by a PVM_GDM on each of said computing nodes; A virtual device corresponding to the local physical device is created in a segment corresponding to the virtual device space.

[0122] According to an embodiment of the present disclosure, the execution module 830 further comprises: The segment is configured to perform read and write operations on the virtual device of an application process running on each of the computing nodes to a local physical device on the computing node to which the segment is mapped.

[0123] However, the above explanations for the embodiment of the method for implementing the cluster operating system also apply to the apparatus for implementing the cluster operating system of this embodiment, and will not be further described here.

[0124] To realize the above embodiment, the present disclosure further proposes a server 900. As shown in Fig. 9, the server 900 includes the device 800 for implementing the cluster operating system and implements the method for implementing the cluster operating system described above.

[0125] FIG. 10 is a block diagram of an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a cloud server, and multiple cloud servers may constitute a cluster operating system according to an embodiment of the present disclosure, and may be used to execute the steps of the method for implementing the cluster operating system and to perform the operations of the computing nodes in the cluster operating system. Referring to FIG. 10 , the electronic device 1900 includes one or more processors 1922 and a storage device 1932 for storing computer programs executable by the processors 1922. The computer programs stored in the storage device 1932 may include one or more modules, each corresponding to a set of instructions. The processor 1922 may be configured to execute the computer programs to execute the steps of the method for implementing the cluster operating system and to perform the operations of the computing nodes in the cluster operating system.

[0126] The electronic device 1900 may further include a power component 1926 and a communication component 1950. The power component 1926 may be configured to perform power management for the electronic device 1900, and the communication component 1950 may be configured to enable communication, such as wired or wireless communication, for the electronic device 1900. The electronic device 1900 may further include an input / output (I / O) interface 1958. The electronic device 1900 may operate an operating system, such as Windows Server®, Mac OS X®, Unix®, or Linux®, stored in a storage device 1932.

[0127] In another exemplary embodiment, there is further provided a computer-readable storage medium containing program instructions that, when executed by a processor, perform the steps of the method for implementing the cluster operating system described above and perform the operations of the computing nodes in the cluster operating system described above. For example, the computer-readable storage medium may be the storage device 1932 described above containing program instructions that, when executed by the processor 1922 of the electronic device 1900, complete the steps of the method for implementing the cluster operating system described above and perform the operations performed by the computing nodes in the cluster operating system described above.

[0128] In another exemplary embodiment, there is further provided a computer program product, the computer program product including a computer program executable by a programmable device, the computer program having code portions for, when executed by the programmable device, performing the method for implementing the above-described cluster operating system and for performing the operation of computing nodes in the aforementioned cluster operating system.

[0129] To realize the above embodiment, the present disclosure further proposes a computing processing device, the computing processing device comprising: a storage device in which computer readable code is stored; and one or more processors, wherein the computer readable code, when executed by the one or more processors, causes the computing processing device to perform a method for implementing the aforementioned cluster operating system.

[0130] To realize the above embodiment, the present disclosure further proposes a computer program including computer-readable code, which, when executed on a computing device, causes the computing device to perform the method for realizing the aforementioned cluster operating system.

[0131] To realize the above embodiment, the present disclosure further proposes a computer-readable storage medium on which the aforementioned computer program is stored.

[0132] FIG. 11 is a structural schematic diagram of a computing device according to an embodiment of the present disclosure. The computing device typically includes a processor 1110 and a computer program product or computer-readable medium in the form of a storage device 1130. The storage device 1130 may be, for example, an electronic storage device such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. The storage device 1130 has a storage space 1150 for program code 1151 for performing any of the steps in the above-described methods. For example, the storage space 1150 for program code may include respective program codes 1151 for implementing various steps in the above-described methods. These program codes can be read from or written to one or more computer program products. These computer program products include program code carriers such as a hard disk, a compact disk (CD), a memory card, or a floppy disk. Such computer program products are typically portable or fixed storage units, as shown in FIG. 12. The storage unit may have storage segments, storage spaces, etc. arranged similarly to storage device 1130 in the server of Figure 11. The program code may, for example, be compressed in a suitable format. Typically, the storage unit includes computer-readable code 1151', i.e., code readable by a processor, such as processor 1110, which, when executed by the server, causes the server to perform the steps in the methods described above.

[0133] In the description herein, references to "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with that embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, general references to the above terms do not necessarily refer to the same embodiment or example. Furthermore, specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, where not inconsistent, those skilled in the art may combine different embodiments or examples described herein, and features of different embodiments or examples.

[0134] It should be noted that the terms "first" and "second" are for descriptive purposes only and should not be understood to indicate or imply relative importance or the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the feature. In the description of this disclosure, "plurality" means at least two, e.g., two, three, etc., unless otherwise specifically limited.

[0135] Any process or method description in a flowchart or otherwise described herein may be understood as a module, segment, or portion representing executable instruction code including one or more steps for implementing a customized logical function or process. Also, the scope of the preferred embodiments of the present disclosure includes alternative implementations, which may not follow the order shown or discussed, and include performing functions essentially simultaneously or in reverse order based on the associated functionality, as should be understood by those skilled in the art.

[0136] The logic and / or steps depicted in flowcharts or otherwise described herein may be considered, for example, as a sequential listing of executable instructions to implement logical functions, and may be tangibly embodied on any computer-readable medium for use with or in combination with an instruction execution system, device, or apparatus (e.g., a computer-based system, a system including a processor, or other system capable of retrieving and executing instructions from an instruction execution system, device, or apparatus). For purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate, or transmit a program for use with or in combination with an instruction execution system, device, or apparatus. More specific examples (non-limiting list) of computer-readable media include an electrical connection having one or more wires (electronic devices), a portable computer disk box (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable editable read-only memory (EPROM or flash memory), a fiber optic device, and a portable optical disk read-only memory (CD-ROM). The computer readable medium may also be paper or other suitable medium on which the program may be printed, for example so that the paper or other medium can be optically scanned, edited, interpreted, or processed in any other suitable manner as needed to obtain the program electronically and store it in a computer memory device.

[0137] It should be understood that each part of the present disclosure may be realized by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods may be realized by software or firmware stored in a storage device and executed by an appropriate instruction execution system. For example, when realized by hardware, as in other embodiments, the hardware may be realized by any one or combination of techniques known in the art, such as a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application specific integrated circuit having appropriate combinational logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0138] Those skilled in the art can understand that realizing all or part of the steps carried out in the method of the above embodiments can be completed by instructing relevant hardware through a program, and the program may be stored in a computer-readable storage medium, and when the program is executed, it will include one or a combination of the steps of the method embodiments.

[0139] Note that the functional units in each embodiment of the present disclosure may be integrated into one processing module, each unit may exist physically separately, or two or more units may be integrated into one module. The integrated module may be realized in the form of hardware or in the form of a software functional module. When the integrated module is realized in the form of a software functional module and sold or used as an independent product, it may be stored in a computer-readable storage medium.

[0140] The storage medium mentioned above may be a read-only storage device, a magnetic disk, an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it should be understood that the above embodiments are merely illustrative and should not be construed as limitations of the present disclosure, and those skilled in the art may change, modify, substitute, and alter the above embodiments within the scope of the present disclosure.

Claims

1. 1. A method for implementing a cluster operating system, comprising: Mapping physical memory provided by each computing node in the cluster operating system to obtain a virtual memory space consisting of segments of a multi-type virtual space; mapping a plurality of said segments in said virtual memory space to devices on each of said computing nodes; Executing an operation on a segment in the virtual memory space of an application process on a device in the computing node to which the segment is mapped, the application process being executed on each of the computing nodes in the cluster operating system; 1. A method for implementing a cluster operating system, comprising:

2. The multi-type virtual space includes a virtual process address space, a virtual file system space, and a virtual device space; The step of mapping the plurality of segments in the virtual memory space to devices in each of the computing nodes comprises: mapping a segment corresponding to the virtual process address space and a segment corresponding to the virtual file system space to physical memory on each of the computing nodes; and mapping a local physical device at each of the computing nodes to a corresponding segment of the virtual device space to obtain a virtual device corresponding to the local physical device.

3. The virtual process address space includes a plurality of process spaces, and the step of mapping segments corresponding to the virtual process address space to physical memory in each of the computing nodes includes: determining a process space corresponding to an application process in the cluster operating system from a plurality of process spaces in the virtual process address space, wherein the process space in the virtual process address space of each of the application processes is fixed; and mapping a segment of process space corresponding to the application process to physical memory on each of the computing nodes, respectively.

4. The step of the application process executing on each of the computing nodes performing an operation on a device on the computing node to which the segment is mapped by performing an operation on a segment in the virtual process address space, includes: loading non-execution period data of the application process into the process space corresponding to the application process based on the address of the process space; When each of the computing nodes executes the application process, allocating execution period data corresponding to the application program in a process space corresponding to the application program; 4. The method of claim 3, further comprising: executing the application program in physical memory in the computing node corresponding to the process space by causing a program counter pointer in the computing node executing the application program to point to an address in the process space corresponding to the application process.

5. The method comprises: When a data transmission demand exists in any one of the application processes, transferring a target pointer in the virtual process address space of data corresponding to the data transmission demand to a target application process; The method according to any one of claims 2 to 4, further comprising the step of the target application process accessing the virtual process address space by means of the object pointer to complete the data transfer.

6. The step of mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to physical memory in each of the computing nodes includes: registering physical memory provided by each of said computing nodes in said virtual memory space; Dividing the physical memory registered in the virtual memory space into physical memory segments having the same size as the segments; and mapping a segment corresponding to the virtual process address space and a segment corresponding to the virtual file system space to the physical memory segments in each of the computing nodes.

7. Mapping segments corresponding to the virtual file system space to physical memory in each of the computing nodes includes: mapping segments corresponding to the virtual file system to physical memory segments of a storage device in each of the computing nodes according to a predetermined policy; The step of executing an operation on a segment in the virtual memory space of an application process on a device in the computing node to which the segment is mapped includes:

7. The method of claim 6, further comprising the step of causing an application process running on each of the computing nodes to read and write data in the virtual file system space by the application process running on each of the computing nodes reading and writing data in a physical memory segment of a storage device on the computing node.

8. the step of mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to physical memory in each of the computing nodes, 5. The method according to claim 2, further comprising the step of mapping the segments corresponding to the virtual process address space and the segments corresponding to the virtual file system space to physical memory in each of the computing nodes using a redundant mapping scheme and / or a distributed consistency scheme.

9. The step of mapping a local physical device at each computing node to a corresponding segment in the virtual device space to obtain a virtual device corresponding to the local physical device includes: mapping local physical devices on each of the computing nodes to the virtual device space by a Permanent Virtual Memory Global Device Manager (PVM_GDM) on each of the computing nodes; A method according to any one of claims 2 to 4, further comprising the step of: creating a virtual device corresponding to said local physical device in a segment corresponding to said virtual device space.

10. The step of executing an operation of an application process on a segment in the virtual memory space on a device in the computing node to which the segment is mapped includes:

10. The method of claim 9, further comprising: performing read and write operations on the virtual device of an application process running on each of the computing nodes to a local physical device on the computing node to which the segment is mapped.

11. An apparatus for implementing a cluster operating system, comprising: a first mapping module configured to map a physical memory provided by each computing node in the cluster operating system to obtain a virtual memory space consisting of segments of multiple space types; a second mapping module configured to map the segments in the virtual memory space to devices on each of the computing nodes; an execution module configured to execute operations on a segment in the virtual memory space of an application process on a device in the computing node to which the segment is mapped, the application process executing on each of the computing nodes in the cluster operating system; 1. An apparatus for implementing a cluster operating system, comprising:

12. 1. A computing processing device, comprising: a storage device in which computer readable code is stored; one or more processors, wherein the computer readable code, when executed by the one or more processors, causes the computing processing device to perform a method for implementing a cluster operating system according to any one of claims 1 to 4; 1. A computing processing device comprising:

13. 5. A computer program comprising computer readable code that, when executed on a computing processing device, causes the computing processing device to perform a method for implementing a cluster operating system according to any one of claims 1 to 4.

14. A computer-readable storage medium having the computer program of claim 13 stored thereon.

Images