Method and device for allocating vnc resources in supercomputing cluster

Abstract

The invention relates to the technical field of super-computing clusters, and discloses a vnc resource allocation method and device in a super-computing cluster, and the method comprises the steps: a control node splices a job script to obtain a job script file, compares the computing resource of a computing node and the vnc resource data of a display node with a preset scheduling condition, and sends the job script file to the control node; if the vnc resource data and the computing resources meet the preset scheduling condition, sending the job script file to the computing node; the computing node executes the job script file and starts the vnc service; after the operation script file is executed, the computing node obtains and executes the user application program; and the control node calls the operation state in the computing node, sends a vnc service stop request to the display node based on the operation state, and controls the display node to stop the vnc service. According to the method and the device, the vnc resources in the supercomputing cluster are reasonably allocated, node collapse caused by unreasonable allocation of the vnc resources is solved, and the graphic display effect of the application program is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of supercomputing clusters, and in particular to a method and device for allocating VNC resources in a supercomputing cluster. Background Art

[0002] VNC (virtual network computing) is a remote desktop sharing technology used to remotely access and control computers in a supercomputing cluster; VNC resource allocation plays an important role in improving system performance, ensuring user experience, enhancing system security, and facilitating system management and maintenance.

[0003] The VNC resource allocation method in the related supercomputing cluster does not consider the number of VNC services that can be started by the display node when allocating resources, causing the display node to start too many VNC services, resulting in poor graphics display effects and display node crashes. Summary of the invention

[0004] In view of this, the present invention provides a VNC resource allocation method within a supercomputing cluster to solve the problem that the related VNC resource allocation method within a supercomputing cluster does not consider the number of VNC services that can be started by a display node when allocating resources, causing the display node to start too many VNC services, thereby resulting in poor graphics display effect and display node crash.

[0005] In a first aspect, the present invention provides a method for allocating VNC resources in a supercomputing cluster, wherein the supercomputing cluster includes computing nodes, control nodes, and display nodes, and the computing nodes, control nodes, and display nodes are interconnected; the method includes:

[0006] The control node splices the job script to obtain the job script file, and compares the computing resources of the computing node and the vnc resource data of the display node with the preset scheduling conditions. If the vnc resource data and the computing resources meet the preset scheduling conditions, the job script file is sent to the computing node;

[0007] The computing node executes the job script file, and the display node uses the job script file to start the vnc service;

[0008] When the job script file is executed, the computing node obtains and executes the user application;

[0009] The control node retrieves the job status in the computing node, and sends a VNC service stop request to the display node based on the job status;

[0010] The display node receives the vnc service stop request and stops the vnc service based on the vnc service stop request.

[0011] The VNC resource allocation method in the supercomputing cluster provided by the present embodiment obtains a job script file by splicing the job scripts of the control nodes, and compares the computing resources of the computing nodes and the VNC resource data of the display nodes with the preset scheduling conditions. If the VNC resource data meets the preset scheduling conditions, the job script file is sent to the computing nodes. When submitting a job, reasonable application of job resources is achieved by uniformly allocating VNC resources, thereby avoiding the problem of poor VNC display effect caused by excessive resource occupation and avoiding deadlock or node crash caused by loop waiting. The computing nodes execute the job script files, and the display nodes use the job script files to start VNC. nc service; further, when the job script file is executed, the computing node obtains and executes the user application; the control node calls the job status in the computing node, and sends a vnc service stop request to the display node based on the job status; the display node receives the vnc service stop request, and stops the vnc service based on the vnc service stop request; the vnc service that is no longer in use is closed in time to avoid the waste of system resources; finally, the vnc resource allocation result in the supercomputing cluster is obtained, the reasonable allocation of vnc resources in the supercomputing cluster is realized, the node crash problem caused by unreasonable vnc resource allocation is solved, and the graphic display effect of the application is improved.

[0012] In an optional implementation, the control node includes a manager service and a slurmctld service; the control node splices the job script to obtain a job script file, and compares the computing resources of the computing node and the vnc resource data of the display node with the preset scheduling conditions, and if the vnc resource data and the computing resources meet the preset scheduling conditions, the job script file is sent to the computing node, including:

[0013] The manager service assembles the job scripts, obtains the job script file, and sends the job script file to the slurmctld service;

[0014] The slurmctld service adds the job script file to the waiting queue and assigns a job identifier to the job script file;

[0015] The slurmctld service determines whether the computing resources of the computing node meet the job requirements and displays whether the node has vnc resources;

[0016] When the computing resources of the computing node meet the job requirements and the display node has vnc resources, the slurmctld service schedules the job script files in the waiting queue to the computing node.

[0017] The method for allocating vnc resources in a supercomputing cluster provided by the present embodiment splices job scripts through a manager service to obtain a job script file, and sends the job script file to a slurmctld service; the slurmctld service adds the job script file to a waiting queue and allocates a job identifier to the job script file; the slurmctld service determines whether the computing resources of a computing node meet the job requirements and whether a vnc resource exists on the display node; when the computing resources of the computing node meet the job requirements and the display node has vnc resources, the slurmctld service dispatches the job script file in the waiting queue to the computing node; when the job script file is dispatched to the computing node, a reasonable application of job resources is achieved by uniformly allocating vnc resources, thereby avoiding the problem of poor vnc display effect caused by occupying too many resources, and avoiding deadlock or node crash caused by loop waiting.

[0018] In an optional implementation, the computing node executes the job script file, and the display node uses the job script file to start the vnc service, including:

[0019] The computing node sends a request instruction to the manager service;

[0020] The computing node receives the vnc node information sent by the manager service, and sends the vnc node information to the display node; wherein the vnc node information is the information retrieved by the manager service according to the request instruction;

[0021] Display the node connected to the vnc node information, and start the vnc service based on the vnc node information;

[0022] The display node obtains the vnc service information during the execution of the vnc service, and sends the vnc service information to the manager service through the computing node; wherein the manager service stores the vnc service information in the job data and updates the vnc resources of the display node.

[0023] The method for allocating vnc resources in a supercomputing cluster provided by the present embodiment sends a request instruction to a manager service through a computing node; the computing node receives the vnc node information sent by the manager service, and sends the vnc node information to a display node; wherein the vnc node information is information retrieved by the manager service according to the request instruction; the manager service selects a display node for starting vnc through a scheduling algorithm, thereby avoiding starting the vnc service on a small number of display nodes, improving the reliability of the vnc service, and improving the operating efficiency of the display node; the display node receives the vnc node information, and starts the vnc service based on the vnc node information; the display node obtains the vnc service information during the execution of the vnc service, and sends the vnc service information to the manager service through the computing node; wherein the manager service stores the vnc service information in the job data, and updates the vnc resources of the display node, thereby realizing the startup of the vnc service in the display node, and laying a foundation for the graphic display of the user application.

[0024] In an optional implementation, the control node retrieves the job status in the computing node, and sends a VNC service stop request to the display node based on the job status, including:

[0025] The control node retrieves the job status in the computing node, and when the job status is the job end status, the job cancellation status, or the job failure status, a vnc service stop request is sent to the display node corresponding to the job status.

[0026] The method for allocating VNC resources in a supercomputing cluster provided by this embodiment retrieves the job status in a computing node through a control node, and when the job status is a job end status, a job cancellation status, or a job failure status, a VNC service stop request is sent to a display node corresponding to the job status; the recycling of VNC resources in a supercomputing cluster is realized, invalid VNC resource occupation is cleared, and the utilization rate of VNC resources is improved.

[0027] In an optional implementation, the control node splices the job script to obtain a job script file, and compares the computing resources of the computing node and the vnc resource data of the display node with the preset scheduling conditions. If the vnc resource data and the computing resources meet the preset scheduling conditions, before sending the job script file to the computing node, it also includes:

[0028] The hardware configuration data of the display node is obtained, the number of VNC service startups corresponding to the display node is calculated according to the hardware configuration data of the display node, and VNC resources are configured for the computing node and the control node based on the number of VNC service startups.

[0029] The method for allocating vnc resources in a supercomputing cluster provided in this embodiment obtains the hardware configuration data of the display node, calculates the number of vnc service startups corresponding to the display node according to the hardware configuration data of the display node, and configures vnc resources for the computing node and the control node based on the number of vnc service startups; implements the check before vnc resource allocation, and lays a foundation for scheduling jobs on the control node.

[0030] In an optional implementation, it also includes:

[0031] When the display node stops the vnc service, the computing node obtains the current number of available vncs and increases the current number of available vncs by one.

[0032] The method for allocating vnc resources in a supercomputing cluster provided in this embodiment is as follows: when a display node stops the vnc service, the computing node obtains the current available vnc number and increases the current available vnc number by one; by adjusting the current available vnc number of the computing node, the vnc resource is recovered and the utilization rate of the vnc resources is improved.

[0033] In a second aspect, the present invention provides a device for allocating vnc resources in a supercomputing cluster, the device comprising a supercomputing cluster, the supercomputing cluster comprising computing nodes, control nodes and display nodes, the computing nodes, control nodes and display nodes being interconnected;

[0034] The control node is used to splice the job script to obtain the job script file, and compare the computing resources of the computing node and the vnc resource data of the display node with the preset scheduling conditions. If the vnc resource data and the computing resources meet the preset scheduling conditions, the job script file is sent to the computing node;

[0035] Compute nodes, used to execute job script files;

[0036] Display node, used to start vnc service using job script file;

[0037] The computing node is used to obtain and execute the user application after the job script file is executed;

[0038] The control node is used to retrieve the job status in the computing node and send a VNC service stop request to the display node based on the job status;

[0039] The display node is used to receive a vnc service stop request and stop the vnc service based on the vnc service stop request.

[0040] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the VNC resource allocation method within a supercomputing cluster of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0041] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for allocating VNC resources within a supercomputing cluster of the first aspect or any corresponding embodiment thereof.

[0042] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, the computer instructions being used to enable a computer to execute the method for allocating VNC resources within a supercomputing cluster according to the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 is a structural block diagram of a VNC resource allocation device in a supercomputing cluster according to an embodiment of the present invention;

[0045] Figure 2 It is a flow chart of a method for allocating vnc resources in a supercomputing cluster according to an embodiment of the present invention;

[0046] Figure 3 It is a flowchart of another method for allocating VNC resources in a supercomputing cluster according to an embodiment of the present invention;

[0047] Figure 4 It is a flowchart of another method for allocating VNC resources in a supercomputing cluster according to an embodiment of the present invention;

[0048] Figure 5 Schematic diagram of a method for allocating VNC resources in a supercomputing cluster according to an embodiment of the present invention

[0049] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0051] Supercomputing refers to the use of aggregated computing power to handle data-intensive computing jobs that cannot be completed by standard workstations, including simulation, modeling, and rendering. Data-intensive jobs require a large amount of calculations, and a general-purpose computer cannot complete the work within a reasonable time, or the job cannot be executed due to the large amount of data required and limited available resources. Supercomputing can effectively overcome time and resource limitations by integrating the computing power of multiple units.

[0052] Slurm (simple linux utility for resource management) is an open source cluster resource management and job scheduling system, which is widely used in supercomputers. Slurm can manage multiple resources such as CPU (Central Processing Unit), GPU (Graphics Processing Unit), memory and bandwidth, and allocate corresponding resources to jobs according to job requirements.

[0053] VNC is a remote desktop sharing technology used to remotely access and control computers. It is achieved by transmitting the remote computer's screen to the local computer and transmitting input to the remote computer at the same time. The remote computer screen is viewed through the VNC viewer on the local computer, achieving remote control and viewing.

[0054] When submitting a job using a supercomputing cluster, you need to write a job script to specify the CPU (central processing unit), GPU (graphic processing unit), memory and other resources required by the job. At the same time, you also need to configure environment variables and set the application you want to run. Taking the cluster with the slurm cluster scheduler installed as an example, after completing the job script writing, the user can submit the job script file to the slurm queue. If the cluster idle resources meet the job requirements, slurm will allocate resources to the job and start executing the job. If the cluster idle resources cannot temporarily meet the job requirements, slurm will schedule the job to the waiting queue until the cluster idle resources meet the job requirements.

[0055] There are two application scenarios for the graphical display of jobs in a supercomputing cluster:

[0056] The first application scenario is before job submission or after job execution; before job submission, users can adjust job parameters and process input files in a visual way; after job execution, users can view job execution results in a visual way; in this application scenario, job operation and application graphical display are separate and independent of each other; before submitting a job, the cluster will start a vnc service, and the user connects to the cluster's vnc through a vnc client; the user can preprocess the job input data and save the preprocessing results. After completing the preprocessing, the user closes the vnc service and submits the job to slurm, where the job input is the saved preprocessing results; after the job is submitted, the user waits for the job to be scheduled and executed. At the same time, the graphical interface is not displayed during the job execution.

[0057] In the second application scenario, the supercomputing cluster performs simulation calculations and realizes graphical display at the same time; in this application scenario, the operation process of the job includes steps such as starting the vnc service, executing the simulation calculation, and closing the vnc service; taking the industrial simulation job as an example, the specific process is as follows: first, before the simulation calculation program is started, a vnc service is started; then, the vnc service is used to display the simulation calculation program graphically, and the user can view the graphical interface of the job running process in real time through the vnc client; finally, when the simulation calculation program is finished, the vnc service is closed.

[0058] For the second scenario, the vnc resource allocation method in the relevant supercomputing cluster realizes the graphical display of the application by modifying the job script file: before running the user application, the vnc service will be started first; when slurm executes the job, the vnc service is first started on the display node, and then the user's application is run on the computing node, and the graphical display of the application is output to the vnc service. Finally, the user can operate and view the application locally through the vnc client; the vnc resource allocation method in the relevant supercomputing cluster schedules the CPU, GPU and memory of the computing node as the resources required for the job, and when the resources required for the job meet the conditions, slurm executes the job; however, starting the vnc service on the display node consumes GPU and memory resources, so the number of display nodes that can start the vnc service is limited.

[0059] However, since slrum or other supercomputer schedulers do not consider the number of vnc services that can be started by display nodes when allocating resources, too many vnc services are started, resulting in poor display of application graphics and display node crashes.

[0060] To solve the above technical problems, an embodiment of the present invention provides a method for allocating VNC resources in a supercomputing cluster, such as Figure 1 As shown, the supercomputer cluster includes computing nodes 101, control nodes 102 and display nodes 103, and the computing nodes 101, control nodes 102 and display nodes 103 are connected to each other; the control node 102 includes manager service and slurmctld service; the computing node includes slurmd service; the display node includes agent service and vnc server service.

[0061] Among them, the supercomputer cluster is a cluster composed of multiple computers, consisting of a control node 102, multiple computing nodes 101 and multiple display nodes 103. The control node 102 is responsible for managing the cluster; the computing node 101 is responsible for running the application, and the display node 103 is responsible for graphically displaying the application; slurmd is the execution service of slurm, responsible for executing jobs; the slurmctld service is the scheduling service of slurm, responsible for managing the CPU, GPU and memory resources of the cluster, and allocating resources to jobs; the manager service is responsible for managing vnc resources, submitting jobs and monitoring job status; the agent service is responsible for executing commands issued by the manager service; the vnc server service is responsible for starting the vnc service.

[0062] According to an embodiment of the present invention, an embodiment of a method for allocating VNC resources within a supercomputing cluster is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0063] In this embodiment, a method for allocating VNC resources in a supercomputing cluster is provided, which can be used in the above-mentioned supercomputing cluster. Figure 2 is a flow chart of a method for allocating vnc resources within a supercomputing cluster according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0064] Step S201, the control node splices the job script to obtain a job script file, and compares the computing resources of the computing node and the vnc resource data of the display node with the preset scheduling conditions. If the vnc resource data and the computing resources meet the preset scheduling conditions, the job script file is sent to the computing node.

[0065] Specifically, the vnc service is abstracted as a system resource for management and allocation, so that the vnc service can be used as a uniformly manageable system resource of the supercomputing cluster and be associated with resources such as the CPU to a certain extent.

[0066] Furthermore, the hardware configuration data of the display node is obtained, the number of VNC service startups corresponding to the display node is calculated according to the hardware configuration data of the display node, and VNC resources are configured for the computing node and the control node based on the number of VNC service startups.

[0067] Furthermore, before the supercomputing cluster is running, vnc resources are manually configured through slurm. According to the hardware configuration of the display nodes, the total number of vnc services that can be started by all display nodes is calculated, and the total number of vnc services that can be started is written into the configuration file of the slurmd service and the slurmd service is started; the IP address of the display node and the number of vnc services that can be started by each display node are recorded in the manager service.

[0068] Step S202, the computing node executes the job script file, and the display node starts the vnc service using the job script file.

[0069] Step S203: After the job script file is executed, the computing node obtains and executes the user application program.

[0070] Step S204: the control node retrieves the job status in the computing node, and sends a VNC service stop request to the display node based on the job status.

[0071] Specifically, the control node retrieves the job status in the computing node, and when the job status is a job end status, a job cancellation status, or a job failure status, a vnc service stop request is sent to a display node corresponding to the job status.

[0072] Specifically, the manager service calls the slurm interface to check the job status; when the job status is ended, canceled or failed, the manager node sends a request to stop VNC to the agent service of the display node corresponding to the job.

[0073] Furthermore, the manager service polls and checks the job status, reclaims relevant resources for jobs with a status of "ended, canceled, or failed", and promptly closes the vnc service that is no longer in use.

[0074] Step S205: the display node receives the vnc service stop request, and stops the vnc service based on the vnc service stop request.

[0075] Specifically, after the display node stops the vnc service, the computing node obtains the current available vnc number and increases the current available vnc number by one.

[0076] Furthermore, when the display node stops the vnc service, the manager service updates the data related to the display node and sends a request to stop the vnc to the agent service of the display node corresponding to the operation; the agent service stops the corresponding vnc service.

[0077] The VNC resource allocation method in the supercomputing cluster provided by the present embodiment obtains a job script file by splicing the job scripts of the control nodes, and compares the computing resources of the computing nodes and the VNC resource data of the display nodes with the preset scheduling conditions. If the VNC resource data and the computing resources meet the preset scheduling conditions, the job script file is sent to the computing nodes. When submitting a job, reasonable application of job resources is achieved by uniformly allocating VNC resources, thereby avoiding the problem of poor VNC display effect caused by excessive resource occupation and avoiding deadlock or node crash caused by loop waiting. The computing nodes execute the job script files, and the display nodes start the job script files using the job script files. The invention discloses a method for starting a vnc service; further, when the job script file is executed, the computing node obtains and executes the user application; the control node retrieves the job status in the computing node, and sends a vnc service stop request to the display node based on the job status; the display node receives the vnc service stop request, and stops the vnc service based on the vnc service stop request; the vnc service that is no longer used is closed in time, so as to avoid the waste of system resources; finally, the vnc resource allocation result in the supercomputing cluster is obtained, so as to realize the reasonable allocation of vnc resources in the supercomputing cluster, solve the node crash problem caused by the unreasonable allocation of vnc resources, and improve the graphic display effect of the application.

[0078] In this embodiment, a method for allocating VNC resources in a supercomputing cluster is provided, which can be used in the above-mentioned supercomputing cluster. Figure 3 is a flow chart of a method for allocating vnc resources within a supercomputing cluster according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0079] Step S301, the control node splices the job script to obtain the job script file, and compares the computing resources of the computing node and the vnc resource data of the display node with the preset scheduling conditions. If the vnc resource data and the computing resources meet the preset scheduling conditions, the job script file is sent to the computing node.

[0080] Specifically, the above step S301 includes:

[0081] Step S3011, the manager service splices the job scripts to obtain a job script file, and sends the job script file to the slurmctld service.

[0082] Specifically, the manager service splices the job script and adds related commands for starting VNC before the command for starting the user application in the job script.

[0083] Step S3012, the slurmctld service adds the job script file to the waiting queue and allocates a job identifier for the job script file.

[0084] Specifically, the manager service submits the job script file to the slurmctld service, and the slurmctld service puts the job into a waiting queue, assigns a job id to the job, and sends job-related information to the manager service.

[0085] Step S3013, the slurmctld service determines whether the computing resources of the computing node meet the job requirements, and displays whether the node has vnc resources.

[0086] Step S3014, when the computing resources of the computing node meet the job requirements and the display node has vnc resources, the slurmctld service dispatches the job script file in the waiting queue to the computing node.

[0087] Specifically, the slurmctld service checks whether the resources of the computing node meet the needs of the job, and checks whether the display node has vnc resources; when both conditions are met, the slurmctld service takes the job in the job script out of the waiting queue, schedules it to the computing node, allocates resources such as CPU and memory to the job, and reduces the number of currently available vncs by 1.

[0088] Step S302: The computing node executes the job script file, and the display node uses the job script file to start the vnc service. Figure 2 Step S202 of the illustrated embodiment will not be described in detail here.

[0089] Step S303: When the job script file is executed, the computing node obtains and executes the user application. Figure 2 Step S203 of the illustrated embodiment will not be described in detail here.

[0090] Step S304: The control node retrieves the job status in the computing node and sends a VNC service stop request to the display node based on the job status. Figure 2 Step S204 of the illustrated embodiment will not be described in detail here.

[0091] Step S305: The display node receives the vnc service stop request and stops the vnc service based on the vnc service stop request. Figure 2 Step S205 of the illustrated embodiment will not be described in detail here.

[0092] The method for allocating vnc resources in a supercomputing cluster provided by the present embodiment splices job scripts through a manager service to obtain a job script file, and sends the job script file to a slurmctld service; the slurmctld service adds the job script file to a waiting queue and allocates a job identifier to the job script file; the slurmctld service determines whether the computing resources of a computing node meet the job requirements and whether a vnc resource exists on the display node; when the computing resources of the computing node meet the job requirements and the display node has vnc resources, the slurmctld service dispatches the job script file in the waiting queue to the computing node; when the job script file is dispatched to the computing node, a reasonable application of job resources is achieved by uniformly allocating vnc resources, thereby avoiding the problem of poor vnc display effect caused by occupying too many resources, and avoiding deadlock or node crash caused by loop waiting.

[0093] In this embodiment, a method for allocating VNC resources in a supercomputing cluster is provided, which can be used for the above-mentioned electronic device. Figure 4 is a flow chart of a method for allocating vnc resources within a supercomputing cluster according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0094] Step S401: The control node assembles the job scripts to obtain the job script file, and compares the computing resources of the computing node and the vnc resource data of the display node with the preset scheduling conditions. If the vnc resource data and computing resources meet the preset scheduling conditions, the job script file is sent to the computing node. Figure 3 Step S301 of the illustrated embodiment will not be described in detail here.

[0095] Step S402, the computing node executes the job script file, and the display node starts the vnc service using the job script file.

[0096] Specifically, the above step S402 includes:

[0097] Step S4021, the computing node sends a request instruction to the manager service.

[0098] Specifically, the slurmd service of the computing node executes the job script file, and executes the commands in the script file in sequence.

[0099] Step S4022, the computing node receives the vnc node information sent by the manager service, and sends the vnc node information to the display node; wherein the vnc node information is the information retrieved by the manager service according to the request instruction.

[0100] Specifically, during the process of the computing node executing the commands in the script file, the job script sends a request to the manager service to obtain the information of the display node used to start vnc; wherein the manager service selects the display node used to start vnc through the scheduling algorithm, and sends the vnc node information to the job script file in the computing node.

[0101] Furthermore, a load balancing algorithm is configured in the manager service, which is based on a weighted dynamic round-robin mechanism to make the load of multiple display nodes more even and avoid the VNC service being started on a small number of display nodes.

[0102] Step S4023, display the node to vnc node information, and start the vnc service based on the vnc node information.

[0103] Specifically, the job script file generates a vnc password on the display node and starts the vnc service; the job script configures the environment variable value on the computing node to be "DISPLAY (value of the variable)".

[0104] Step S4024, the display node obtains the vnc service information during the execution of the vnc service, and sends the vnc service information to the manager service through the computing node; wherein the manager service stores the vnc service information in the job data, and updates the vnc resources of the display node.

[0105] Specifically, the display node sends vnc service information such as vnc password and port to the manager service; the manager service stores the vnc service information in the job data and updates the vnc resources of the display node.

[0106] Step S403: After the job script file is executed, the computing node obtains and executes the user application program.

[0107] Specifically, after updating the vnc resources of the display node, the computing node obtains and executes the user application; the graphics of the user application will be displayed in the vnc, and the user obtains the vnc's ip, port, password and other information through the manager service, uses the vnc client to connect to the vnc, and can operate and view the application.

[0108] Step S404: The control node retrieves the job status in the computing node and sends a VNC service stop request to the display node based on the job status. Figure 3 Step S304 of the illustrated embodiment will not be described in detail here.

[0109] Step S405: The display node receives the vnc service stop request and stops the vnc service based on the vnc service stop request. Figure 3Step S305 of the illustrated embodiment will not be described in detail here.

[0110] The method for allocating vnc resources in a supercomputing cluster provided by the present embodiment sends a request instruction to a manager service through a computing node; the computing node receives the vnc node information sent by the manager service, and sends the vnc node information to a display node; wherein the vnc node information is information retrieved by the manager service according to the request instruction; the manager service selects a display node for starting vnc through a scheduling algorithm, thereby avoiding starting the vnc service on a small number of display nodes, improving the reliability of the vnc service, and improving the operating efficiency of the display node; the display node receives the vnc node information, and starts the vnc service based on the vnc node information; the display node obtains the vnc service information during the execution of the vnc service, and sends the vnc service information to the manager service through the computing node; wherein the manager service stores the vnc service information in the job data, and updates the vnc resources of the display node, thereby realizing the startup of the vnc service in the display node, and laying a foundation for the graphic display of the user application.

[0111] The following is a specific example to illustrate the specific steps of a method for allocating VNC resources in a supercomputing cluster.

[0112] Embodiment 1:

[0113] like Figure 6 As shown, the specific steps of the VNC resource allocation method in the supercomputing cluster include:

[0114] (1) Initialize the control node, computing node, and display node: manually configure VNC resources; calculate the total number of VNC services that can be started by all display nodes based on the hardware configuration of the display nodes, write the total number of VNC services that can be started into the slurm configuration file and start slurm; record the IP address of the display node, the number of VNC services that can be started by each node, and other information into the manager service.

[0115] (2) The manager service splices the job script and adds the relevant command to start VNC before the command to start the user application in the job script.

[0116] (3) The manager service submits the job script file to the slurmctld service. The slurmctld service puts the job into the waiting queue, assigns a job ID to the job and sends job-related information to the manager service.

[0117] (4) The slurmctld service checks whether the resources of the computing node meet the needs of the job, and checks whether the display node has vnc resources; when both conditions are met, the slurmctld service takes the job out of the waiting queue, schedules it to the computing node, allocates resources such as CPU and memory to the job, and reduces the number of currently available vncs by 1.

[0118] (5) The slurmd service in the computing node executes the job script file and executes the commands in the script file in sequence.

[0119] (6) The job script in the computing node sends a request to the manager service to obtain the vnc node information; the manager service selects the display node for starting vnc through the scheduling algorithm and sends the vnc node information to the job script file in the computing node.

[0120] (7) The job script file in the computing node generates a vnc password on the display node and starts the vnc service; sets the computing node configuration environment variable value to "DISPLAY"; and then sends the vnc password, port and other information to the manager service.

[0121] (8) The manager service stores the vnc service information in the job data and updates the vnc resources of the display node.

[0122] (9) After updating the vnc resources of the display node, the job script file in the computing node executes the user application; the graphics of the user application will be displayed in the vnc, and the user obtains the vnc's ip, port, password and other information through the manager service, and uses the vnc client to connect to the vnc to operate and view the application.

[0123] (10) The manager service calls the slurm interface to check the job status; when the job status is ended, canceled or failed, the manager node sends a request to stop VNC to the agent service of the display node corresponding to the job.

[0124] (11) The agent service stops the corresponding vnc service.

[0125] (12) The manager service updates and displays the data related to the node.

[0126] (13) slurm increases the number of currently available vncs by 1.

[0127] In this embodiment, a VNC resource allocation device within a supercomputing cluster is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0128] This embodiment provides a VNC resource allocation device in a supercomputing cluster, the device comprising a supercomputing cluster, such as Figure 1 As shown, the supercomputing cluster includes a computing node 101, a control node 102, and a display node 103, and the computing node 101, the control node 102, and the display node 103 are interconnected;

[0129] The control node 102 is used to splice the job script to obtain the job script file, and compare the computing resources of the computing node and the vnc resource data of the display node with the preset scheduling conditions. If the vnc resource data and the computing resources meet the preset scheduling conditions, the job script file is sent to the computing node;

[0130] Computing node 101, used to execute job script files;

[0131] Display node 103 is used to start the vnc service using the job script file;

[0132] Computing node 101, used to obtain and execute the user application after the job script file is executed;

[0133] The control node 102 is used to retrieve the job status in the computing node and send a VNC service stop request to the display node based on the job status;

[0134] The display node 103 is used to receive a vnc service stop request and stop the vnc service based on the vnc service stop request.

[0135] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0136] In this embodiment, a VNC resource allocation device within a supercomputing cluster is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0137] The embodiment of the present invention also provides a computer device having the above Figure 1A VNC resource allocation device within a supercomputing cluster is shown.

[0138] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0139] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0140] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0141] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0142] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0143] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0144] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0145] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0146] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0147] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for allocating vnc resources in a supercomputing cluster, characterized in that: The supercomputing cluster includes computing nodes, control nodes and display nodes, and the computing nodes, control nodes and display nodes are interconnected; the method includes: The control node splices the job scripts to obtain a job script file, and compares the computing resources of the computing node and the vnc resource data of the display node with the preset scheduling conditions, and sends the job script file to the computing node if the vnc resource data and the computing resources meet the preset scheduling conditions; The computing node executes the job script file, and the display node starts the vnc service using the job script file; When the job script file is executed, the computing node obtains and executes the user application program; The control node retrieves the job status in the computing node, and sends a VNC service stop request to the display node based on the job status; The display node receives the vnc service stop request, and stops the vnc service based on the vnc service stop request.

2. The method according to claim 1, characterized in that The control node includes a manager service and a slurmctld service; the control node splices the job script to obtain a job script file, and compares the computing resources of the computing node and the vnc resource data of the display node with the preset scheduling conditions, and if the vnc resource data and the computing resources meet the preset scheduling conditions, the job script file is sent to the computing node, including: The manager service splices the job scripts to obtain the job script file, and sends the job script file to the slurmctld service; The slurmctld service adds the job script file to a waiting queue and allocates a job identifier to the job script file; The slurmctld service determines whether the computing resources of the computing node meet the job requirements, and whether the display node has vnc resources; When the computing resources of the computing node meet the job requirements and the display node has vnc resources, the slurmctld service dispatches the job script file in the waiting queue to the computing node.

3. The method according to claim 1, characterized in that: The computing node executes the job script file, and the display node uses the job script file to start the vnc service, including: The computing node sends a request instruction to the manager service; The computing node receives the vnc node information sent by the manager service, and sends the vnc node information to the display node; wherein the vnc node information is information retrieved by the manager service according to the request instruction; The display node receives the vnc node information and starts the vnc service based on the vnc node information; The display node obtains vnc service information during the execution of the vnc service, and sends the vnc service information to the manager service through the computing node; wherein the manager service stores the vnc service information in the job data and updates the vnc resources of the display node.

4. The method according to claim 1, characterized in that: The control node retrieves the job status in the computing node, and sends a VNC service stop request to the display node based on the job status, including: The control node retrieves the job status in the computing node, and when the job status is a job end status, a job cancellation status, or a job failure status, sends the VNC service stop request to the display node corresponding to the job status.

5. The method according to claim 1, characterized in that The control node splices the job script to obtain a job script file, and compares the computing resources of the computing node and the VNC resource data of the display node with the preset scheduling conditions. If the VNC resource data and the computing resources meet the preset scheduling conditions, before sending the job script file to the computing node, the method further includes: The hardware configuration data of the display node is obtained, the number of VNC service startups corresponding to the display node is calculated according to the hardware configuration data of the display node, and VNC resources are configured for the computing node and the control node based on the number of VNC service startups.

6. The method according to claim 1, characterized in that Also includes: After the display node stops the vnc service, the computing node obtains the current available vnc quantity and increases the current available vnc quantity by one.

7. A VNC resource allocation device in a supercomputing cluster, characterized in that: The device includes a supercomputing cluster, which includes computing nodes, control nodes and display nodes, and the computing nodes, control nodes and display nodes are interconnected; The control node is used to splice the job script to obtain a job script file, and compare the computing resources of the computing node and the vnc resource data of the display node with the preset scheduling conditions, and send the job script file to the computing node if the vnc resource data and the computing resources meet the preset scheduling conditions; The computing node is used to execute the job script file; The display node is used to start the vnc service using the job script file; The computing node is used to obtain and execute the user application program after the job script file is executed; The control node is used to retrieve the job status in the computing node, and send a VNC service stop request to the display node based on the job status; The display node is used to receive the vnc service stop request and stop the vnc service based on the vnc service stop request.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the VNC resource allocation method within a supercomputing cluster according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the VNC resource allocation method within a supercomputing cluster according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to enable a computer to execute the VNC resource allocation method within a supercomputing cluster according to any one of claims 1 to 6.

Images