Method for optimizing resource use efficiency, computer system and medium

Abstract

The invention provides a method for optimizing resource use efficiency, a computer system and a medium, and the method comprises the steps: querying a resource table which comprises usable resource lists corresponding to CAE software, and representing priority information of using the resource lists in sequence; determining resource lists according to job requirements, and checking idle conditions of all resource queues in sequence according to the sequence of the resource lists to obtain calculation queue information: when resources required by jobs are met, the calculation queue information is the resource queue information; if not, the calculation queue information is resource queue information with the minimum job queue waiting number; transmitting calculation queue information to a scheduling system, and performing calculation according to the calculation queue information; and after calculation is completed, releasing resources. According to the invention, the computing power and the resource utilization efficiency of a high-performance computing platform can be improved.

Description

Technical Field

[0001] The present invention relates to the automotive field, and in particular to a method, a computer system and a medium for optimizing resource utilization efficiency. Background Art

[0002] In high-performance computing, the mainstream scheduling platforms used by most computing platforms include Altair PBS Professional or open PBS, Sugon's SLURM, IBM LSF, etc. When jobs are submitted for calculation, these scheduling platforms will use default (script method) or manually specified (graphic method) computing resources for calculation.

[0003] However, the default method will cause the busy queue to be very busy and the idle queue to be very idle; the manual method requires checking the current usage of computing resources first, and then selecting the idle queue for calculation based on the usage. This operation is neither convenient nor accurate (time difference). Therefore, the computing platform often has some resource queues that are fully used (hot queues, busy), while some resource queues are ignored (cold queues, idle). In addition, the license fees of most CAE (computer-aided engineering) software are relatively expensive, and the low usage rate of licenses indirectly leads to waste of resources. Summary of the invention

[0004] In view of the defects in the prior art, the purpose of the present invention is to provide a method, a computer system and a medium for optimizing resource utilization efficiency, so as to increase the computing power and resource utilization efficiency of a high-performance computing platform.

[0005] In order to achieve the above-mentioned object of the invention, the present invention adopts the following technical solution:

[0006] A first aspect of the present invention provides a method for optimizing resource utilization efficiency, comprising:

[0007] Query a resource table, wherein the resource table contains a list of available resources corresponding to each CAE software, and indicates priority information of using each resource list in a sequential order;

[0008] Determine the resource list based on the job requirements, and check the idle status of each resource queue in the order of the resource list to obtain the computing queue information:

[0009] When the resources required by the job are met, the computing queue information is the resource queue information;

[0010] If none of them are satisfied, the computing queue information is the resource queue information with the least number of jobs waiting in queue;

[0011] The calculation queue information is transmitted to the scheduling system, and the calculation is performed according to the calculation queue information;

[0012] The calculation is completed and the resources are released.

[0013] Preferably, the priority information of each resource list is preset according to the resource characteristics in the resource queue.

[0014] Preferably, when the scheduling system detects a calculation anomaly in the job calculation process, the anomaly information is recorded.

[0015] According to a second aspect of the present invention, a computer system is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is used to implement the method for optimizing resource utilization efficiency as described above when executing the computer program.

[0016] According to a third aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for optimizing resource utilization efficiency as described above is executed.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1. The present invention can view the idle status of each resource in real time, and the job selects the optimal resource calculation according to the idle status. Without affecting the scheduling system, the computing power of the high-performance computing platform and the efficiency of using the CAE License are improved.

[0019] 2. The present invention does not need to destroy the existing architecture and software of the high-performance computing platform, and only needs to add a program for the usage of the resources, which can be used immediately after adding. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0021] Figure 1 Schematic diagram of the process of the method described in Examples 1-3 of the present invention;

[0022] Figure 2 This is a resource checking flow chart in Example 1 of the present invention;

[0023] Figure 3 It is a conditional structure diagram of the method described in Examples 1-3 of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0025] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0026] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0027] First Embodiment

[0028] The technical solution of the present invention is described in detail below through examples. The following examples are merely exemplary and can only be used to explain and illustrate the technical solution of the present invention, but are not to be construed as limiting the technical solution of the present invention.

[0029] This embodiment provides a method for optimizing resource utilization efficiency, which mainly realizes the effective utilization of various computing resources under the high-performance computing platform. Without changing the strategy, configuration, and architecture of the high-performance computing platform, combined with the characteristics of various CAE (computer-aided engineering) software, it configures which resource queues can be used by each CAE software, and expresses the priority of use (resource table) in the order of resources. When a job is submitted for calculation, the program first accesses the resource table, and checks the resource idleness in the order of the resource list corresponding to the CAE software. When the resources that meet the needs of this job can be met, the resource queue is allocated to the job; when all resource lists cannot meet the needs of the job, the program selects the resource queue with the least number of job waiting and allocates it to the job. After the allocation is completed, the allocation information is transmitted to the scheduling system. The scheduling system allocates a clear computing node or a queue waiting in a certain resource queue according to the job information. After the calculation is completed, all corresponding resources are released and the status of the used resources is updated.

[0030] Specifically, Figure 1 , Figure 3 As shown, the method provided in this embodiment includes the following steps:

[0031] In step S1, a resource table is queried, which contains a list of available resources corresponding to each CAE software, and indicates the priority information of using each resource list in a sequential order. If it is a newly added CAE software, the newly added CAE software and the corresponding resource list can be written into the resource table.

[0032] Preferably, the priority information of each resource list is pre-set and determined by the resource characteristics in the resource queue. Different computing resources or computing nodes have different hardware configurations and performance characteristics, such as the number of CPU cores, memory size, storage performance, network bandwidth, etc. Determining the priority information according to the resource characteristics and arranging the order of each resource in accordance with the priority information can improve computing efficiency and resource utilization.

[0033] In step S2, the resource availability of each resource list is checked in order. The computing job finds the available resource list in the resource table according to the name of the CAE software used, and checks the availability of each resource queue in the order of the resource list. When the resources required by the job are met, the resource queue is allocated to the job. After checking the availability of all resource queues in the order of the resource list, if none of them are met, the resource queue with the least number of jobs waiting in the queue is allocated to the job.

[0034] Specifically, Figure 2 As shown, assuming that the CAE software used for the calculation job is Abaqus, its queue contains five available computing resource queues, namely: short, thr, four, fives, six. The scheduling system checks the idle resource status in sequence and allocates resources according to the queue order.

[0035] In step S3, the computing queue information is transmitted to the scheduling system. After the computing resource queue is determined, the queue information is written to the scheduling system. When the job needs to wait in line, the job enters the waiting state; when the job is calculated, the scheduling system allocates the corresponding idle computing nodes according to the resource queue information to calculate the job.

[0036] In step S4, the calculation is completed and the resources are released. When the job calculation is completed, the scheduling system will release the resources and modify the status of the job and resources.

[0037] Preferably, if the scheduling system detects a computing anomaly in the job computing process, it will record the abnormal information. The computing anomaly includes hardware failure (such as CPU failure, memory error, disk failure), software error (such as program crash, memory leak, deadlock), network problem (such as network delay, packet loss, network disconnection), etc. When an abnormality occurs in the computing job, the scheduling system can record the abnormal information, including the type of abnormality, the time of occurrence, the computing nodes involved, the job ID, the error log, etc. The abnormal information can be stored in the log file or in a centralized abnormal management system for subsequent analysis and processing. When a serious abnormality occurs, the system can automatically send notifications and alarm information to the system administrator so that timely measures can be taken. The notification method can include email, text message, instant message, etc.

[0038] The method provided in this embodiment adds real-time viewing of the idle status of each resource, and the job selects the optimal resource calculation according to the idle status, which improves the computing power of the high-performance computing platform and the efficiency of CAELicense (license) without affecting the scheduling system. At the same time, the present invention does not need to destroy the architecture and software of the existing high-performance computing platform, but only needs to add the program of the resource usage status of this part, which can be added and used immediately. In addition, because the high-performance computing platform is a Linux environment, this part of the program is written in the shell of the Linux system, which is compatible with all Linux versions and does not require compilation.

[0039] Second Embodiment

[0040] This embodiment provides a computer system, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement Figure 1 , Figure 3 The steps of the method for optimizing resource utilization efficiency are shown in .

[0041] Embodiment 3

[0042] This embodiment provides a computer-readable storage medium on which a computer program is stored. The computer program is executed when a processor is running. Figure 1 , Figure 3 The steps of a method for optimizing resource utilization efficiency are shown in . Wherein, the storage medium can be a volatile or non-volatile computer-readable storage medium.

[0043] The above describes the specific embodiments of the present invention. Based on the above description, relevant personnel can make various changes and modifications without departing from the technical concept of the present invention.

Claims

1. A method for optimizing resource utilization efficiency, characterized in that: include: Query a resource table, wherein the resource table contains a list of available resources corresponding to each CAE software, and indicates priority information of using each resource list in a sequential order; Determine the resource list based on the job requirements, and check the idle status of each resource queue in the order of the resource list to obtain the computing queue information: When the resources required by the job are met, the computing queue information is the resource queue information; If none of them are satisfied, the computing queue information is the resource queue information with the least number of jobs waiting in queue; The calculation queue information is transmitted to the scheduling system, and the calculation is performed according to the calculation queue information; The calculation is completed and the resources are released.

2. A method for optimizing resource utilization efficiency according to claim 1, characterized in that: The priority information of each resource list is preset according to the resource characteristics in the resource queue.

3. The method for optimizing resource utilization efficiency according to claim 1, characterized in that: When the scheduling system detects a calculation anomaly in the job calculation process, it records the anomaly information.

4. A computer system, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is used to implement the method for optimizing resource utilization efficiency as claimed in any one of claims 1 to 3 when executing the computer program.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for optimizing resource utilization efficiency as claimed in any one of claims 1 to 3 is executed.

Images