Automatic host configuration system and method supporting high-concurrency task scheduling

Abstract

The invention relates to an automatic host configuration system and method supporting high-concurrency task scheduling. The system comprises an automatic operating system installation module for receiving host information input by a user, performing automatic verification and performing parallel operating system installation of a plurality of hosts based on the host information; the automatic cluster deployment module is used for automatically completing installation and configuration of distributed software of a plurality of hosts; the automatic interface test module is used for executing a historical function regression test; the intelligent report module is used for generating an incremental interactive report based on the execution result of each module, and performing failure root cause automatic positioning; the multi-task parallel processing module is used for receiving the same or different tasks concurrently submitted by each user, and calling the module to execute a plurality of tasks in parallel based on the task content; the assembly line control module, the series module and the parallel step depend on checking. Compared with the prior art, standardization, automation and parallelization of the host deployment process are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer distributed storage system testing, in particular to a host automatic configuration system and method supporting high-concurrency task scheduling. BACKGROUND

[0002] In the testing process of distributed storage products, each round of system testing usually includes new function testing, full new installation and deployment testing, last version rolling upgrade testing, and old function interface testing, etc. repetitive work. Among them, the full new installation and deployment testing needs to complete the minimum installation of the operating system and the deployment of the distributed product software, and the historical function interface testing needs to verify the stability of a large number of existing functions. These works are extremely low in efficiency in manual operation in the multi-node, multi-operating system scenario.

[0003] With the development of distributed storage systems towards multi-node, multi-hardware platform, and multi-operating system (OS) distribution, the complexity of the testing scenario is greatly improved. In the traditional testing mode, manual operation accounts for a high proportion, and the existing technology has the following pain points: first, manual installation of a single-node operating system takes about 1 hour, and five-node cluster at least needs 5 hours, and in the multi-OS scenario, the time is multiplied, and although the automatic tool can shorten the time, it lacks the adaptation ability of multi-hardware platform; second, when multiple users concurrently submit test tasks, the resource scheduling is chaotic, and high-urgency tasks cannot be executed in priority, resulting in delay in processing core problems; third, the test report is mostly static data stacking, and the user cannot quickly locate the failure root cause, and needs to manually create a Bug work order, the information transmission is incomplete and inefficient; fourth, the test process is fixed, and cannot be flexibly combined with modules according to actual needs, and the resources are not recycled in time after being occupied in the testing process, causing resource waste.

[0004] To solve the above efficiency problems, the industry gradually introduces automatic tools, such as Jenkins to build a basic test framework, such as Chinese patent application CN119806779A, which discloses a distributed scheduling method based on task, node, and link resources, which realizes the automatic execution of part of the process, such as simplifying the OS installation or software deployment steps through scripts, shortening the single-node OS installation time to about 20 minutes, and reducing the distributed software deployment time to 1.25 hours. However, the existing automation scheme based on Jenkins can only meet the basic automation needs, and has not been deeply optimized for the complex scenarios of distributed storage testing, and still has many technical pain points, and cannot face multiple-user concurrency, multiple-task parallelism, and multiple-error situations, and is difficult to adapt to large-scale, high-complexity distributed storage product testing requirements. SUMMARY

[0005] The application aims at providing a host automatic configuration system and method supporting high-concurrency task scheduling to overcome the defects of the prior art, integrating and automating the test process with high repeatability and complicated operation, and significantly improving the test efficiency.

[0006] The application aims at providing a host automatic configuration system and method supporting high-concurrency task scheduling to overcome the defects of the prior art, integrating and automating the test process with high repeatability and complicated operation, and significantly improving the test efficiency. A host automatic configuration system supporting high-concurrency task scheduling, comprising: An automatic operating system installation module: receiving host information input by a user, automatically checking and installing operating systems of a plurality of hosts in parallel based on the host information; An automatic cluster deployment module: automatically completing installation and configuration of distributed software of a plurality of hosts based on the host information input by the user and preset parameters; An automatic interface test module: performing historical function regression test based on the installed operating systems and software, testing functions of each software and interface, and generating a distributed test incremental report; An intelligent report module: generating an incremental interactive report based on execution results of the automatic operating system installation module, the automatic cluster deployment module and the automatic interface test module, and automatically locating failure root causes; A multi-task parallel processing module: receiving same or different tasks submitted by a plurality of users in parallel, calling the automatic operating system installation module, the automatic cluster deployment module or the automatic interface test module to execute a plurality of tasks in parallel based on the task content, wherein each task is independent of each other, and the execution order and resource allocation of each task are determined based on the priority of each task during the calling of each module; A pipeline control module: connecting the automatic operating system installation module, the automatic cluster deployment module, the automatic interface test module, the multi-task parallel processing module and the intelligent report module in series, and performing step dependency check on each module based on the task content and the task completion.

[0007] Further, the host information input by the user includes a MAC address, an IPMI IP, an IPMI username / password, a management network port IP and an operating system type.

[0008] Further, the preset parameters in the automatic cluster deployment module include a cluster Web IP, a software download address, a node management port IP, a storage / computing role IP, a transmission mode, a metadata IP and a VIP setting.

[0009] Further, the tasks include combined tasks and single tasks, the combined tasks are combinations of multiple subtasks, the subtasks include operating system installation tasks, cluster deployment tasks and historical function regression test tasks, the single tasks only contain single subtasks, the historical function regression test tasks contain tests of various software or interfaces of the configured host.

[0010] Further, the process of determining the task execution order and resource allocation in the process of calling the modules of the multi-task parallel processing module based on the priority of each task includes: determining the priority of each subtask in each task based on a preset priority rule; calculating the priority of each task based on the priority of the subtasks, the task submission timestamp, the task urgency and the resource demand matching degree; performing real-time step dependency checking on the task completion of each module by the pipeline control module, and based on the passing of the step dependency checking, executing each task in order in descending order of task priority according to the change of system resources; wherein, after any task execution is completed or fails, the system resources are detected and the task priority is calculated, and the determination of the task execution order and resource allocation is performed again.

[0011] Further, the expression for calculating the priority of each task is: wherein, is the i-th task, is the priority quantization value of the subtask, is the priority weight of the subtask, is the time impact parameter, is the time weight, is the urgency impact parameter, is the urgency weight, is the resource demand matching degree, is the resource demand matching degree weight; wherein, is the weight of the highest priority subtask in the combined task, is the average weight of the subtasks in the combined task, is the score of the highest priority subtask in the combined task, is the priority score of the i-th subtask, ​This represents the number of subtasks in the combined task. For task submission time, For the current time, For the task time window, The preset time limit for completing the task. For emergency time window, For CPU matching rate, For memory matching rate, For network bandwidth matching rate, For the current task requirements, This represents the remaining resources of the current node.

[0012] Furthermore, the process of performing the aforementioned step dependency check on each module based on the task content and task completion status includes: When receiving the initial moment of the same or different tasks submitted concurrently by users, extract the system modules required to process subtasks from each task, and build the module dependency chain of each task based on the preset module-subtask dependency relationship. When calling different modules to execute multiple tasks in parallel based on task content, the execution status, execution result, and completion timestamp of each module's subtask are obtained based on the dependency chain. If the preceding dependent module of the currently executing subtask has completed its corresponding subtask or the currently executing subtask has no preceding dependent module, the dependency is determined to be satisfied. If the preceding dependent module of the currently executing subtask has not completed its corresponding subtask, the dependency is determined to be unsatisfied, a dependency failure report is generated, and the preceding dependent modules that need to be executed for the corresponding subtask are listed.

[0013] Furthermore, when the automated interface testing module executes historical functional regression testing tasks, it immediately generates a report fragment based on the task execution results after completing any software or interface test, and then incrementally updates and outputs the main report in real time through the report aggregation service.

[0014] Furthermore, the root causes of failure include operating system installation failure, distributed software installation and configuration failure, and historical functional regression test failure. The automatic root cause identification process includes: Collect the task execution results of each system module, extract the failure logs and preprocess them. The failure logs include error log keywords of failed tasks, system resource indicators, code change records and historical execution data. Based on a pre-defined root cause label library, failure causes are classified according to pre-processed failure logs, and feature vectors of failed tasks are constructed based on the classification results and failure logs. A pre-trained decision tree model is used to obtain the root cause localization results and confidence scores based on the feature vectors of the failed task and according to the node splitting rules.

[0015] An automated host configuration method based on the host automated configuration system supporting high-concurrency task scheduling as described above, the method comprising: The multi-task parallel processing module receives host information and task content input by the user, and determines the task execution order and resource allocation based on the priority of each task. Based on the task execution order and resource allocation, the automated operating system installation module, automated cluster deployment module, or automated interface testing module are called to execute the task in parallel according to the task content. When executing the task, the step dependency check of each module is performed based on the task content and task completion status. Based on the execution results of the automated operating system installation module, automated cluster deployment module, and automated interface testing module, an incremental interactive report is generated through the intelligent reporting module, and the root cause of failure is automatically located.

[0016] Compared with the prior art, the beneficial effects of the present invention include: 1. This invention discloses a highly integrated system capable of automating multi-user, multi-task, and multi-error handling. It includes operating system installation, cluster deployment, interface testing, and report feedback functions, enabling full-process automation to replace manual operations, shortening operating system installation time, balancing load, and allowing for flexible process combinations to reduce ineffective execution. This invention supports batch input of multiple host information and parameterized configuration for cluster deployment, meeting the full-scenario testing needs of distributed storage products from single machines to large-scale clusters. The multi-task parallel processing module of this invention can determine the task execution order and resource allocation based on the priority of each task, avoiding node overload or idleness, reducing resource waste while improving task execution efficiency. In this invention, each functional module is independently designed, and modules are connected in series through a pipeline control module, supporting individual upgrades or expansions. When adding test modules or adapting to new hardware platforms, there is no need to reconstruct the overall framework, reducing maintenance costs.

[0017] 2. This invention uses decision tree model fusion analysis to automatically extract data such as logs, resources, and code changes, accurately locate the root cause of task errors, reduce the workload of manual investigation, and improve the efficiency of cluster deployment.

[0018] 3. This invention calculates the priority of each task based on multi-dimensional data such as subtask priority, task submission timestamp, task urgency, and resource requirement matching degree, ensuring that high-urgency and high-priority tasks are executed first. At the same time, it dynamically allocates resources such as CPU, memory, and network bandwidth according to resource requirement matching degree, avoiding resource waste or task blocking and improving system resource utilization.

[0019] 4. This invention constructs a module dependency chain through a pipeline control module, and checks in real time whether the prerequisites of subtasks are satisfied, avoiding task execution failures due to missing dependencies and ensuring the accuracy and stability of task execution.

[0020] 5. The automated interface testing module of this invention generates a report fragment of each test task immediately after completion. It continuously receives these fragments using a report aggregation service and incrementally updates the main report. This overcomes the problem of long waiting times for all nodes to complete before generating reports in large-scale distributed testing. Users can see the real-time results of the completed parts during test execution, improving user experience and feedback speed. Attached Figure Description

[0021] Figure 1 This is a system structure diagram of the present invention; Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] Example 1 This embodiment discloses a host automated configuration system that supports high-concurrency task scheduling. The system is as follows: Figure 1 As shown, it includes an automated operating system installation module, an automated cluster deployment module, an automated interface testing module, a multi-task parallel processing module, an intelligent reporting module, and a pipeline control module. The specific functions of each module are described below: Automated operating system installation module: Receives host information input by the user, automatically verifies it, and performs parallel operating system installation on several hosts based on the host information; Automated cluster deployment module: Automatically completes the installation and configuration of distributed software on several hosts based on the host information and preset parameters input by the user; Automated Interface Testing Module: Based on the deployed operating system and software, it performs historical functional regression tests, tests the functions of each software and interface, and generates distributed incremental test reports; Intelligent Reporting Module: Based on the execution results of the automated operating system installation module, automated cluster deployment module, and automated interface testing module, it generates incremental interactive reports and automatically locates the root causes of failures. Multi-task parallel processing module: Receives the same or different tasks submitted concurrently by users, and calls the automated operating system installation module, automated cluster deployment module, or automated interface testing module to execute multiple tasks in parallel based on the content of the tasks. Among them, each task is independent of the others, and the execution order and resource allocation of tasks are determined based on the priority of each task during the calling of each module. The pipeline control module connects the automated operating system installation module, the automated cluster deployment module, the automated interface testing module, the multi-task parallel processing module, and the intelligent reporting module, and performs step dependency checks on each module based on the task content and task completion status.

[0024] In the automated operating system installation module, the host information entered by the user includes the MAC address, IPMI IP, IPMI username / password, management network port IP, and operating system type.

[0025] The preset parameters in the automated cluster deployment module include cluster Web IP, software download address, node management port IP, storage / compute role IP, transmission mode, metadata IP, and VIP settings.

[0026] Tasks include combined tasks and individual tasks. Combined tasks are combinations of multiple sub-tasks, including operating system installation tasks, cluster deployment tasks, and historical function regression testing tasks. Individual tasks contain only a single sub-task, namely, a single operating system installation task, cluster deployment task, or historical function regression testing task.

[0027] Historical function regression testing tasks include various tests on the configured host, such as interface testing and environment cleanup.

[0028] The process by which the multi-task parallel processing module determines the execution order and resource allocation based on the priority of each task during the process of calling other modules includes: Based on preset priority rules, the priority of subtasks in each task is determined; The priority of each task is calculated based on the priority of the subtask, the task submission timestamp, the task urgency, and the resource requirement matching degree. The pipeline control module performs real-time step dependency checks on the task completion status of each module. Based on the success of the step dependency checks, and according to changes in system resources, the tasks are executed sequentially in descending order of task priority. After any task is completed or fails, system resources are checked and task priority is calculated. Then, the task execution order and resource allocation are determined again.

[0029] The preset priority rule is specifically as follows in this embodiment: P0 (highest priority): A combined task that includes both operating system installation and cluster deployment tasks; P1 (High Priority): Other combined tasks; P2 (Medium Priority): Includes single historical functional regression test tasks and multi-node cluster deployment tasks, encompassing various tests. P3 (low priority): contains only historical functional regression test tasks and single-node deployment tasks for a single type of test.

[0030] The default subtask priority is configured by the administrator in the Jenkins web backend. It supports adding / modifying levels and adjusting weights. When users submit tasks, they need to select the corresponding priority from the drop-down list. It cannot be customized.

[0031] The expression for calculating the priority of each task is: in, For the first One task, Quantify the priority of subtasks. For subtask priority weights, For parameters affected by time, As time weight, The parameter is affected by the urgency level. Assigning weights based on urgency For resource demand matching degree, Weighting for the degree of matching between resource demand and demand; in, The weight of the highest priority subtask in the combined task. The average weight of the subtasks in the combined task. This is the score for the highest priority subtask in the combined task. The priority score for the i-th subtask. This represents the number of subtasks in the combined task. For task submission time, For the current time, For the task time window, The preset time limit for completing the task. For emergency time window, For CPU matching rate, For memory matching rate, For network bandwidth matching rate, For the current task requirements, This represents the remaining resources of the current node.

[0032] During testing, if the step dependency check passes, the tasks are executed sequentially in descending order of task priority, based on changes in system resources; otherwise, the test is stopped and a dependency failure report is output.

[0033] Based on changes in system resources, tasks are executed sequentially in descending order of priority. In this embodiment, specifically: When CPU idle time is less than 10% and memory usage is less than 10%, only P0 and P1 priority tasks are allowed to be scheduled, while P2 and P3 tasks are paused. Within the same priority queue, priority is calculated and sorted according to dependency satisfaction status, submission timestamp, urgency, and resource matching degree, supporting high-priority tasks to preempt resources from low-priority non-critical step tasks and subsequent recovery execution. Conversely, when CPU idle time is greater than or equal to 10% or memory usage is greater than or equal to 10%, tasks are sorted and executed based on the task priority calculation results.

[0034] The process of performing step dependency checks on each module based on task content and task completion status includes: When receiving the initial moment of the same or different tasks submitted concurrently by users, extract the system modules required to process subtasks from each task, and build the module dependency chain of each task based on the preset module-subtask dependency relationship. When calling different modules to execute multiple tasks in parallel based on task content, the execution status, execution result, and completion timestamp of each module's subtask are obtained based on the dependency chain. If the preceding dependent module of the currently executing subtask has completed its corresponding subtask or the currently executing subtask has no preceding dependent module, the dependency is determined to be satisfied. If the preceding dependent module of the currently executing subtask has not completed its corresponding subtask, the dependency is determined to be unsatisfied, a dependency failure report is generated, and the preceding dependent modules that need to be executed for the corresponding subtask are listed.

[0035] In this embodiment, the module-subtask dependency relationship is detailed in Table 1 below: Table 1 Module-Subtask Dependency Table The following is an example of the process of performing step dependency checks on each module based on task content and task completion status: Combined tasks: Operating system installation (subtask A) → Cluster deployment (subtask B) → Historical function regression testing (subtask C) Step-by-step dependency check process: Before subtask A is executed: there are no prerequisites, so it is executed directly; After subtask A is executed: the status of subtask A is synchronized to "completed + successful", triggering the dependency check of subtask B, which checks the status and output parameters of subtask A. If the check passes, B is executed. After subtask B is executed: the status of subtask B is synchronized to "completed + successful", triggering the dependency check of subtask C, which checks the status of subtask B and the availability of cluster services. If the check passes, C is executed. If subtask B fails to execute: terminate the execution of C, mark the combined task as "failed", and record the dependent failure nodes.

[0036] When the automated interface testing module executes historical functional regression testing tasks, it immediately generates a report fragment based on the task execution results after completing any software or interface test, and then incrementally updates and outputs the main report in real time through the report aggregation service.

[0037] Root causes of failure include operating system installation failures, distributed software installation and configuration failures, and historical functional regression test failures. The process of automatically locating the root cause of failure includes: Collect the task execution results of each system module, extract failure logs and preprocess them. The failure logs include error log keywords of failed tasks, system resource indicators, code change records and historical execution data. Based on a pre-defined root cause label library, failure causes are classified according to pre-processed failure logs, and feature vectors of failed tasks are constructed based on the classification results and failure logs. A pre-trained decision tree model is used to obtain the root cause localization results and confidence scores of failures based on the feature vectors of failed tasks and according to the node splitting rules.

[0038] In this embodiment, the system is implemented based on the Jenkins platform and programming language, specifically as follows: A graphical interface is provided via a Jenkins plugin, allowing users to input essential information such as the MAC address, IPMI address, IPMI username and password, management network port IP, and OS type of the host to be installed, as well as optional parameters such as system disk information, hostname, and DNS. Once the information is verified, the automated installation process is triggered. This module supports batch installation on single or multiple hosts. Information for multiple hosts is entered sequentially, with fields separated by commas, enabling parallel deployment.

[0039] After all host operating systems are installed, the cluster software deployment phase begins. Users input cluster-related parameters through the Jenkins interface, including the cluster WebIP, software download address, management port IPs for each node, storage role IPs, compute role IPs, transfer mode, metadata IPs, and compute virtual IPs. The system automatically downloads, transfers, installs, and configures the software based on the input information. During deployment, the system automatically matches node management IPs with storage IPs to ensure correct role configuration.

[0040] After successful cluster deployment, the interface testing phase begins. Users input parameters such as the cluster WebIP, IP addresses of various node types, root password, test modules, and test images. The system automatically executes historical functional regression tests and outputs graphical test results.

[0041] To enhance the flexibility and controllability of the testing process, the three modules mentioned above are integrated into a single pipeline within Jenkins. These modules execute sequentially, with the next step only proceeding after each step is successfully completed. Users can also execute one or more modules individually as needed.

[0042] Through the above design, the present invention achieves standardization, automation and parallelization of the testing process, significantly improving testing efficiency and coverage, and is suitable for testing needs of large-scale, multi-scenario distributed storage systems.

[0043] In another embodiment, the system also automatically verifies the format and validity of the user input information. If the verification fails (N), it prompts "Input error" and ends the current task, waiting for the user to re-enter the information.

[0044] Example 2 This embodiment, based on the host automated configuration system supporting high-concurrency task scheduling disclosed in Embodiment 1 above, discloses a host automated configuration method, the specific method as follows: Figure 2 As shown, it includes: Step S1: Receive host information and task content input by the user through the multi-task parallel processing module; Step S2: Determine the task execution order and resource allocation based on the priority of each task; Step S3: Based on the task execution order and resource allocation, the automated operating system installation module, automated cluster deployment module, or automated interface testing module are called to execute tasks in parallel according to the task content; when executing each task, the step dependency of each module is checked based on the task content and task completion status. Step S4: Based on the execution results of the automated operating system installation module, automated cluster deployment module, and automated interface testing module, an incremental interactive report is generated through the intelligent reporting module, and the root cause of failure is automatically located.

[0045] In step S3, if the step dependency check determines that the dependency is not met when each task is executed, the process jumps to step S4 to generate an incremental interactive report containing a report on the failure to meet the dependency requirement.

[0046] Example 3 Based on Embodiments 1 and 2, this embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, the one or more programs including instructions for executing the aforementioned host automated configuration method.

[0047] At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the aforementioned host automated configuration method. Of course, in addition to software implementation, this invention does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0048] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0049] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0050] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A host automation configuration system supporting high concurrency task scheduling, characterized in that, The system comprises: an automatic operating system installation module: receiving host information input by a user, automatically checking and installing operating systems on a plurality of hosts based on the host information; an automatic cluster deployment module: automatically installing and configuring distributed software on a plurality of hosts according to the host information input by the user and preset parameters; an automatic interface test module: performing historical function regression tests based on the deployed operating systems and software, testing the functions of each software and interface, and generating a distributed test incremental report; an intelligent report module: generating an incremental interactive report based on the execution results of the automatic operating system installation module, the automatic cluster deployment module and the automatic interface test module, and automatically locating the root cause of failure; a multi-task parallel processing module: receiving the same or different tasks submitted concurrently by users, calling the automatic operating system installation module, the automatic cluster deployment module or the automatic interface test module to execute multiple tasks in parallel based on the content of the tasks, wherein each task is independent of the others, and the execution order of the tasks and the allocation of resources are determined based on the priority of each task during the calling of each module; a pipeline control module: connecting the automatic operating system installation module, the automatic cluster deployment module, the automatic interface test module, the multi-task parallel processing module and the intelligent report module in series, and checking the step dependency of each module based on the content of the tasks and the completion of the tasks. 2.The host automation configuration system supporting high concurrency task scheduling of claim 1, wherein, The host information input by the user includes a MAC address, an IPMI IP, an IPMI username / password, a management network port IP and an operating system type. 3.The host automation configuration system supporting high concurrency task scheduling of claim 1, wherein, The preset parameters in the automatic cluster deployment module include a cluster Web IP, a software download address, a node management port IP, a storage / computing role IP, a transmission mode, a metadata IP and a VIP setting. 4.The host automation configuration system supporting high concurrency task scheduling of claim 1, wherein, The tasks include combined tasks and single tasks, the combined tasks are a combination of multiple subtasks, the subtasks include operating system installation tasks, cluster deployment tasks and historical function regression test tasks, the single tasks only include a single subtask, and the historical function regression test task includes tests of a plurality of software or interfaces on a configured host.

5. The host automation configuration system supporting high concurrency task scheduling according to claim 4, wherein, The process of determining the execution order of the tasks and the allocation of resources during the calling of each module by the multi-task parallel processing module based on the priority of each task includes: determining the priority of each subtask in each task based on a preset priority rule; calculating the priority of each task based on the priority of the subtask, a task submission timestamp, a task urgency and a resource demand matching degree; performing real-time step dependency checking on the completion of the tasks of each module based on the pipeline control module, executing each task in order in descending order of task priority based on the step dependency checking pass and changes in system resources; wherein after the completion or failure of any task, the system resources are detected and the priority of the task is calculated, and the determination of the execution order of the tasks and the allocation of resources is performed again. 6.The host automation configuration system supporting high concurrency task scheduling of claim 5, wherein, The expression for calculating the priority of each task is: in, For the first One task, Quantify the priority of subtasks. For subtask priority weights, For parameters affected by time, As time weight, The parameter is affected by the urgency level. Assigning weights based on urgency, For resource demand matching degree, Weighting for the degree of matching between resource demand and demand; wherein, is a weight of a highest priority subtask in a combined task, is an average weight of subtasks in a combined task, is a score of a highest priority subtask in a combined task, is a priority score of an i-th subtask, is a number of subtasks in a combined task, is a task submission time, is a current time, is a task time window, is a preset task must complete time, is an emergency time window, is a CPU matching rate, is a memory matching rate, is a network bandwidth matching rate, is a task demand of a current single task, is a current node remaining resource.

7. The host automation configuration system supporting high concurrency task scheduling according to claim 4, wherein, The process of performing the step dependency checking of each module based on the content of the tasks and the completion of the tasks includes: When receiving initial moments of concurrent submission of same or different tasks by each user, system modules required for processing sub-tasks are extracted from each task, and module dependency chains of each task are constructed based on preset module-sub-task dependency relationships; When calling different modules to execute multiple tasks in parallel based on task content, the execution status, execution result and completion timestamp of each module are obtained based on the dependency chain, and if the pre-dependent module of the currently executed sub-task has completed the corresponding sub-task or the currently executed sub-task has no pre-dependent module, it is determined that the dependency has been met; if the pre-dependent module of the currently executed sub-task has not completed the corresponding sub-task, it is determined that the dependency has not been met, and a dependency unfulfilled report is generated, listing the pre-dependent module that needs to execute the corresponding sub-task. 8.The host automation configuration system supporting high concurrency task scheduling of claim 4, wherein, When the automated interface test module performs historical function regression test tasks, a report segment is generated based on the task execution result immediately after any software or interface test is completed, and the main report is updated in real time by the report aggregation service and output. 9.The host automation configuration system supporting high concurrency task scheduling of claim 1, wherein, The failure root cause includes an operating system installation failure root cause, a distributed software installation and configuration failure root cause, and a historical function regression test failure root cause; The process of automatic positioning of the failure root cause includes: Collecting task execution results of each system module, extracting failure logs and preprocessing, the failure logs including error log keywords, system resource indicators, code change records and historical execution data of failed tasks; Based on a preset root cause label library, the failure reason is classified according to the preprocessed failure logs, and a feature vector of the failed task is constructed based on the classification result and the failure logs; A pre-trained decision tree model is used to obtain the failure root cause positioning result and confidence based on the feature vector of the failed task according to the node splitting rule.

10. A host automation configuration method based on the host automation configuration system supporting high concurrency task scheduling according to any one of claims 1-9, characterized in that, The method includes: Receiving host information and task content input by the user through the multi-task parallel processing module, and determining the task execution order and resource allocation based on the priority of each task; Based on the task execution order and resource allocation, calling the automated operating system installation module, the automated cluster deployment module or the automated interface test module to execute the tasks according to the task content, and performing step-dependent checks on each module based on the task content and the task completion status when executing the tasks; Based on the execution results of the automated operating system installation module, the automated cluster deployment module and the automated interface test module, an incremental interactive report is generated by the intelligent report module, and the failure root cause is automatically positioned.

Images