Software-defined test resource containerization affinity scheduling deployment method

Abstract

The invention provides a software-defined test resource containerization affinity scheduling deployment method. The method mainly comprises three aspects of content: test resource affinity deployment strategy generation, test resource mirror image packaging and multi-user test resource parallel container deployment. The invention provides a software-defined test resource containerization affinity scheduling deployment technology for the first time. The method has the advantages that minute-level deployment generation of software-defined digital resources under limited physical resources can be realized, and the construction efficiency and flexibility of a self-adaptive system test environment are greatly improved; and secondly, parallel deployment of a multi-user and multi-sample self-adaptive system test environment can be realized, and evaluation and verification of the self-adaptive adaptive capability of the self-adaptive system are supported.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of civil cloud computing, and particularly relates to a software-defined test resource containerization affinity scheduling and deployment method. BACKGROUND

[0002] With the rapid development and application of emerging technologies such as 5G, Internet of Things, cloud computing, and machine learning, adaptive systems are facing requirements such as task diversification, control object scaling, strong confrontation in operating environment, and high real-time system response, and urgently need stronger adaptability and autonomy. A software-defined adaptive system needs to autonomously perceive changes in tasks, environments, and its own state during operation, and quickly recover and optimize system task guarantee capabilities by autonomously adjusting system structure, function, and information interaction relationship, so as to adapt to changes in the external environment. Therefore, it is urgent to build a software-defined adaptive system test verification environment to evaluate and optimize the adaptive capability of the system.

[0003] A software-defined adaptive system test environment involves a large number of environment, reconnaissance, decision-making, platform, and other software digital resources. These resources not only have inconsistencies in operating basic hardware platforms, operating systems, and operating dependencies, and are closely related to the deployment and installation of the hardware environment and the operating system, but also have software-defined test resource working states, working parameters, and working flows, information interaction relationships, and other software-defined test resources that can be controlled and managed by an external master console for unified control of test resources. How to shield the differences in the underlying basic hardware platform and operating system and other operating environments, achieve flexible and efficient deployment of large-scale software-defined test resources, and improve the efficiency and flexibility of the test environment construction are the core problems that need to be solved in building a software-defined adaptive system test environment. The technical difficulties are as follows:

[0004] First, for software-defined heterogeneous cross-platform test resources, how to balance test resource deployment efficiency and cloud computing resource utilization, and avoid test resource automatic deployment deadlock. There are software-defined reconnaissance, platform, and other digital resources in the test environment. The running platforms of such resources are different, the image size and the cloud computing resources required for deployment are different (CPU, memory, and hard disk resources), and the digital resource running load varies greatly (associated with the target size of the test scenario, the number of scenarios is large, and the memory space required for digital resource loading and running is large). How to balance the test resource deployment efficiency and the cloud computing resource application and allocation utilization, optimize the allocation of cloud computing resources, generate batch heterogeneous test resource deployment strategies, and achieve cloud computing resource optimization allocation and load balancing.

[0005] Second, how to establish a digital resource deployment strategy that is related to each other and frequently communicates to reduce the amount of communication data for cross-node deployment and running, and improve the communication efficiency of test resource deployment and running. SUMMARY

[0006] The present application aims to solve the technical problems of the prior art, and provides a software-defined test resource containerization affinity scheduling and deployment method, comprising the following steps:

[0007] Step 1, apply for scheduling cloud computing resources;

[0008] Step 2, generate a test resource affinity deployment strategy;

[0009] Step 3, test resource image containerization encapsulation;

[0010] Step 4, multi-user test resource parallel container deployment.

[0011] In step 1, the test sample size, test scenario size, digital simulation resource image size, and memory running load are comprehensively considered, and a grouping deployment mechanism is adopted to apply for cloud computing resources, and the test resources to be deployed are divided into two or more groups, and the resources are applied for as a whole unit.

[0012] Step 1 includes: first, obtaining the current intelligent computing platform cluster resource size and resource capacity;

[0013] Secondly, the software-defined digital test resources to be deployed are grouped, and the same type of test resources are divided into the same group, and the grouping deployment algorithm calculates the cluster resource capacity required by each group of test resources, including running load and disk size, judges whether the cloud computing resources required by the multi-user and multi-sample test resource deployment can be met, if the deployment task requirements can be met, the required cloud computing resources are allocated as a unit; otherwise, feedback the deployment resource application failure reason; the deployment resource application algorithm is represented as:

[0014]

[0015] Where St represents the limitation, ri represents the test resource to be deployed, such as digital test resource, f i is the physical resource requirement for test resource i deployment, NRC is the cluster resource capacity, and N is the total number of test resources;

[0016] Image represents the image size, and the digital test resource image size in different scenarios differs greatly, directly affecting the physical resource allocation of the cluster node;

[0017] RunEnv represents the test resource running platform requirement, including Fengton and Maichuang platform, for AI algorithm type test resource, it needs to run on Maichuang platform, and for digital test resource, it needs to run on Fengton platform;

[0018] Mem represents the running load of the test resource, which mainly depends on the number of test scenario targets, because for digital test resources, the larger the test scenario target data of the running load, the higher the CPU and memory occupancy.

[0019] Com represents the communication cost between test resources. For digital test resources that need to interact frequently, in order to reduce the communication volume between cluster nodes, the test resources that need to communicate frequently are distributed on the same physical resource node of the cluster;

[0020] com ij represents the communication cost between test resource i and test resource j.

[0021] Step 2 includes: based on the application scheduling cloud computing resources, establishing the mapping relationship between the digital test resources and the physical computing resources;

[0022] The mapping relationship between the test resources and the cloud computing resources refers to further allocating the test resources in the same group to the corresponding intelligent computing platform cluster resources to form the overall deployment strategy of the test resources, and generating the mapping relationship between the digital test resources and the physical resources;

[0023] Firstly, according to the requirements of the running platform of the deployed test resources, the corresponding cluster resources are selected. For AI algorithm resources, the Maicun platform cluster resources are allocated; for digital test resources, the Feiteng platform cluster resources are selected;

[0024] Secondly, the cluster resources required by the deployed test resources are sorted from small to large, and the test resources with small cluster resources are preferentially selected for deployment;

[0025] Finally, the resource utilization greedy algorithm is used. The resource utilization greedy algorithm tries to ensure that any occupied cluster node is as fully occupied as possible, which avoids deploying empty test resources to the occupied cluster nodes. The more full the occupied cluster node is, the more likely it is to be allocated and deployed. The resource utilization greedy algorithm prioritizes the queue with the lowest expected resource utilization rate by calculating the CPU, memory, and disk utilization rate of each cluster node, and preferentially allocates and deploys the cluster node with the lowest utilization rate. At this time, a single digital test resource container is regarded as a separate deployment unit.

[0026] In step 3, the test resource image container is encapsulated to generate a lightweight container image.

[0027] Step 3 includes: firstly, the digital test resources and their running dependency packages are encapsulated into images, and the corresponding code compiler is dynamically called for code compilation to generate an executable code package, which is loaded into the container for compilation, running and testing; the generated executable code package and the running dependent basic environment are integrally encapsulated to form a test resource image;

[0028] Secondly, a test resource container image environment variable is generated, including declaring the port listened by the service in the image, the default entry instruction of the container image, the username and ID when the container is run;

[0029] Finally, the container image is generated by calling the Build command and uploaded to the image library for sharing scheduling.

[0030] In step 4, the multi-user test resource parallel container deployment includes building a test user characterization model and test resource parallel container deployment.

[0031] Step 4 includes: first, building a test user characterization model, and the specific characterization elements include: user name, user token, test task identification, test sample identification, and digital simulation resource name; when the test user logs in to access the portal, the test user access token information is intercepted as the unique identification of the test user;

[0032] Secondly, the test resource parallel container deployment loads the test user permission and identity and the test sample class identification information into the container environment variable, loads the scheduling and deployment container image from the image library to the corresponding cluster node, deploys the test resource container, and starts the container to run.

[0033] The application also provides an electronic device including a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor executes the steps of the method.

[0034] The application also provides a storage medium storing computer programs or instructions, and when the computer programs or instructions are run on a computer, the steps of the method are executed.

[0035] The application optimizes the allocation of cloud computing resources required for software-defined test resource deployment by comprehensively considering the deployment requirements of test sample size, software-defined digital resource image size, and test scenario size, and generates a test resource deployment strategy.

[0036] The software-defined test resource containerization affinity scheduling and deployment technology proposed by the application has the following two advantages compared with the existing test resource deployment technology:

[0037] First, it can realize minute-level deployment generation of software-defined digital resources under limited physical resources, greatly improving the efficiency and flexibility of software-defined adaptive system test environment construction;

[0038] Second, it can realize the parallel deployment of multi-user and multi-sample software-defined adaptive system test environment, and support the evaluation test of multi-sample adaptive system self-adaptive capability. BRIEF DESCRIPTION OF DRAWINGS

[0039] The above and / or other aspects of the present application will become apparent and more readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which:

[0040] Figure 1 is a software-defined test resource image automatic distribution and deployment technology schematic diagram.

[0041] Figure 2 is a software-defined test resource affinity deployment strategy generation technology schematic diagram.

[0042] Figure 3 is a test user characterization model schematic diagram.

[0043] Figure 4 is a multi-user software-defined test resource parallel container deployment schematic diagram. DETAILED DESCRIPTION

[0044] The embodiment of the present application provides a software-defined test resource containerized affinity scheduling deployment method, and the core technical principle is described as follows: the deployment requirements of the test sample size, the digital resource image size and the test scenario target quantity are comprehensively considered, the cloud computing resources required for software-defined digital resource deployment are optimized and allocated, and a large-scale test resource deployment strategy is generated. By constructing a test user model, the purpose of multi-user online collaborative parallel deployment of test resources is realized, multi-user test parallel running is supported, and the problem of flexible and efficient generation of multi-user large-sample test resources is solved. The specific implementation route includes three steps: software-defined test resource affinity deployment strategy generation, test resource image packaging, and multi-user test resource parallel container deployment, as shown in Figure 1 .

[0045] 1. Software-defined test resource affinity deployment strategy generation

[0046] Considering the test sample size, test scenario size, digital resource image size and multi-dimensional constraint conditions of memory running load, the mapping relationship between software-defined digital resources and physical computing resources is generated, so that the test resources can be allocated to the required computing resources at the same time, and the resource deadlock problem existing in the existing k8S resource deployment is solved.

[0047] The proposed test resource affinity deployment strategy generation includes two parts: application scheduling of cloud computing resources and establishment of mapping relationship between test resources and cloud computing resources, as shown in Figure 2 .

[0048] (1) Application scheduling of cloud computing resources

[0049] Application scheduling cloud computing resources: adopt a grouping deployment mechanism to apply for cloud computing resources, divide the test resources to be deployed into multiple groups, and apply for resources in groups as a whole.

[0050] Firstly, the current intelligent computing platform cluster resource size and resource capacity are obtained.

[0051] Secondly, the software-defined digital test resources to be deployed are grouped, the same type of test resources are divided into the same group, the grouping deployment algorithm calculates the cluster resource capacity required by each group of test resources, including running load, disk size and Feng platform, Maicre platform, judges whether the cloud computing resources required by the deployment of multi-user and multi-sample test resources can be met, if the deployment task requirements can be met, the required cloud computing resources are allocated in groups as a unit; otherwise, feedback the reasons for the failure of the deployment resource application. This stage can ensure that the test resources meet the requirements of the cloud computing resource running platform. The above deployment resource application algorithm is described as shown in the following figure.

[0052]

[0053] Where St represents the limitation, ri represents the test resources to be deployed, such as digital test resources, f i The physical resource requirement required for the deployment of test resource i, NRC is the cluster resource capacity, and N is the total number of test resources.

[0054] Image represents the image size, and the image size of digital test resources in different scenarios differs greatly, directly affecting the physical resource allocation of the cluster node.

[0055] RunEnv represents the test resource running platform requirement, including Feng and Maicre platforms, for AI algorithm type test resources, it needs to run on the Maicre platform, and for digital test resources, it needs to run on the Feng platform.

[0056] Mem represents the test resource running load, which mainly depends on the number of test scenario targets, because for digital test resources, the larger the test scenario target data running load is, the higher the CPU and memory occupancy rate is.

[0057] Com represents the communication cost between test resources, for digital test resources that need to interact frequently, in order to reduce the communication amount between cluster nodes, the test resources that need to communicate frequently are distributed on the same cluster physical resource node.

[0058] com ij represents the communication cost between test resource i and test resource j.

[0059] Considering the above cluster resource allocation factors, the test resources after grouping are taken as a whole, and the cloud computing resources required by the application can avoid container deployment scheduling deadlock problems.

[0060] (2) Building a mapping relationship between test resources and cloud computing resources

[0061] The mapping relationship between test resources and cloud computing resources refers to further allocating test resources within the same group to corresponding intelligent computing platform cluster resources to form a test resource overall deployment strategy and generate a mapping relationship between software-defined digital resources and physical resources.

[0062] The present patent proposes a software-defined test resource affinity deployment strategy generation algorithm for generating a mapping relationship between digital test resources and cloud computing resources.

[0063] First, according to the requirements of the deployment test resource running platform, the corresponding cluster resources are selected. For AI algorithm resources, allocate Maicun platform cluster resources; for digital test resources, select Feiteng platform cluster resources.

[0064] Second, the cluster resources required for the deployed test resources are sorted from small to large, and test resources with small cluster resources are preferentially selected for deployment.

[0065] Finally, the resource utilization greedy algorithm is used, which attempts to ensure that any occupied cluster node is as fully occupied as possible. It avoids deploying empty test resources to occupied cluster nodes. The more full an occupied cluster node is, the more likely it is to be allocated and deployed. This algorithm prioritizes the queue with the lowest expected resource utilization rate by calculating the CPU, memory, and disk utilization of each cluster node, and preferentially allocates and deploys the cluster node with the lowest utilization rate. At this time, a single digital test resource container is considered as a separate deployment unit.

[0066] The following Table 1 shows the execution flow of the test resource affinity deployment strategy generation algorithm.

[0067] Table 1

[0068]

[0069]

[0070] 2. Test resource image containerization packaging

[0071] Test resources include digital resources such as environment, reconnaissance, decision-making, and platform. Due to the heterogeneity of test resources in terms of running platform and development environment, it is difficult to achieve unified deployment generation of test resources. Therefore, the present patent proposes a test resource containerization packaging technology for generating lightweight container images.

[0072] First, test resource packaging. Package the digitized resource and its running dependency into an image, dynamically call the corresponding code compiler for code compilation, generate an executable code package, and load it into the container for compilation and testing. The generated executable code package and the running dependency of the basic environment are integrated and packaged to form the corresponding test resource image.

[0073] Second, generate test resource container image environment variables. Load the container base image that the digitized simulation resource running dependency depends on, set the environment variables of the container base image running dependency, declare the port that the image service listens to, and specify the default entry instruction of the container image, the username and ID when running the container.

[0074] Finally, create a container image. Generate a container image by calling the Build command, and upload the generated container image to the image library. Once the container image is successfully uploaded, it can be queried and reused through the image service.

[0075] 3. Multi-user test resource parallel container deployment

[0076] Multi-user test resource parallel container deployment aims to enable different test users to carry out different software-defined system adaptive test tasks in parallel, deploy the required digitized resources on demand, isolate them, and carry out software-defined system adaptive tests in parallel. This patent proposes a multi-user online parallel deployment technology based on a test user model to achieve parallel deployment of digitized test resources.

[0077] First, build a test user representation model, which includes the following specific representation elements: user name, user identification, user access rights, test task identification, test sample identification, and digitized test resource image, as shown in Figure 3 When a test user logs in to access the portal, intercept the test user access token information, which includes the user name, user identification, and user access rights, as the unique identification of the intelligent test user. Based on the token, different test users are distinguished to avoid cross-transmission of instructions between multiple users. Test users can select test tasks, multiple test subjects, and multiple test samples for container deployment according to test requirements to generate test description information. After completing the current test, the test user's token information is deleted to achieve parallel test running and management of multiple test users.

[0078] Second, test resource parallel container deployment. Test resources refer to digitized test resources. Based on the test resource deployment strategy, load the test user permissions and identity, test sample identification, and other identification information into the container environment variables, load the scheduled and deployed container image from the image library to the corresponding cluster node, deploy the test resource container, and start the container running. The specific technical implementation route is shown in Figure 4 .

[0079] Finally, the test resource deployment state monitoring. The test resource deployment service feeds back the running state of each type of test resource deployment in real time, including deployment execution progress, deployment time, deployment node IP address, and whether the deployment is faulty. After the container deployment is started and runs, the test user, test sample set, and other identification information are obtained from the environment variable, the test resource deployment service downloads the test scenario sample, starts the digital resource running, and realizes the flexible and efficient generation of multi-user test resources.

[0080] The application provides a software-defined test resource containerization affinity scheduling and deployment method. There are many methods and approaches to realize the technical solution, and the above description is only the preferred embodiment of the application. It should be pointed out that for ordinary technical personnel in the technical field, some improvements and refinements can be made without departing from the principle of the application, and these improvements and refinements should be regarded as the protection scope of the application. The components not explicitly described in the embodiment can be realized by using the existing technology.

Claims

1. A software-defined test resource containerization affinity scheduling deployment method, characterized in that, Comprising the following steps: Step 1, application scheduling cloud computing resources; Step 2, generating test resource affinity deployment strategy; Step 3, test resource image containerization packaging; Step 4, multi-user test resource parallel container deployment.

2. The method of claim 1, wherein, In step 1, considering the test sample size, test scenario size, digital simulation resource image size, memory running load, the grouping deployment mechanism is adopted to apply for cloud computing resources, and the test resources to be deployed are divided into two or more groups, and the resources are applied for as a whole unit.

3. The method of claim 2, wherein, Step 1 includes: first, obtaining the current intelligent computing platform cluster resource size and resource capacity; Secondly, the software-defined digital test resources to be deployed are grouped, and the same type of test resources are divided into the same group. The grouping deployment algorithm calculates the cluster resource capacity required by each group of test resources, including running load and disk size, and judges whether the cloud computing resources required by the multi-user and multi-sample test resource deployment can be met. If the deployment task requirements can be met, the required cloud computing resources are allocated in groups; otherwise, feedback the deployment resource application failure reason; the deployment resource application algorithm is represented as: where St denotes is limited to, ri represents the test resources needed to be deployed, such as digital test resources, f i the physical resource requirement needed for test resource i to be deployed, NRC is the cluster resource capacity, and N is the total number of test resources; Image represents the image size; RunEnv represents the test resource running platform requirements, including Feng and Maichuang platforms. AI algorithm type test resources need to run on the Maichuang platform, and digital test resources need to run on the Feng platform; Mem represents the test resource running load; Com represents the communication cost between test resources; com ij represents the communication cost between test resource i and test resource j.

4. The method of claim 3, wherein, Step 2 includes: based on the application scheduling cloud computing resources, the mapping relationship between digital test resources and physical computing resources is established; The mapping relationship between test resources and cloud computing resources is that the test resources in the same group are further allocated to the corresponding intelligent computing platform cluster resources to form the overall test resource deployment strategy, and the mapping relationship between digital test resources and physical resources is generated; Firstly, according to the running platform requirements of the deployed test resources, the corresponding cluster resources are selected. For AI algorithm type resources, Maichuang platform type cluster resources are allocated; for digital test resources, Feng platform type cluster resources are selected; Secondly, the cluster resources required by the deployed test resources are sorted from small to large, and the test resources with small cluster resources are preferentially selected for deployment; Finally, the resource utilization greedy algorithm is adopted. The resource utilization greedy algorithm calculates the CPU, memory and disk utilization of each cluster node, and prioritizes the queue with the lowest expected resource utilization, and preferentially allocates and deploys the cluster node with the lowest utilization. At this time, each digital test resource container is regarded as a separate deployment unit.

5. The method of claim 4, wherein, In step 3, the test resource image containerization packaging is used to generate a lightweight container image.

6. The method of claim 5, wherein, Step 3 includes: first, packaging the digital test resources and their running dependency packages into an image, dynamically calling the corresponding code compiler for code compilation, generating an executable code package, and loading it into the container for compilation and testing; the generated executable code package and the running dependent basic environment are integrated and packaged to form a test resource image; Secondly, generate the test resource container image environment variable, including declaring the port that the service in the image listens to, the default entry instruction of the container image, the username and ID when running the container; Finally, generate the container image by calling the Build command and upload it to the image library for sharing scheduling.

7. The method of claim 6, wherein, In step 4, the multi-user test resource parallel container deployment includes building a test user characterization model and test resource parallel container deployment.

8. The method of claim 7, wherein, Step 4 includes: first, building a test user characterization model, and the specific characterization elements include: user name, user token, test task identification, test sample identification, and digital simulation resource name; when the test user logs in to access the portal, intercept the test user access token information as the unique identification of the test user; Secondly, test resource parallel container deployment, load the test user permission and identity and test sample class identification information into the container environment variable, load the scheduling and deployment container image from the image library to the corresponding cluster node, deploy the test resource container, and start the container to run.

9. An electronic device, comprising: The processor and the memory, the memory stores program code, when the program code is executed by the processor, makes the processor execute the steps of the method in any one of claims 1 to 8.

10. A storage medium, characterized by The computer program or instructions are stored, and when the computer program or instructions run on the computer, the steps of the method in any one of claims 1 to 8 are executed.

Images