Compatibility test method, related device and medium

Abstract

The invention provides a compatibility testing method, a related device and a medium. The method comprises the following steps: aiming at a to-be-tested application program, constructing a plurality of container mirror images matched with a plurality of heterogeneous instruction set architecture nodes; converting a test case template for the to-be-tested application program into a plurality of special test codes matched with the plurality of heterogeneous instruction set architecture nodes; the container mirror images and the special test codes are scheduled to the corresponding heterogeneous instruction set architecture nodes, so that the corresponding heterogeneous instruction set architecture nodes create container instances, the corresponding special test codes are operated, and test results of the to-be-tested application program in the corresponding heterogeneous instruction set architecture nodes are obtained; according to the method, the test results of the to-be-tested application program in the multiple heterogeneous instruction set architecture nodes are collected and compared, the compatibility of the to-be-tested application program and the multiple heterogeneous instruction set architecture nodes is obtained, the test cost is reduced, the test period is shortened, and the test efficiency and the test coverage rate are improved.

Description

Technical Field

[0001] The present disclosure belongs to the field of virtualization technology, and particularly relates to a compatibility testing method, related devices and media. Background Art

[0002] With the rapid development of cloud computing and edge computing technologies, the instruction set architectures of nodes in data centers are gradually evolving towards heterogeneity. Heterogeneous instruction set architectures can provide higher performance and lower energy consumption, meeting the diverse workload requirements of data centers. Among them, the RISC-V instruction set architecture, as an open-source instruction set architecture, has been widely used in servers and edge devices due to its low power consumption, high performance and other characteristics. The X86 instruction set architecture dominates in data centers due to its mature ecosystem and extensive software support. In a heterogeneous instruction set architecture environment, the importance of implementing software compatibility testing across nodes with different instruction set architectures has become increasingly prominent. However, traditional compatibility testing methods mainly rely on building and maintaining a large number of heterogeneous hardware platforms and running the same test cases on each platform. This method has many problems: 1. The testing cost is high because building and maintaining multiple hardware platforms requires a large amount of capital investment, and the utilization rate of hardware resources is relatively low; 2. The testing cycle is long, as testing needs to be carried out on different hardware platforms, consuming a large amount of time and unable to meet the rapid iteration requirements of software development; 3. It is difficult to automate the deployment and execution of test cases on different hardware platforms, resulting in low testing efficiency; 4. Due to the limitations of hardware platforms, it is difficult to cover all target instruction set architectures and software configurations, resulting in insufficient test coverage. Therefore, there is an urgent need to provide a compatibility testing method to solve the above problems of testing software compatibility across nodes with different instruction set architectures in a heterogeneous instruction set architecture environment. Summary of the Invention

[0003] In view of the above problems, the present disclosure provides a compatibility testing method, related devices and media, aiming to reduce the testing cost and testing cycle of testing software compatibility across nodes with different instruction set architectures in a heterogeneous instruction set architecture environment, and improve testing efficiency and test coverage.

[0004] According to the first aspect of the present disclosure, there is provided a compatibility testing method, including:

[0005] For the application to be tested, build multiple container images adapted to multiple heterogeneous instruction set architecture nodes;

[0006] Convert the test case template for the application to be tested into multiple dedicated test codes respectively adapted to the multiple heterogeneous instruction set architecture nodes;

[0007] Schedule the multiple container images and the multiple dedicated test codes to corresponding heterogeneous instruction set architecture nodes, so that the corresponding heterogeneous instruction set architecture nodes create container instances and run the corresponding dedicated test codes, and obtain the test results of the application under test on the corresponding heterogeneous instruction set architecture nodes;

[0008] Collect and compare the test results of the application under test on the multiple heterogeneous instruction set architecture nodes to obtain the compatibility between the application under test and the multiple heterogeneous instruction set architecture nodes.

[0009] Optionally, constructing multiple container images adapted to multiple heterogeneous instruction set architecture nodes for the application under test includes:

[0010] Establish a construction environment for constructing the multiple container images adapted to the multiple heterogeneous instruction set architecture nodes;

[0011] Establish a configuration file template for constructing container images for the application under test;

[0012] Use an automated build script to establish the multiple container images adapted to the multiple heterogeneous instruction set architecture nodes based on the configuration file template.

[0013] Optionally, converting the test case template for the application under test into multiple dedicated test codes respectively adapted to the multiple heterogeneous instruction set architecture nodes includes:

[0014] Establish a test case template for the application under test;

[0015] Parse the test case template to obtain test information for performing compatibility testing on the application under test, where the test information at least includes test case names, test steps, assertion conditions, and test parameters;

[0016] Use a test code generator to generate the multiple dedicated test codes adapted to the multiple heterogeneous instruction set architecture nodes based on the test information.

[0017] Optionally, scheduling the multiple container images and the multiple dedicated test codes to corresponding heterogeneous instruction set architecture nodes, so that the corresponding heterogeneous instruction set architecture nodes create container instances and run the corresponding dedicated test codes, and obtain the test results of the application under test on the corresponding heterogeneous instruction set architecture nodes includes:

[0018] Establish a test execution task template for the application under test, where the test execution task template includes the running environment of the test execution task, the configuration information of the container under test, and the test case name;

[0019] Using a scheduler, based on the information in the test execution task template, schedule a container image and dedicated test code adapted to the heterogeneous instruction set architecture node to the heterogeneous instruction set architecture node;

[0020] Wherein, the heterogeneous instruction set architecture node creates a container instance of the application under test using a container image adapted to the heterogeneous instruction set architecture node, and executes dedicated test code adapted to the heterogeneous instruction set architecture node, so as to test the created container instance, and obtain the test result of the application under test on the heterogeneous instruction set architecture node.

[0021] Optionally, the collecting and comparing the test results of the application under test on the multiple heterogeneous instruction set architecture nodes to obtain the compatibility between the application under test and the multiple heterogeneous instruction set architecture nodes includes:

[0022] Define the data structure of the test result, and the data structure includes the test case name, instruction set architecture type,

[0023] Execution status, execution time, and test log;

[0024] Collect the test results of the application under test on the multiple heterogeneous instruction set architecture nodes;

[0025] Compare the test results of the application under test on the multiple heterogeneous instruction set architecture nodes across the instruction set architectures, and obtain the compatibility between the application under test and the multiple heterogeneous instruction set architecture nodes based on the comparison result.

[0026] Optionally, the comparing the test results of the application under test on the multiple heterogeneous instruction set architecture nodes across the instruction set architectures, and obtaining the compatibility between the application under test and the multiple heterogeneous instruction set architecture nodes based on the comparison result includes:

[0027] Compare the program execution time, resource utilization efficiency, and performance bottleneck of the application under test on the multiple heterogeneous instruction set architecture nodes, and obtain the performance compatibility between the application under test and the multiple heterogeneous instruction set architecture nodes based on the comparison result; and / or

[0028] Compare the consistency of the program execution behavior of the application under test on the multiple heterogeneous instruction set architecture nodes, and the correctness of instruction translation and simulation, and obtain the instruction set compatibility between the application under test and the multiple heterogeneous instruction set architecture nodes based on the comparison result; and / or

[0029] Compare the consistency of system call parameter passing and return values and the equivalence of system call behaviors of the application under test on the multiple heterogeneous instruction set architecture nodes, and obtain the system call interface compatibility between the application under test and the multiple heterogeneous instruction set architecture nodes based on the comparison result; and / or

[0030] Compare the function call conventions, data type representations, memory alignment requirements, and exception handling mechanisms of the application under test on the multiple heterogeneous instruction set architecture nodes, and obtain the application binary interface compatibility between the application under test and the multiple heterogeneous instruction set architecture nodes based on the comparison result.

[0031] Optionally, after collecting and comparing the test results of the application under test on the multiple heterogeneous instruction set architecture nodes to obtain the compatibility between the application under test and the multiple heterogeneous instruction set architecture nodes, the compatibility testing method further includes:

[0032] Generate a compatibility test report for the multiple heterogeneous instruction set architecture nodes for the application under test according to the comparison result of the test results of the application under test on the multiple heterogeneous instruction set architecture nodes.

[0033] According to a second aspect of the present disclosure, there is provided a compatibility testing apparatus, including:

[0034] A container image builder for building multiple container images adapted to multiple heterogeneous instruction set architecture nodes for an application under test;

[0035] A test case converter for converting a test case template for the application under test into multiple dedicated test codes respectively adapted to the multiple heterogeneous instruction set architecture nodes;

[0036] A distributed test execution engine for scheduling the multiple container images and the multiple dedicated test codes to corresponding heterogeneous instruction set architecture nodes, so that the corresponding heterogeneous instruction set architecture nodes create container instances and run the corresponding dedicated test codes to obtain the test results of the application under test on the corresponding heterogeneous instruction set architecture nodes;

[0037] A result comparer for collecting and comparing the test results of the application under test on the multiple heterogeneous instruction set architecture nodes to obtain the compatibility between the application under test and the multiple heterogeneous instruction set architecture nodes.

[0038] According to a third aspect of the present disclosure, there is provided an electronic device including a memory, a processor, and a program stored on the memory and executable on the processor, and when the program is executed by the processor, the above-mentioned method can be implemented.

[0039] In a fourth aspect of the present disclosure, there is provided a storage medium having a computer program or instructions stored thereon, and when the computer program or instructions are executed by a processor, the steps of the method described in any one of the above are implemented.

[0040] The present disclosure brings the following beneficial effects:

[0041] For the compatibility testing method provided by the present disclosure, for the application under test, a plurality of container images adapted to a plurality of heterogeneous instruction set architecture nodes are constructed, the test case template for the application under test is converted into a plurality of dedicated test codes respectively adapted to the plurality of heterogeneous instruction set architecture nodes, and the plurality of container images and the plurality of dedicated test codes are scheduled to the corresponding heterogeneous instruction set architecture nodes, so that the corresponding heterogeneous instruction set architecture nodes create container instances in parallel, run the corresponding dedicated test codes, obtain the test results of the application under test on the corresponding heterogeneous instruction set architecture nodes, collect and compare the test results of the application under test on the plurality of heterogeneous instruction set architecture nodes, and obtain the compatibility of the application under test with the plurality of heterogeneous instruction set architecture nodes. In this way, through the application of container technology, efficient and reliable technical support is provided for the cross-instruction set architecture transplantation of software, and it is possible to quickly deploy and automatically manage the test environment and test cases of the application program in a heterogeneous instruction set architecture environment, greatly shortening the test cycle, improving the test efficiency, reducing the dependence on the hardware platform, and reducing the construction and maintenance costs of the test platform. By constructing different container images and dedicated test codes adapted to different heterogeneous instruction set architecture nodes, compatibility testing can be conveniently performed in a heterogeneous instruction set architecture environment, improving the test coverage rate.

[0042] Other features and advantages of the present disclosure will be described in the following specification, and in part, will be obvious from the specification, or will be understood by implementing the present disclosure. The objectives and other advantages of the present disclosure are achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0043] To make the above objectives, features, and advantages of the present disclosure more obvious and understandable, the following specific preferred embodiments are given, and in conjunction with the accompanying drawings, the detailed description is as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, the above and other objectives, features, and advantages of the present disclosure will become clearer. In the drawings:

[0045] Figure 1 is a structural diagram of a data center applied in an embodiment of the present disclosure;

[0046] Figure 2 is an internal structural diagram of a working node according to an embodiment of the present disclosure;

[0047] Figure 3 A flowchart showing a compatibility testing method provided according to an embodiment of the present disclosure;

[0048] Figure 4 A schematic structural diagram showing a compatibility testing apparatus provided according to an embodiment of the present disclosure;

[0049] Figure 5 A schematic structural diagram showing an electronic device provided according to an embodiment of the present disclosure. Detailed implementation manners

[0050] The various embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. In each of the drawings, the same elements are denoted by the same or similar reference numerals. For clarity, the various parts in the drawings are not drawn to scale.

[0051] The following terms are used herein:

[0052] Container: As a lightweight virtualization technology, it is a group of processes that are resource - restricted and isolated from each other. Container technology creates an independent running environment for different application programs, realizes resource isolation, configuration, and security guarantee, and can meet the resource requirements of applications allocated on - demand and ensure the isolation and availability of applications. The application program carried in the container is called a container instance, or can also be called a containerized application. To meet the needs of large - scale applications, in practice, it is often necessary to deploy many containers in a computer cluster for unified management and external service provision. Therefore, container orchestration tools are required. Container orchestration tools use container services and orchestrate them to determine how containers interact with each other, extend the lifecycle management ability to complex, multi - container workloads deployed on a large number of computer clusters, and provide an abstraction layer for developers and infrastructure teams to handle large - scale containerized deployments. Container orchestration tools such as the K8s (full name is Kubernetes, a system for running and coordinating containerized application processes) system, which is an open - source system for automatically deploying, scaling, and managing containerized applications. Docker is an open - source application container engine, a tool for starting and stopping containers, which allows developers to package their application programs and dependent packages into a portable container and then publish it to any popular machine, and can also achieve virtualization. Containers use a sandbox mechanism completely and there will be no interfaces between them.

[0053] Pod: It is the basic operation unit of container orchestration tools (e.g., K8s), and the smallest deployable unit that can be created, debugged, and managed. All containers in the same pod share the same IP address, IPC, host name, and other resources. A pod abstracts network and storage from the underlying containers, making it easier to move containers in a cluster. Each pod can encapsulate one or more containers for hosting applications. The containers in a pod are scheduled as a whole to run on a worker node.

[0054] Compatibility testing is to verify the degree of dependence of the software under test on the heterogeneous instruction set architecture environment it depends on, including the degree of dependence on the hardware platform and software platform, that is, to detect the portability of the software between heterogeneous instruction set architecture nodes.

[0055] Figure 1 It is the structural diagram of the data center applied in an embodiment of the present disclosure. As Figure 1 shown, for convenience of description, Figure 1 only the control node 110 and the worker node 120 are shown. In implementation, the control node 110 or the worker node 120 can be a server or a virtual machine on the server. It should be understood that although Figure 1 only a limited number of worker nodes 120 are shown, the present disclosure should not be limited thereto. In some embodiments, the worker node 120 is a heterogeneous instruction set architecture node, including worker node A 220 and worker node B 220, where worker node A 220 supports the RISC-V instruction set architecture and worker node B 220 supports the X86 instruction set architecture. It can be understood that the worker node 120 can also support other instruction set architectures such as the ARM instruction set architecture. For convenience of description, Figure 1 it is not shown.

[0056] In some embodiments, the data center can be applied to various application scenarios such as Content Delivery Network (CDN), e-commerce, gaming, audio and video, Internet of Things, logistics, industrial brain, urban brain, etc., and provide computing services for end users in various scenarios. Specifically, for each application scenario, an application program that can provide computing services in the application scenario can be deployed in the working node 120 of the data center. Considering that a large number of applications may need to be deployed in the data center, container technology can be adopted to build a container group in the working node 120, carry the application program through the container, and then deploy the application program in units of containers. By running these container instances deployed on the working node 120, corresponding computing services can be provided for end users. It should be noted that for the working node A 220, the containers deployed thereon support the RISC-V instruction set architecture. For the working node B 220, the containers deployed thereon support the X86 instruction set architecture.

[0057] In some embodiments, a container orchestration tool (e.g., K8s) can run on the control node 110 to achieve the orchestration and management of container instances in the data center to which it belongs. Among them, the orchestration and management of container instances include at least one of the following: creation, elastic scaling, rolling update, reconstruction, migration, shutdown, etc. of container instances. In some embodiments, in addition to the container orchestration management function, the control node 110 can also be responsible for other management operations in the data center to which it belongs, such as monitoring and management of operation and maintenance, logs, network status, etc. Commands can be sent to each working node 120 through the control node 110. Simply put, the control node 110 is the manager, and the working node 120 is the managed. The background services running on the control node 110 generally can include an Application Programming Interface server (APIserver) 111 and a Scheduler 130. The Application Programming Interface server 111 is the front-end interface of the container orchestration tool (e.g., K8s), and various client tools and other components of the container orchestration tool (e.g., K8s) can manage various resources of the data center cluster through the Application Programming Interface server 111. The Scheduler 130 can determine on which working node to run the container group and determine the resources allocated to the container group.

[0058] In some embodiments, since the working node 120 is the real processing device of the data center, Figure 2The internal structure diagram of a working node according to an embodiment of the present disclosure is shown. In some embodiments, the working node 120 may include a plurality of processors 22 and a memory 29. The memory 29 in the working node 120 may be a main memory (abbreviated as main storage or memory), which is used to store instruction information and / or data information represented by data signals. For example, it stores data provided by the processor 22 (such as operation results), and can also be used to implement data exchange between the processor 22 and an external storage device 26 (or referred to as auxiliary storage or external memory).

[0059] In some cases, the processor 22 may need to access the memory 29 to obtain data in the memory 29 or modify the data in the memory 29. Since the access speed of the memory 29 is relatively slow, in order to alleviate the speed gap between the processor 22 and the memory 29, the working node 120 further includes a cache memory 28 coupled to the bus 21. The cache memory 28 is used to cache some program data or message data that may be repeatedly called in the memory 29. The cache memory 28 is implemented by a storage device such as a static random access memory (abbreviated as SRAM). The cache memory 28 may be a multi-level structure, such as a three-level cache structure having a level 1 cache (L1 Cache), a level 2 cache (L2 Cache), and a level 3 cache (L3 Cache), or may be a cache structure with three or more levels or other types of cache structures. In some embodiments, a part of the cache memory 28 (such as the level 1 cache, or the level 1 cache and the level 2 cache) may be integrated inside the processor 22 or integrated with the processor 22 on the same system-on-chip.

[0060] Based on this, the processor 22 may include an instruction execution unit 221, a memory management unit 222, and other parts. When the instruction execution unit 221 executes some instructions that need to modify the memory, it initiates a write access request, and the write access request specifies the write data to be written into the memory and the corresponding physical address; the memory management unit 222 is used to translate the virtual address specified by these instructions into the physical address mapped by the virtual address, and the physical address specified by the write access request may be the same as the physical address specified by the corresponding instruction.

[0061] The information interaction between the memory 29 and the cache 28 is usually organized in blocks. In some embodiments, the cache 28 and the memory 29 can be divided into data blocks according to the same spatial size, and the data blocks can be used as the minimum unit of data exchange between the cache 28 and the memory 29 (including one or more data of a preset length). For the sake of concise and clear expression, each data block in the cache 28 is hereinafter simply referred to as a cache block (which can be called a cache line or a cache line), and different cache blocks have different cache block addresses; each data block in the memory 29 is simply referred to as a memory block, and different memory blocks have different memory block addresses. The cache block address includes, for example, a physical address tag for locating the data block.

[0062] Due to space and resource limitations, the cache 28 cannot cache all the content in the memory 29, that is, the storage capacity of the cache 28 is usually smaller than that of the memory 29, and the cache block addresses provided by the cache 28 cannot correspond to all the memory block addresses provided by the memory 29. When the processor 22 needs to access the memory, it first accesses the cache 28 via the bus 21 to determine whether the content to be accessed has been stored in the cache 28. If so, the cache 28 is hit, and at this time, the processor 22 directly calls the content to be accessed from the cache 28; if the content that the processor 22 needs to access is not in the cache 28, the processor 22 needs to access the memory 29 via the bus 21 to find the corresponding information in the memory 29. Since the access rate of the cache 28 is very fast, when the cache 28 is hit, the efficiency of the processor 22 can be significantly improved, and thus the performance and efficiency of the entire working node 120 are also improved.

[0063] As shown in the figure above, the processor 22, the cache 28, and the memory 29 are encapsulated in a system-on-chip (SoC) 201. Designers can configure the SoC architecture so that the communication between the components in the working node 120 is secure.

[0064] In this example, the working node 120 may also include various software. In some embodiments, such as Figure 2As shown in the figure, an operating system 202, container support 203, and container group 204 are provided on the underlying hardware (i.e., the system-on-chip 201). The operating system 202 is, for example, an operating system such as UNIX operating system, Linux operating system, etc. that can be used on a server. The worker node 120 supports the RISC-V instruction set architecture or the X86 instruction set architecture or other instruction set architectures. The container support 203 is various underlying implementations required to support the containers running thereon. For example, for docker, the container support 203 needs to implement two technologies of cgroup (abbreviation of controlgroups, control group) and namespace (namespace). Cgroup realizes resource quota, and namespace realizes resource isolation. Docker allows developers to package their applications and the running environments on which they depend into a portable container and then publish it to the worker node 120. As Figure 2 shown, based on the container support 203, multiple container groups 204 can be run, and the running environment and running application programs are implemented in each container of the container group 204, and isolation and non-interference between various application programs are achieved through container technology. The application programs can include, but are not limited to, programs for controlling or responding to external devices (e.g., biometric sensors, printers, microphones, speakers, flow valves, or other I / O components, sensors, actuators, or devices), programs for various I / O tasks, security programs, authentication programs, various computing modules, communication programs, communication support protocols, or other programs, or combinations thereof.

[0065] In some embodiments, as Figure 2 shown, on the underlying hardware (i.e., the system-on-chip 201), there is also provided

[0066] The service proxy (Kubelet) 205 of the container orchestration tool (e.g., K8s) on the worker node 120. In implementation, the service proxy (Kubelet) 205 can be a program module of software or hardware, such as implemented based on FPGA or CPLD, etc. In some embodiments, the service proxy 205 can receive and execute instructions sent by the control node 110, and manage the container group and the containers in the container group. The service proxy 205 can register the information of the worker node 120 where it is located on the application programming interface server 111 of the control node 110, monitor the resource usage of the worker node 120, and regularly report the resource usage data of the worker node 120 to the control node 110. In some embodiments, a garbage collection (GC) module (not shown in the figure) can also be provided on top of the underlying hardware (i.e., the system-on-chip 201). In implementation, the garbage collection (GC) module can be a program module of software or hardware, such as implemented based on FPGA or CPLD, etc. In some embodiments, the garbage collection (GC) module can establish a garbage collection process for each container, and clear the configuration information of the container group in the case where the container group is destroyed.

[0067] In addition, the worker node 120 may further include hardware devices such as a display device 23, an audio device 24, an input / output device 25, a storage device 26, a communication device 27, etc. Of course, the structures of different computer systems may vary according to different motherboards, operating systems, and instruction set architectures.

[0068] Refer again to Figure 1 , as Figure 1As shown, the background service running on the control node 110 may further include a compatibility testing device 140. The compatibility testing device 140 can provide efficient and reliable technical support for the cross-instruction set architecture transplantation of software through the application of container technology, and realize the rapid deployment and automated management of the test environment and test cases of application programs in a heterogeneous instruction set architecture environment, and test the compatibility of application programs with multiple heterogeneous instruction set architecture nodes (including worker node A 220 and worker node B 220). Specifically, the compatibility testing device 140 includes a container image builder 141, a test case converter 142, a distributed test execution engine 143, a result comparer 144, a test report generator 145, a test management layer 146, and a user interface layer 147. In some embodiments, the user interface layer 147 may provide a test task configuration interface for the user to configure the compatibility test task of the application program to be tested, display the test execution status, and present the test results and reports. In some embodiments, the test management layer 146 may receive the test tasks configured in the user interface layer 147, schedule the test tasks, and thus submit the test tasks to the container image builder 141, the test case converter 142, and the distributed test execution engine 143.

[0069] In some embodiments, the container image builder 141 builds multiple container images adapted to a plurality of heterogeneous instruction set architecture nodes (including worker node A 220 and worker node B 220) for the application under test. In some embodiments, a build environment for building multiple container images adapted to a plurality of heterogeneous instruction set architecture nodes is established. For the application under test, a configuration file (Dockerfile) template for building container images is established. Using an automated build script, multiple container images adapted to a plurality of heterogeneous instruction set architecture nodes are established based on the configuration file template. In some embodiments, an application container engine (Docker) and a container image building tool (BuildKit) can be installed on the control node 110 to prepare for subsequent building of container images across instruction set architecture nodes. Based on BuildKit, the "docker buildx create" function can be used to create and use a new builder instance. In some embodiments, a multi-stage build method can be used to build the Dockerfile template for building container images across instruction set architecture nodes. It should be noted that the Dockerfile template is uniformly managed and modified on the control node 110 without the need to modify it on the worker node 120, and is automatically adapted to different instruction set architectures through the multi-platform build feature of DockerBuildX. It should be noted that the automated build script for building container images across instruction set architectures is only executed on the control node 110 without the need to be executed on each heterogeneous instruction set architecture node separately. On the control node 110, the build task of container images across instruction set architectures is managed through the multi-platform build feature of Docker BuildX. The QEMU emulator can be used to implement the build task of container images across instruction set architectures.

[0070] In some embodiments, the test case converter 142 converts a test case template for the application under test into multiple dedicated test codes respectively adapted to multiple heterogeneous instruction set architecture nodes. In some embodiments, a test case template for the application under test is established. The test case template is parsed to obtain test information for performing a compatibility test on the application under test. Using a test code generator, based on the test information, multiple dedicated test codes adapted to multiple heterogeneous instruction set architecture nodes are generated. It should be noted that the test case template uses a structured way to describe the test steps. The test case template is a predefined format for recording the detailed information of the test case. The test information included in the test case template includes the test case name (name), test steps (steps), assertion conditions (assertions), and test parameters (parameters). The test steps (steps) is a list, and each test step has an action, an element, and a value corresponding to the element. For example, the action can be click, input, etc. The element is the identifier of the page element to be operated, which can be a button (button#submit), an input box (input#username), etc. The value corresponding to the element can be the value of the input action. For example, the value of the input box. The assertion conditions (assertions) is a list containing the assertions to be checked after the test ends. Each assertion has a type and an element. The type can be the type of assertion to be checked, such as element_present, etc. The element is the identifier of the page element to be checked. For example, a message box (div.success-message), that is, to check whether the operation success message appears. In some embodiments, the "def parse_test_case(yaml_file)" function is used to parse the test case template in YAML format to obtain the test information for performing a compatibility test on the application under test. In some embodiments, the "def generate_test_code(test_case, architecture)" function is used to generate multiple dedicated test codes adapted to multiple heterogeneous instruction set architecture nodes according to the test case template (test_case) and the target instruction set architecture (architecture). It can be understood that the test code generator is deployed on the control node 110. Through the "def generate_test_code(test_case, architecture)" function, the test case template (test_case) is converted into a dedicated test code adapted to the target instruction set architecture (architecture).For example, if the target instruction set architecture is "x86_64", then "generate_x86_code(test_case)" is returned, that is, dedicated test code adapted to the "x86_64" instruction set architecture is generated. It can be understood that the test code generator can convert a general test case template into dedicated test code adapted to each heterogeneous instruction set architecture.

[0071] In some embodiments, the distributed test execution engine 143 schedules multiple container images and multiple dedicated test codes to corresponding heterogeneous instruction set architecture nodes, so that the corresponding heterogeneous instruction set architecture nodes create container instances and run the corresponding dedicated test codes to obtain the test results of the application under test on the corresponding heterogeneous instruction set architecture nodes. In some embodiments, for the application under test, a test execution task (Job) template is established. The test execution Job template includes the running environment of the test execution task, the configuration information of the container under test, and the test case name. In some embodiments, such as Figure 1The data center shown can also be referred to as a Kubernetes cluster, and kubeadm or a managed Kubernetes service provided by a cloud service provider can be used. The established test execution Job template is a Kubernetes Job template. It can be understood that in Kubernetes, a Job is a controller responsible for creating one or more Pods to execute a task (Job), and then stopping these Pods after the task is completed. In other words, a Job is used to run limited tasks or batch tasks. The test execution Job template includes the version information (apiVersion) of the Kubernetes API, the type of Kubernetes resource to be created (kind), and the resource metadata (metadata). Among them, apiVersion: batch / v1 indicates that the batch / v1 version of the Kubernetes API is being used, and the API of this version includes some batch-related resource types such as Job. kind: Job indicates that the type of resource being created is Job. Metadata is used to store metadata about the Job, such as the name of the Job, the specification (spec) of the Job, labels, and annotations. The specification (spec) of the Job includes the configuration information of the container and container image to be tested and the information about the command to be run after the container starts (i.e., the test-specific code). In some embodiments, using a scheduler, based on the information in the test execution Job template, the container image and dedicated test code adapted to the heterogeneous instruction set architecture node are scheduled to this heterogeneous instruction set architecture node. In some embodiments, "def schedule_test(test_name, test_image)" is used to schedule the container image and dedicated test code adapted to the heterogeneous instruction set architecture node to the heterogeneous instruction set architecture node. For example, the implementation code for using a scheduler to perform test scheduling to schedule the container image and dedicated test code adapted to the heterogeneous instruction set architecture node to the heterogeneous instruction set architecture node is as follows:

[0072] from kubernetes import client, config

[0073] def schedule_test(test_name, test_image):

[0074] """

[0075] Schedule the test task to be executed in the Kubernetes cluster

[0076] Parameters:

[0077] test_name: The name of the test task

[0078] test_image: Test container image

[0079] """

[0080] # Load Kubernetes configuration

[0081] config.load_kube_config()

[0082] # Create BatchV1Api client

[0083] batch_v1 = client.BatchV1Api()

[0084] # Read Job template file

[0085] with open("job_template.yaml") as f:

[0086] job_manifest = yaml.safe_load(f)

[0087] # Replace variables in the template

[0088] job_manifest['metadata']['name']= f"test-job-{test_name}"

[0089] job_manifest['spec']['template']['spec']['containers'][0]['image']=test_image

[0090] # Create Job in the default namespace

[0091] batch_v1.create_namespaced_job(

[0092] namespace="default",

[0093] body=job_manifest )

[0095] In some embodiments, the steps of performing test scheduling using a scheduler include receiving a test request, checking the resource availability of worker node 120, selecting a target worker node 120, creating a test execution Job on the target worker node 120, setting the container image to be tested on the target worker node 120 according to the information in the test execution Job, and allocating resources for the container. After the container is started, the target worker node 120 runs dedicated test code adapted to the instruction set architecture supported by the target worker node 120 for the container, and monitors the test execution process. It can be understood that on different heterogeneous instruction set architecture nodes, container instances of the application to be tested can be created in parallel using container images adapted to the heterogeneous instruction set architecture nodes, and dedicated test code adapted to the heterogeneous instruction set architecture nodes can be executed to test the created container instances, so as to obtain the test results of the application to be tested on the heterogeneous instruction set architecture nodes.

[0096] In some embodiments, the result comparer 144 collects and compares the test results of the application to be tested on multiple heterogeneous instruction set architecture nodes to obtain the compatibility of the application to be tested with multiple heterogeneous instruction set architecture nodes (for example, RISC-V instruction set architecture nodes, X86 instruction set architecture nodes, and ARM instruction set architecture nodes, etc.). In some embodiments, the data structure of the test results is defined. For example, the data structure includes the test case name (test_name), the instruction set architecture type (architecture), the execution status (status, for example, success / failure), the execution time (execution_time), and the test log (logs). The test results of the application to be tested on multiple heterogeneous instruction set architecture nodes are collected. The test results of the application to be tested on multiple heterogeneous instruction set architecture nodes are compared across instruction set architectures, and the compatibility of the application to be tested with multiple heterogeneous instruction set architecture nodes is obtained based on the comparison results.

[0097] In some embodiments, the compatibility of the application to be tested with multiple heterogeneous instruction set architecture nodes includes instruction set compatibility. The consistency of the program execution behavior of the application to be tested on multiple heterogeneous instruction set architecture nodes, as well as the correctness of instruction translation and simulation, etc. can be compared. In this way, the instruction set compatibility of the application to be tested with multiple heterogeneous instruction set architecture nodes is obtained based on the comparison results. For example, if the program execution behavior of the application to be tested on an instruction set architecture node (for example, an ARM instruction set architecture node) is inconsistent with that on other instruction set architecture nodes (for example, RISC-V instruction set architecture nodes and X86 instruction set architecture nodes), it is obtained that the application to be tested is not instruction set compatible with this instruction set architecture node. For example, if the instruction translation and simulation of the application to be tested on an instruction set architecture node (such as an X86 instruction set architecture node) are incorrect, the application to be tested is not instruction set compatible with this instruction set architecture node.

[0098] In some embodiments, the compatibility of the application under test with multiple heterogeneous instruction set architecture nodes includes system call interface compatibility. The consistency of the system call parameter passing and return values of the application under test on multiple heterogeneous instruction set architecture nodes and the equivalence of the system call behaviors can be compared. In this way, the system call interface compatibility of the application under test with multiple heterogeneous instruction set architecture nodes can be obtained based on the comparison results. For example, if the system call parameter passing and return values of the application under test on an instruction set architecture node (e.g., an ARM instruction set architecture node) are inconsistent with those on other instruction set architecture nodes (e.g., a RISC-V instruction set architecture node and an X86 instruction set architecture node), it is obtained that the system call interface of the application under test with this instruction set architecture node is incompatible. For example, if the system call behavior of the application under test on an instruction set architecture node (such as an ARM instruction set architecture node) is not equivalent to that on other instruction set architecture nodes (such as a RISC-V instruction set architecture node and an X86 instruction set architecture node), the system call interface of the application under test with this instruction set architecture node is incompatible.

[0099] In some embodiments, the compatibility of the application under test with multiple heterogeneous instruction set architecture nodes includes ABI (Application Binary Interface) compatibility. The function call conventions, data type representations, memory alignment requirements, exception handling mechanisms, etc. of the application under test on multiple heterogeneous instruction set architecture nodes can be compared. In this way, the ABI compatibility of the application under test with multiple heterogeneous instruction set architecture nodes can be obtained based on the comparison results. For example, if the function call convention, data type representation, memory alignment requirement, or exception handling mechanism of the application under test on an instruction set architecture node (such as an ARM instruction set architecture node) is different from those on other instruction set architecture nodes (such as a RISC-V instruction set architecture node and an X86 instruction set architecture node), the ABI of the application under test with this instruction set architecture node is incompatible.

[0100] In some embodiments, the compatibility of the application under test with multiple heterogeneous instruction set architecture nodes includes performance compatibility. The program execution time, resource utilization efficiency, performance bottlenecks, etc. of the application under test on multiple heterogeneous instruction set architecture nodes can be compared. In this way, the performance compatibility of the application under test with multiple heterogeneous instruction set architecture nodes can be obtained based on the comparison results. For example, if the program execution time of the application under test on an instruction set architecture node (e.g., an ARM instruction set architecture node) is significantly longer than that on other instruction set architecture nodes (e.g., a RISC-V instruction set architecture node and an X86 instruction set architecture node), then the application under test is not performance-compatible with that instruction set architecture node. For example, if the resource utilization efficiency of the application under test on an instruction set architecture node (e.g., an ARM instruction set architecture node) is significantly worse than that on other instruction set architecture nodes (e.g., a RISC-V instruction set architecture node and an X86 instruction set architecture node), then the application under test is not performance-compatible with that instruction set architecture node. For example, if the application under test reaches a performance bottleneck on an instruction set architecture node (e.g., an ARM instruction set architecture node) compared to other instruction set architecture nodes (e.g., a RISC-V instruction set architecture node and an X86 instruction set architecture node), then the application under test is not performance-compatible with that instruction set architecture node.

[0101] In some embodiments, the test report generator 145 generates a compatibility test report for multiple heterogeneous instruction set architecture nodes for the application under test according to the comparison results of the test results of the application under test on multiple heterogeneous instruction set architecture nodes. In some embodiments, a test report template for the test results is established. Based on the comparison results of the test results of the application under test on multiple heterogeneous instruction set architecture nodes, the test report generator is used to generate a compatibility test report for multiple heterogeneous instruction set architecture nodes for the application under test. It should be noted that the compatibility test method implemented by the compatibility test device 140 in the embodiments of the present disclosure is controlled and executed by a pipeline designed by a CI / CD (Continuous Integration / Continuous Deployment) tool. In some embodiments, in the software development practice of the application under test, developers often merge code changes into the main branch, usually multiple times a day. Each merge runs automated tests through a pipeline designed by a CI / CD tool, which covers the entire process from building container images across instruction set architecture nodes, converting test case templates into dedicated test cases adapted to multiple heterogeneous instruction set architecture nodes, parallelly executing test cases on corresponding container instances on multiple heterogeneous instruction set architecture nodes, collecting and comparing test results, to generating a final test report. It can be understood that the embodiments of the present disclosure fully consider the requirements of software compatibility testing in a cross-instruction set architecture environment, and through container technology and an automated tool chain, an efficient and reliable test process is achieved.

[0102] Figure 3 A flowchart showing a compatibility testing method provided according to an embodiment of the present disclosure. As Figure 3 shown, the compatibility testing method of the embodiments of the present disclosure may include:

[0103] In step S310, for the application under test, a plurality of container images adapted to a plurality of heterogeneous instruction set architecture nodes are constructed.

[0104] In step S320, the test case template for the application under test is converted into a plurality of dedicated test codes respectively adapted to the plurality of heterogeneous instruction set architecture nodes.

[0105] In step S330, the plurality of container images and the plurality of dedicated test codes are scheduled to the corresponding heterogeneous instruction set architecture nodes, so that the corresponding heterogeneous instruction set architecture nodes create container instances and run the corresponding dedicated test codes to obtain the test results of the application under test on the corresponding heterogeneous instruction set architecture nodes.

[0106] In step S340, the test results of the application under test on the plurality of heterogeneous instruction set architecture nodes are collected and compared to obtain the compatibility between the application under test and the plurality of heterogeneous instruction set architecture nodes.

[0107] Since the process of using the compatibility testing method of the embodiments of the present disclosure to test the software compatibility across instruction set architecture nodes in a heterogeneous instruction set architecture environment has been described in detail in the device embodiments above, it will not be elaborated here.

[0108] Figure 4 A schematic diagram showing a compatibility testing device provided according to an embodiment of the present disclosure. As Figure 4 shown, the compatibility testing device 400 includes a container image builder 410, a test case converter 420, a distributed test execution engine 430, and a result comparer 440.

[0109] The container image builder 410 is configured to construct a plurality of container images adapted to a plurality of heterogeneous instruction set architecture nodes for the application under test.

[0110] The test case converter 420 is configured to convert the test case template for the application under test into a plurality of dedicated test codes respectively adapted to the plurality of heterogeneous instruction set architecture nodes.

[0111] A distributed test execution engine 430 is configured to schedule the multiple container images and the multiple dedicated test codes to corresponding heterogeneous instruction set architecture nodes, so that the corresponding heterogeneous instruction set architecture nodes create container instances, run the corresponding dedicated test codes, and obtain test results of the application under test on the corresponding heterogeneous instruction set architecture nodes.

[0112] A result comparer 440 is configured to collect and compare the test results of the application under test on the multiple heterogeneous instruction set architecture nodes, and obtain the compatibility between the application under test and the multiple heterogeneous instruction set architecture nodes.

[0113] Since the process of testing the software compatibility across instruction set architecture nodes in a heterogeneous instruction set architecture environment by using the compatibility testing method of the embodiments of the present disclosure has been described in detail in the device embodiments above, it will not be elaborated here.

[0114] Embodiments of the present disclosure further provide an electronic device, as Figure 5 shown, including a memory 520, a processor 510, and a program stored on the memory 520 and executable on the processor 510. When the program is executed by the processor 510, it can implement each process of the above-mentioned compatibility testing method in various embodiments, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0115] Those of ordinary skill in the art can understand that all or part of the steps in the above-mentioned various methods can be completed by instructions, or by controlling related hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. For this reason, embodiments of the present disclosure further provide a storage medium, on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, it can implement each process of the above-mentioned compatibility testing method in various embodiments.

[0116] Since the instructions stored in the storage medium can execute the steps in the compatibility testing method provided by the embodiments of the present disclosure, the beneficial effects that can be achieved by the compatibility testing method provided by the embodiments of the present disclosure can be realized. For details, refer to the previous embodiments and will not be elaborated here. The specific implementation of each of the above operations can refer to the previous embodiments and will not be elaborated here.

[0117] In summary, for the compatibility testing method provided by the present disclosure, for the application to be tested, multiple container images adapted to multiple heterogeneous instruction set architecture nodes are constructed, the test case templates for the application to be tested are converted into multiple dedicated test codes respectively adapted to the multiple heterogeneous instruction set architecture nodes, the multiple container images and the multiple dedicated test codes are scheduled to the corresponding heterogeneous instruction set architecture nodes, so that the corresponding heterogeneous instruction set architecture nodes create container instances in parallel, run the corresponding dedicated test codes, obtain the test results of the application to be tested on the corresponding heterogeneous instruction set architecture nodes, collect and compare the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes, and obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes. In this way, through the application of container technology, efficient and reliable technical support is provided for the cross-instruction set architecture transplantation of software, and it is possible to quickly deploy and automatically manage the test environment and test cases of applications in a heterogeneous instruction set architecture environment, greatly shortening the test cycle, improving the test efficiency, reducing the dependence on the hardware platform, and reducing the construction and maintenance costs of the test platform. By constructing different container images and dedicated test cases adapted to different heterogeneous instruction set architecture nodes, compatibility testing can be conveniently carried out in a heterogeneous instruction set architecture environment, improving the test coverage rate.

[0118] Finally, it should be noted that: Obviously, the above embodiments are only examples for clearly illustrating the present disclosure, rather than limitations on the implementation manners. For those of ordinary skill in the art, other different forms of changes or variations can be made based on the above description. It is not necessary and impossible to enumerate all the implementation manners here. And the obvious changes or variations derived therefrom are still within the protection scope of the present disclosure.

Claims

1. A compatibility testing method, comprising: For the application to be tested, build multiple container images that are compatible with multiple heterogeneous instruction set architecture nodes; Converting the test case template for the application to be tested into a plurality of dedicated test codes respectively adapted to the plurality of heterogeneous instruction set architecture nodes; Scheduling the multiple container images and the multiple dedicated test codes to a corresponding heterogeneous instruction set architecture node, so that the corresponding heterogeneous instruction set architecture node creates a container instance, runs the corresponding dedicated test code, and obtains a test result of the application to be tested on the corresponding heterogeneous instruction set architecture node; The test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes are collected and compared to obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes.

2. The compatibility testing method according to claim 1, wherein: The step of constructing multiple container images adapted to multiple heterogeneous instruction set architecture nodes for the application to be tested includes: Establishing a construction environment for constructing the plurality of container images adapted to the plurality of heterogeneous instruction set architecture nodes; For the application to be tested, establish a configuration file template for building a container image; Using an automated build script, based on the configuration file template, the multiple container images adapted to the multiple heterogeneous instruction set architecture nodes are established.

3. The compatibility testing method according to claim 1, wherein: The converting of the test case template for the application to be tested into a plurality of dedicated test codes respectively adapted to the plurality of heterogeneous instruction set architecture nodes comprises: Establishing a test case template for the application to be tested; Parsing the test case template to obtain test information for compatibility testing of the application to be tested, the test information at least including a test case name, test steps, assertion conditions, and test parameters; Using a test code generator, based on the test information, the plurality of dedicated test codes adapted to the plurality of heterogeneous instruction set architecture nodes are generated.

4. The compatibility testing method according to claim 1, wherein: The step of scheduling the multiple container images and the multiple dedicated test codes to a corresponding heterogeneous instruction set architecture node so that the corresponding heterogeneous instruction set architecture node creates a container instance, runs the corresponding dedicated test code, and obtains a test result of the application to be tested on the corresponding heterogeneous instruction set architecture node includes: For the application to be tested, a test execution task template is established, wherein the test execution task template includes a running environment of the test execution task, configuration information of the container to be tested, and a test case name; Using a scheduler, based on information in the test execution task template, the container image and the dedicated test code adapted to the heterogeneous instruction set architecture node are scheduled to the heterogeneous instruction set architecture node; Among them, the heterogeneous instruction set architecture node uses the container image adapted to the heterogeneous instruction set architecture node to create a container instance of the application to be tested, executes the dedicated test code adapted to the heterogeneous instruction set architecture node, thereby testing the created container instance and obtaining the test result of the application to be tested on the heterogeneous instruction set architecture node.

5. The compatibility testing method according to claim 1, wherein: The collecting and comparing the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes to obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes includes: Define the data structure of the test results, the data structure includes the test case name, instruction set architecture type, Execution status, execution time and test logs; Collecting test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes; The test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes are compared across instruction set architectures, and the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes is obtained based on the comparison results.

6. The compatibility testing method according to claim 5, wherein: The step of comparing the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes across instruction set architectures, and obtaining the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes based on the comparison results, includes: Comparing the program execution time, resource utilization efficiency, and performance bottleneck of the application to be tested on the multiple heterogeneous instruction set architecture nodes, and obtaining the performance compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes based on the comparison results; and / or Comparing the consistency of program execution behaviors of the application to be tested in the multiple heterogeneous instruction set architecture nodes, and the correctness of instruction translation and simulation, and obtaining the instruction set compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes based on the comparison results; and / or Comparing the consistency of system call parameter passing and return values ​​of the application to be tested in the multiple heterogeneous instruction set architecture nodes, and the equivalence of system call behaviors, and obtaining the compatibility of the system call interface of the application to be tested with the multiple heterogeneous instruction set architecture nodes based on the comparison results; and / or The function calling conventions, data type representations, memory alignment requirements, and exception handling mechanisms of the application to be tested in the multiple heterogeneous instruction set architecture nodes are compared, and the application binary interface compatibility of the application to be tested and the multiple heterogeneous instruction set architecture nodes is obtained based on the comparison results.

7. The compatibility testing method according to claim 1, wherein: After collecting and comparing the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes to obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes, the compatibility testing method further includes: Based on the comparison results of the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes, a compatibility test report of the multiple heterogeneous instruction set architecture nodes for the application to be tested is generated.

8. A compatibility testing device, comprising: A container image builder, used to build multiple container images adapted to multiple heterogeneous instruction set architecture nodes for the application to be tested; A test case converter, used for converting the test case template for the application to be tested into a plurality of dedicated test codes respectively adapted to the plurality of heterogeneous instruction set architecture nodes; A distributed test execution engine, configured to schedule the plurality of container images and the plurality of dedicated test codes to a corresponding heterogeneous instruction set architecture node, so that the corresponding heterogeneous instruction set architecture node creates a container instance, runs the corresponding dedicated test code, and obtains a test result of the application to be tested on the corresponding heterogeneous instruction set architecture node; A result comparer is used to collect and compare the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes to obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes.

9. An electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein the program can implement the method according to any one of claims 1 to 7 when executed by the processor.

10. A storage medium having a computer program or instruction stored thereon, wherein the computer program or instruction, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

Images