Compatibility testing methods, related devices and media

Abstract

The present disclosure provides a compatibility testing method, related devices and media. The method includes: constructing multiple container images adapted to multiple heterogeneous instruction set architecture nodes for an application to be tested; converting a test case template for the application to be tested into multiple dedicated test codes adapted to the multiple heterogeneous instruction set architecture nodes respectively; scheduling the multiple container images and the multiple dedicated test codes to the corresponding heterogeneous instruction set architecture nodes, so that the corresponding heterogeneous instruction set architecture nodes create container instances, run the corresponding dedicated test codes, and obtain the test results of the application to be tested on the corresponding heterogeneous instruction set architecture nodes; collecting and comparing the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes, and obtaining the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes, thereby reducing the testing cost and testing cycle, and improving the testing efficiency and test coverage.

Description

Technical Field

[0001] The present disclosure belongs to the field of virtualization technology, and specifically relates to a compatibility testing method, related devices, and media. Background Art

[0002] With the rapid development of cloud computing and edge computing technologies, the instruction set architectures (ISAs) of nodes in data centers are gradually becoming heterogeneous. Heterogeneous ISAs can provide higher performance and lower energy consumption, meeting the diverse workload demands of data centers. Among them, the RISC-V ISA, as an open-source ISA, has been widely adopted in servers and edge devices due to its low power consumption and high performance. The X86 ISA, with its mature ecosystem and extensive software support, holds a dominant position in data centers. In this heterogeneous ISA environment, the importance of implementing software compatibility testing across ISA nodes is becoming increasingly prominent. However, traditional compatibility testing methods rely primarily on building and maintaining a large number of heterogeneous hardware platforms and running the same test cases on each platform. This approach has many problems: 1. High testing costs, as building and maintaining multiple hardware platforms requires a significant capital investment, while hardware resource utilization is relatively low. 2. Long testing cycles, requiring testing on different hardware platforms, consume significant time and fail to meet the rapid iteration requirements of software development. 3. Difficulties in automating the deployment and execution of test cases on different hardware platforms lead to low testing efficiency. 4. Due to hardware platform limitations, it is difficult to cover all target ISAs and software configurations, resulting in insufficient test coverage. Therefore, there is an urgent need to provide a compatibility testing method to address the above issues of testing software compatibility across ISA nodes in a heterogeneous ISA environment. Summary of the Invention

[0003] In view of the above problems, the present disclosure provides a compatibility testing method, related devices and media, aiming to reduce the testing cost and testing cycle of testing software compatibility across instruction set architecture nodes in a heterogeneous instruction set architecture environment, and improve testing efficiency and test coverage.

[0004] According to a first aspect of the present disclosure, a compatibility testing method is provided, comprising:

[0005] For the application to be tested, build multiple container images that are compatible with multiple heterogeneous instruction set architecture nodes;

[0006] Converting a test case template for the application to be tested into a plurality of dedicated test codes respectively adapted to the plurality of heterogeneous instruction set architecture nodes;

[0007] Dispatching the multiple container images and the multiple dedicated test codes to a corresponding heterogeneous instruction set architecture node, so that the corresponding heterogeneous instruction set architecture node creates a container instance, runs the corresponding dedicated test code, and obtains a test result of the application to be tested on the corresponding heterogeneous instruction set architecture node;

[0008] The test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes are collected and compared to obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes.

[0009] Optionally, the step of constructing multiple container images adapted to multiple heterogeneous instruction set architecture nodes for the application to be tested includes:

[0010] Establishing a construction environment for constructing the plurality of container images adapted to the plurality of heterogeneous instruction set architecture nodes;

[0011] For the application to be tested, establish a configuration file template for building a container image;

[0012] Using an automated build script, based on the configuration file template, the multiple container images adapted to the multiple heterogeneous instruction set architecture nodes are created.

[0013] Optionally, converting the test case template for the application to be tested into a plurality of dedicated test codes respectively adapted to the plurality of heterogeneous instruction set architecture nodes comprises:

[0014] Creating a test case template for the application to be tested;

[0015] Parsing the test case template to obtain test information for compatibility testing of the application to be tested, the test information including at least a test case name, test steps, assertion conditions, and test parameters;

[0016] A test code generator is used to generate the plurality of dedicated test codes adapted to the plurality of heterogeneous instruction set architecture nodes based on the test information.

[0017] Optionally, scheduling the multiple container images and the multiple dedicated test codes to a corresponding heterogeneous instruction set architecture node so that the corresponding heterogeneous instruction set architecture node creates a container instance, runs the corresponding dedicated test code, and obtains a test result of the application to be tested on the corresponding heterogeneous instruction set architecture node includes:

[0018] For the application to be tested, a test execution task template is established, wherein the test execution task template includes the running environment of the test execution task, the configuration information of the container to be tested, and the test case name;

[0019] Using a scheduler, based on information in the test execution task template, scheduling a container image and dedicated test code adapted to the heterogeneous instruction set architecture node to the heterogeneous instruction set architecture node;

[0020] Among them, the heterogeneous instruction set architecture node uses the container image adapted to the heterogeneous instruction set architecture node to create a container instance of the application to be tested, executes the dedicated test code adapted to the heterogeneous instruction set architecture node, thereby testing the created container instance and obtaining the test result of the application to be tested on the heterogeneous instruction set architecture node.

[0021] Optionally, collecting and comparing test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes to obtain compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes includes:

[0022] Define the data structure of the test results, which includes the test case name, instruction set architecture type,

[0023] Execution status, execution time, and test logs;

[0024] Collecting test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes;

[0025] The test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes are compared across instruction set architectures, and the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes is obtained based on the comparison results.

[0026] Optionally, performing a cross-instruction set architecture comparison on the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes, and obtaining the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes based on the comparison results, includes:

[0027] Comparing the program execution time, resource utilization efficiency, and performance bottlenecks of the application to be tested on the multiple heterogeneous instruction set architecture nodes, and obtaining the performance compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes based on the comparison results; and / or

[0028] Comparing the consistency of program execution behavior of the application to be tested on the multiple heterogeneous instruction set architecture nodes, as well as the correctness of instruction translation and simulation, and obtaining the instruction set compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes based on the comparison results; and / or

[0029] Comparing the consistency of system call parameter passing and return values ​​of the application to be tested on the multiple heterogeneous instruction set architecture nodes, and the equivalence of system call behaviors, and obtaining the system call interface compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes based on the comparison results; and / or

[0030] The function calling conventions, data type representations, memory alignment requirements, and exception handling mechanisms of the application to be tested in the multiple heterogeneous instruction set architecture nodes are compared, and the application binary interface compatibility of the application to be tested and the multiple heterogeneous instruction set architecture nodes is obtained based on the comparison results.

[0031] Optionally, after collecting and comparing the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes to obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes, the compatibility testing method further includes:

[0032] Based on the comparison results of the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes, a compatibility test report of the multiple heterogeneous instruction set architecture nodes for the application to be tested is generated.

[0033] According to a second aspect of the present disclosure, a compatibility testing device is provided, comprising:

[0034] A container image builder, used to build multiple container images adapted to multiple heterogeneous instruction set architecture nodes for the application to be tested;

[0035] A test case converter, configured to convert a test case template for the application to be tested into a plurality of dedicated test codes respectively adapted to the plurality of heterogeneous instruction set architecture nodes;

[0036] A distributed test execution engine is configured to schedule the multiple container images and the multiple dedicated test codes to corresponding heterogeneous instruction set architecture nodes, so that the corresponding heterogeneous instruction set architecture nodes create container instances, run the corresponding dedicated test codes, and obtain test results of the application to be tested on the corresponding heterogeneous instruction set architecture nodes;

[0037] A result comparer is used to collect and compare the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes to obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes.

[0038] According to a third aspect of the fundamental disclosure, an electronic device is provided, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein the program can implement the method described above when executed by the processor.

[0039] According to a fourth aspect of the fundamental disclosure, a storage medium is provided, on which a computer program or instruction is stored, and when the computer program or instruction is executed by a processor, the steps of any of the methods described above are implemented.

[0040] The present disclosure brings the following beneficial effects:

[0041] The compatibility testing method provided by the present disclosure constructs multiple container images adapted to multiple heterogeneous instruction set architecture nodes for the application to be tested, converts the test case template for the application to be tested into multiple dedicated test codes adapted to the multiple heterogeneous instruction set architecture nodes respectively, schedules the multiple container images and the multiple dedicated test codes to the corresponding heterogeneous instruction set architecture nodes, so that the corresponding heterogeneous instruction set architecture nodes create container instances in parallel, run the corresponding dedicated test codes, obtain the test results of the application to be tested on the corresponding heterogeneous instruction set architecture nodes, collect and compare the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes, and obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes. In this way, through the application of container technology, efficient and reliable technical support is provided for the cross-instruction set architecture porting of software, and it can realize the rapid deployment and automated management of the application test environment and test cases in the heterogeneous instruction set architecture environment, significantly shortening the test cycle, improving the test efficiency, reducing the dependence on the hardware platform, and reducing the construction and maintenance costs of the test platform. By building different container images and dedicated test codes that are compatible with different heterogeneous instruction set architecture nodes, compatibility testing can be easily performed in a heterogeneous instruction set architecture environment, thereby improving test coverage.

[0042] Other features and advantages of the present disclosure will be described in the following description, and in part will become apparent from the description, or understood by practicing the present disclosure. The objectives and other advantages of the present disclosure are realized and obtained by the structures particularly pointed out in the description and the drawings.

[0043] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0045] Figure 1 is a structural diagram of a data center to which an embodiment of the present disclosure is applied;

[0046] Figure 2 is an internal structure diagram of a working node according to one embodiment of the present disclosure;

[0047] Figure 3 A flow chart showing a compatibility testing method according to an embodiment of the present disclosure is shown;

[0048] Figure 4 A schematic structural diagram of a compatibility testing device according to an embodiment of the present disclosure is shown;

[0049] Figure 5 A schematic structural diagram of an electronic device provided according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0050] Various embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. In each of the drawings, identical elements are represented by identical or similar reference numerals. For clarity, the various parts in the drawings are not drawn to scale.

[0051] The following terms are used in this article:

[0052] Containers, as a lightweight virtualization technology, are a group of resource-constrained, isolated processes. Container technology creates independent runtime environments for different applications, enabling resource isolation, configuration, and security. This allows applications to meet on-demand resource allocation requirements while ensuring isolation and availability. Applications hosted in containers are called container instances, or containerized applications. To meet the demands of large-scale applications, it is often necessary to deploy many containers in a computer cluster for unified management and external service provisioning. This requires container orchestration tools. Container orchestration tools consume container services and orchestrate how they interact with each other, extending lifecycle management capabilities to complex, multi-container workloads deployed across large computer clusters. This provides an abstraction layer for developers and infrastructure teams to handle large-scale containerized deployments. An example of a container orchestration tool is Kubernetes (a system for running and coordinating containerized application processes), an open source system for automating the deployment, scaling, and management of containerized applications. Docker is an open source application container engine and a tool for starting and stopping containers. It allows developers to package their applications and dependency packages into a portable container and then publish it to any popular machine. It can also achieve virtualization. The containers completely use the sandbox mechanism and there are no interfaces between them.

[0053] A pod is the basic unit of operation for container orchestration tools (such as Kubernetes) and the smallest deployment unit that can be created, debugged, and managed. All containers in the same pod share the same IP address, IPC, host name, and other resources. A pod abstracts networking and storage from the underlying containers, making it easier to move containers around the cluster. Each pod can contain one or more containers that host applications. The containers in a pod are scheduled as a group to run on a worker node.

[0054] Compatibility testing is to verify the degree of dependence of the software under test on the heterogeneous instruction set architecture environment it relies on, including the degree of dependence on the hardware platform and software platform, that is, to detect the portability of the software between heterogeneous instruction set architecture nodes.

[0055] Figure 1 This is a structural diagram of a data center used in one embodiment of the present disclosure. Figure 1 For the convenience of description, Figure 1 Only the control node 110 and the working node 120 are shown in FIG. In implementation, the control node 110 or the working node 120 can be a server or a virtual machine on the server. Figure 1 Only a limited number of worker nodes 120 are shown, but the present disclosure should not be limited thereto. In some embodiments, the worker nodes 120 are heterogeneous instruction set architecture nodes, including worker node A 220 and worker node B 220, wherein worker node A 220 supports the RISC-V instruction set architecture and worker node B 220 supports the X86 instruction set architecture. It is understandable that the worker nodes 120 may also support other instruction set architectures such as the ARM instruction set architecture. For the sake of convenience, Figure 1 Not shown in the figure.

[0056] In some embodiments, the data center can be applied to various application scenarios such as content delivery network (CDN), e-commerce, games, audio and video, Internet of Things, logistics, industrial brain, city brain, etc., and provide computing services to end users in various scenarios. Specifically, for each application scenario, an application that can provide computing services in the application scenario can be deployed in the working node 120 of the data center. Taking into account that a large number of applications may need to be deployed in the data center, container technology can be used to build a container group on the working node 120, carry the application through the container, and then deploy the application in units of containers. By running these container instances deployed on the working node 120, corresponding computing services can be provided to end users. It should be noted that for the working node A 220, the container deployed thereon supports the RISC-V instruction set architecture. For the working node B 220, the container deployed thereon supports the X86 instruction set architecture.

[0057] In some embodiments, the control node 110 can run a container orchestration tool (e.g., Kubernetes) to orchestrate and manage container instances in its data center. This orchestration and management of container instances includes at least one of the following: container instance creation, elastic scaling, rolling updates, rebuilds, migrations, and shutdowns. In some embodiments, in addition to container orchestration and management, the control node 110 can also be responsible for other management operations within its data center, such as operations and maintenance, log monitoring, and network status monitoring and management. Commands can be sent to each worker node 120 via the control node 110. Simply put, the control node 110 is the manager, and the worker nodes 120 are the managed. The backend services running on the control node 110 typically include an application programming interface (API) server 111 and a scheduler 130. The API server 111 serves as the front-end interface for the container orchestration tool (e.g., Kubernetes). Various client tools and other components of the container orchestration tool (e.g., Kubernetes) can manage various resources in the data center cluster through the API server 111. The scheduler 130 may determine which worker node to place the container group on and determine the resources to allocate to the container group.

[0058] In some embodiments, since the working node 120 is the real processing device of the data center, Figure 2The internal structure of a worker node according to one embodiment of the present disclosure is shown. In some embodiments, a worker node 120 may include multiple processors 22 and a memory 29. The memory 29 in the worker node 120 may be a main memory (abbreviated as main memory or internal memory), which is used to store instruction information and / or data information represented by data signals, such as data provided by the processor 22 (e.g., calculation results). It may also be used to facilitate data exchange between the processor 22 and an external storage device 26 (also referred to as auxiliary memory or external memory).

[0059] In some situations, processor 22 may need to access memory 29 to retrieve or modify data in memory 29. Because memory 29 has a relatively slow access speed, to mitigate the speed gap between processor 22 and memory 29, worker node 120 also includes a cache memory 28 coupled to bus 21. Cache memory 28 is used to cache data in memory 29, such as program data or message data, that may be repeatedly accessed. Cache memory 28 is implemented, for example, by a storage device such as static random access memory (SRAM). Cache memory 28 can have a multi-level structure, such as a three-level cache structure with a first-level cache (L1 cache), a second-level cache (L2 cache), and a third-level cache (L3 cache). It can also have a cache structure with more than three levels or other types of cache structures. In some embodiments, a portion of cache memory 28 (e.g., the first-level cache, or both the first-level cache and the second-level cache) can be integrated within processor 22 or integrated into the same system-on-chip as processor 22.

[0060] Based on this, the processor 22 may include an instruction execution unit 221, a memory management unit 222, and other components. When executing instructions that require memory modification, the instruction execution unit 221 initiates a write access request, which specifies the write data to be written into the memory and the corresponding physical address. The memory management unit 222 is configured to translate the virtual addresses specified by these instructions into physical addresses mapped to the virtual addresses. The physical addresses specified by the write access request may be consistent with the physical addresses specified by the corresponding instructions.

[0061] Information exchange between memory 29 and cache memory 28 is typically organized in blocks. In some embodiments, cache memory 28 and memory 29 can be divided into data blocks of the same spatial size. A data block can serve as the minimum unit of data exchange between cache memory 28 and memory 29 (including one or more data of a predetermined length). For the sake of brevity and clarity, each data block in cache memory 28 will be referred to as a cache block (which may also be referred to as a cache line), with different cache blocks having different cache block addresses. Each data block in memory 29 will be referred to as a memory block, with different memory blocks having different memory block addresses. A cache block address, for example, includes a physical address tag used to locate the data block.

[0062] Due to space and resource constraints, cache memory 28 cannot cache all the contents of memory 29. Specifically, the storage capacity of cache memory 28 is typically smaller than that of memory 29, and the cache block addresses provided by cache memory 28 cannot correspond to all the memory block addresses provided by memory 29. When processor 22 needs to access memory, it first accesses cache memory 28 via bus 21 to determine whether the content it wants to access is already stored in cache memory 28. If so, it is a cache hit, and processor 22 directly retrieves the content from cache memory 28. If the content processor 22 wants to access is not in cache memory 28, processor 22 must access memory 29 via bus 21 to search for the corresponding information. Because cache memory 28 has a very fast access rate, a cache hit significantly improves the efficiency of processor 22, thereby enhancing the performance and efficiency of the entire work node 120.

[0063] As shown in the figure, the processor 22, cache memory 28, and memory 29 are packaged in a system on chip (SoC) 201. Designers can configure the SoC architecture so that the communication between the various components in the working node 120 is secure.

[0064] In this example, the working node 120 may also include various software. In some embodiments, such as Figure 2As shown, an operating system 202, container support 203 and container group 204 are provided on the underlying hardware (i.e., system on chip 201). The operating system 202 is, for example, an operating system that can be used on a server, such as a UNIX operating system or a Linux operating system. The working node 120 supports the RISC-V instruction set architecture or the X86 instruction set architecture or other instruction set architectures. The container support 203 is the various underlying implementations required to support the containers running thereon. For example, for docker, the container support 203 needs to implement two technologies, cgroup (short for controlgroups, control group) and namespace (namespace). Cgroup implements resource limits, and namespace implements resource isolation. Docker allows developers to package their applications and dependent operating environments into a portable container, and then publish it to the working node 120. As Figure 2 As shown, based on container support 203, multiple container groups 204 can be run, and an operating environment and running applications can be implemented in each container of container group 204. The container technology is used to achieve isolation and mutual non-interference between the various applications. The applications may include, but are not limited to, programs for controlling or responding to external devices (e.g., biometric sensors, printers, microphones, speakers, flow valves, or other I / O components, sensors, actuators, or devices), programs for various I / O tasks, security programs, authentication programs, various computing modules, communication programs, communication support protocols, or other programs, or a combination thereof.

[0065] In some embodiments, as Figure 2 As shown, there is also a

[0066] The container orchestration tool (e.g., K8s) has a service agent (Kubelet) 205 on the worker node 120. In implementation, the service agent (Kubelet) 205 can be a software program module or hardware, for example, based on an FPGA or CPLD. In some embodiments, the service agent 205 can receive and execute instructions from the control node 110 to manage the container group and the containers within it. The service agent 205 can register information about the worker node 120 on the application program interface server 111 of the control node 110, monitor the resource usage of the worker node 120, and periodically report resource usage data for the worker node 120 to the control node 110. In some embodiments, a garbage collection (GC) module (not shown) can also be provided on the underlying hardware (i.e., the system-on-chip 201). In implementation, the garbage collection (GC) module can be a software program module or hardware, for example, based on an FPGA or CPLD. In some embodiments, the garbage collection (GC) module can establish a garbage collection process for each container and clear the container group's configuration information when the container group is destroyed.

[0067] In addition, the working node 120 may also include hardware devices such as a display device 23, an audio device 24, an input / output device 25, a storage device 26, and a communication device 27. Of course, the structure of different computer systems may vary depending on the motherboard, operating system, and instruction set architecture.

[0068] Reference again Figure 1 ,like Figure 1As shown, the backend services running on the control node 110 may also include a compatibility testing device 140. By applying container technology, the compatibility testing device 140 can provide efficient and reliable technical support for cross-ISA software porting. This enables rapid deployment and automated management of application test environments and test cases in heterogeneous ISA environments, and tests the compatibility of applications with multiple HISA nodes (including worker node A 220 and worker node B 220). Specifically, the compatibility testing device 140 includes a container image builder 141, a test case converter 142, a distributed test execution engine 143, a result comparer 144, a test report generator 145, a test management layer 146, and a user interface layer 147. In some embodiments, the user interface layer 147 can provide a test task configuration interface for users to configure compatibility test tasks for the application under test, display test execution status, and present test results and reports. In some embodiments, the test management layer 146 may receive test tasks configured in the user interface layer 147 , schedule the test tasks, and submit the test tasks to the container image builder 141 , the test case converter 142 , and the distributed test execution engine 143 .

[0069] In some embodiments, the container image builder 141 builds multiple container images for the application under test that are compatible with multiple heterogeneous instruction set architecture nodes (including worker node A 220 and worker node B 220). In some embodiments, a build environment is established for building multiple container images compatible with multiple heterogeneous instruction set architecture nodes. A configuration file (Dockerfile) template is created for the application under test to build the container images. An automated build script is used to build the multiple container images compatible with the multiple heterogeneous instruction set architecture nodes based on the configuration file template. In some embodiments, an application container engine (Docker) and a container image building tool (BuildKit) can be installed on the control node 110 to prepare for the subsequent construction of container images across different instruction set architecture nodes. Based on BuildKit, a new builder instance can be created and used using the "docker buildxcreate" function. In some embodiments, a multi-stage build approach can be used to build the Dockerfile template for building container images across different instruction set architecture nodes. It should be noted that the Dockerfile template is centrally managed and modified on the control node 110, without requiring modification on the worker nodes 120. The multi-platform build feature of DockerBuildX automatically adapts to different instruction set architectures. It should be noted that the automated build script for creating a container image across instruction set architectures is executed only on the control node 110, and does not need to be executed separately on each heterogeneous instruction set architecture node. The build task of the container image across instruction set architectures is managed on the control node 110 through the multi-platform build feature of Docker BuildX. The QEMU emulator can be used to implement the build task of the container image across instruction set architectures.

[0070] In some embodiments, the test case converter 142 converts a test case template for the application to be tested into multiple dedicated test codes that are respectively compatible with multiple heterogeneous instruction set architecture nodes. In some embodiments, a test case template is established for the application to be tested. The test case template is parsed to obtain test information for compatibility testing of the application to be tested. A test code generator is used to generate multiple dedicated test codes that are compatible with multiple heterogeneous instruction set architecture nodes based on the test information. It should be noted that the test case template uses a structured approach to describe test steps. The test case template is a predefined format for recording detailed test case information. The test information included in the test case template includes the test case name, test steps, assertions, and test parameters. The test steps are a list, each of which has an action, an element, and the corresponding value. For example, an action can be click, input, etc. The element is the identifier of the page element to be operated, which can be a button (button#submit) or an input box (input#username). The value corresponding to an element can be the value of an input action, for example, the value of an input box. Assertion conditions (assertions) are a list containing assertions to be checked after the test is completed. Each assertion has a type (type) and an element (element). The type can be the type of assertion to be checked, such as element presence (element_present), etc. The element is the identifier of the page element to be checked, such as a message box (div.success-message), which checks whether the operation success message appears. In some embodiments, the "def parse_test_case(yaml_file)" function is used to parse the YAML format test case template to obtain test information for compatibility testing of the application to be tested. In some embodiments, the "def generate_test_code(test_case, architecture)" function is used to generate multiple dedicated test codes adapted to multiple heterogeneous instruction set architecture nodes based on the test case template (test_case) and the target instruction set architecture (architecture). It is understandable that the test code generator is deployed in the control node 110 and converts the test case template (test_case) into a dedicated test code adapted to the target instruction set architecture (architecture) through the “def generate_test_code(test_case, architecture)” function.For example, if the target instruction set architecture is "x86_64," "generate_x86_code(test_case)" is returned, which generates specialized test code adapted to the "x86_64" instruction set architecture. It is understood that the test code generator can convert a generic test case template into specialized test code adapted to various heterogeneous instruction set architectures.

[0071] In some embodiments, the distributed test execution engine 143 dispatches multiple container images and multiple dedicated test codes to the corresponding heterogeneous instruction set architecture node, so that the corresponding heterogeneous instruction set architecture node creates a container instance, runs the corresponding dedicated test code, and obtains the test results of the application to be tested on the corresponding heterogeneous instruction set architecture node. In some embodiments, a test execution task (Job) template is established for the application to be tested. The test execution Job template includes the running environment of the test execution task, the configuration information of the container to be tested, and the test case name. In some embodiments, as Figure 1The data center shown can also be called a Kubernetes cluster. Kubeadm or a managed Kubernetes service from a cloud provider can be used. The test execution job template created is a Kubernetes job template. As you can understand, in Kubernetes, a job is a controller responsible for creating one or more pods to perform a task (a job) and then stopping these pods after the task is completed. In other words, a job is used to run a limited number of tasks or batch processing tasks. The test execution job template includes information about the Kubernetes API version (apiVersion), the type of Kubernetes resource to be created (kind), and resource metadata (metadata). apiVersion: batch / v1 indicates that the batch / v1 version of the Kubernetes API is being used. This version of the API includes some batch-related resource types, such as Job. kind: Job indicates that the resource type being created is a Job. Metadata stores metadata about the job, such as the job name, job specification (spec), labels, and annotations. The job specification (spec) includes configuration information for the container and container image to be tested, as well as information about the command (i.e., test-specific code) to run after the container is started. In some embodiments, a scheduler is used to schedule a container image and dedicated test code that are compatible with a heterogeneous instruction set architecture node to the heterogeneous instruction set architecture node based on the information in the test execution job template. In some embodiments, "def schedule_test(test_name, test_image)" is used to schedule a container image and dedicated test code that are compatible with a heterogeneous instruction set architecture node to a heterogeneous instruction set architecture node. For example, the implementation code for using a scheduler to perform test scheduling to schedule a container image and dedicated test code that are compatible with a heterogeneous instruction set architecture node to a heterogeneous instruction set architecture node is:

[0072] from kubernetes import client, config

[0073] def schedule_test(test_name, test_image):

[0074] """

[0075] Schedule test tasks to the Kubernetes cluster for execution

[0076] parameter:

[0077] test_name: test task name

[0078] test_image: test container image

[0079] """

[0080] # Load Kubernetes configuration

[0081] config.load_kube_config()

[0082] # Create a BatchV1Api client

[0083] batch_v1 = client.BatchV1Api()

[0084] # Read the Job template file

[0085] with open("job_template.yaml") as f:

[0086] job_manifest = yaml.safe_load(f)

[0087] # Replace variables in templates

[0088] job_manifest['metadata']['name']= f"test-job-{test_name}"

[0089] job_manifest['spec']['template']['spec']['containers'][0]['image']=test_image

[0090] # Create a Job in the default namespace

[0091] batch_v1.create_namespaced_job(

[0092] namespace="default",

[0093] body=job_manifest )

[0095] In some embodiments, the step of using a scheduler to perform test scheduling includes receiving a test request, checking the resource availability of the work node 120, selecting a target work node 120, creating a test execution job at the target work node 120, setting the container image to be tested at the target work node 120 according to the information in the test execution job, and allocating resources for the container. After the container is started, the target work node 120 runs a dedicated test code adapted to the instruction set architecture supported by the target work node 120 on the container, and monitors the test execution process. It is understandable that, at different heterogeneous instruction set architecture nodes, container instances of the application to be tested can be created in parallel using container images adapted to the heterogeneous instruction set architecture nodes, and dedicated test codes adapted to the heterogeneous instruction set architecture nodes can be executed, thereby testing the created container instances and obtaining test results of the application to be tested at the heterogeneous instruction set architecture nodes.

[0096] In some embodiments, the result comparer 144 collects and compares the test results of the application to be tested on multiple heterogeneous instruction set architecture nodes to obtain the compatibility of the application to be tested with multiple heterogeneous instruction set architecture nodes (for example, RISC-V instruction set architecture nodes, X86 instruction set architecture nodes, and ARM instruction set architecture nodes, etc.). In some embodiments, the data structure of the test results is defined. For example, the data structure includes the test case name (test_name), the instruction set architecture type (architecture), the execution status (status, for example, success / failure), the execution time (execution_time), and the test log (logs). The test results of the application to be tested on multiple heterogeneous instruction set architecture nodes are collected. The test results of the application to be tested on multiple heterogeneous instruction set architecture nodes are compared across instruction set architectures, and the compatibility of the application to be tested with multiple heterogeneous instruction set architecture nodes is obtained based on the comparison results.

[0097] In some embodiments, the compatibility of the application to be tested with multiple heterogeneous instruction set architecture nodes includes instruction set compatibility. The consistency of the program execution behavior of the application to be tested on multiple heterogeneous instruction set architecture nodes, as well as the correctness of instruction translation and simulation, can be compared. In this way, the instruction set compatibility of the application to be tested with multiple heterogeneous instruction set architecture nodes is obtained based on the comparison results. For example, if the program execution behavior of the application to be tested on one instruction set architecture node (e.g., an ARM instruction set architecture node) is inconsistent with that on other instruction set architecture nodes (e.g., a RISC-V instruction set architecture node and an X86 instruction set architecture node), then the application to be tested is incompatible with the instruction set of the instruction set architecture node. For example, if the instruction translation and simulation of the application to be tested on an instruction set architecture node (e.g., an X86 instruction set architecture node) are incorrect, then the application to be tested is incompatible with the instruction set of the instruction set architecture node.

[0098] In some embodiments, the compatibility of the application under test with multiple heterogeneous ISA nodes includes system call interface compatibility. The consistency of the system call parameter passing and return values, as well as the equivalence of system call behaviors, of the application under test on the multiple heterogeneous ISA nodes can be compared. Based on the comparison results, the system call interface compatibility of the application under test with the multiple heterogeneous ISA nodes can be determined. For example, if the system call parameter passing and return values ​​of the application under test on one ISA node (e.g., an ARM ISA node) are inconsistent with those on other ISA nodes (e.g., a RISC-V ISA node and an X86 ISA node), then the application under test is incompatible with the system call interface of the ISA node. For example, if the system call behavior of the application under test on one ISA node (e.g., an ARM ISA node) is not equivalent to that on other ISA nodes (e.g., a RISC-V ISA node and an X86 ISA node), then the application under test is incompatible with the system call interface of the ISA node.

[0099] In some embodiments, the compatibility of the application to be tested with multiple heterogeneous instruction set architecture nodes includes ABI (Application Binary Interface) compatibility. The function calling conventions, data type representations, memory alignment requirements, exception handling mechanisms, etc. of the application to be tested in multiple heterogeneous instruction set architecture nodes can be compared, so that the ABI compatibility of the application to be tested with multiple heterogeneous instruction set architecture nodes can be obtained based on the comparison results. For example, if the function calling conventions or data type representations or memory alignment requirements or exception handling mechanisms of the application to be tested in one instruction set architecture node (such as an ARM instruction set architecture node) are different from those in other instruction set architecture nodes (such as a RISC-V instruction set architecture node and an X86 instruction set architecture node), then the application to be tested is incompatible with the ABI of the instruction set architecture node.

[0100] In some embodiments, the compatibility of the application under test with multiple heterogeneous instruction set architecture nodes includes performance compatibility. The program execution time, resource utilization efficiency, and performance bottlenecks of the application under test on multiple heterogeneous instruction set architecture nodes can be compared. Based on the comparison results, the performance compatibility of the application under test with the multiple heterogeneous instruction set architecture nodes can be determined. For example, if the program execution time of the application under test on one instruction set architecture node (e.g., an ARM instruction set architecture node) is significantly longer than that on other instruction set architecture nodes (e.g., a RISC-V instruction set architecture node and an X86 instruction set architecture node), then the application under test is performance-incompatible with the instruction set architecture node. For example, if the resource utilization efficiency of the application under test on one instruction set architecture node (e.g., an ARM instruction set architecture node) is significantly worse than that on other instruction set architecture nodes (e.g., a RISC-V instruction set architecture node and an X86 instruction set architecture node), then the application under test is performance-incompatible with the instruction set architecture node. For example, if the application under test reaches a performance bottleneck in one instruction set architecture node (e.g., an ARM instruction set architecture node) compared to other instruction set architecture nodes (e.g., a RISC-V instruction set architecture node and an X86 instruction set architecture node), then the application under test is incompatible with the performance of the instruction set architecture node.

[0101] In some embodiments, the test report generator 145 generates compatibility test reports for multiple heterogeneous instruction set architecture nodes for the application to be tested based on the comparison results of the test results of the application to be tested on multiple heterogeneous instruction set architecture nodes. In some embodiments, a test report template for the test results is established. Based on the comparison results of the test results of the application to be tested on multiple heterogeneous instruction set architecture nodes, a test report generator is used to generate compatibility test reports for multiple heterogeneous instruction set architecture nodes for the application to be tested. It should be noted that the compatibility testing method implemented by the compatibility testing device 140 in the embodiment of the present disclosure is executed by pipeline control designed by a CI / CD (continuous integration / continuous deployment) tool. In some embodiments, in the software development practice of the application to be tested, developers often merge code changes into the main branch, usually multiple times a day. Each merge runs automated tests through a pipeline designed by CI / CD tools, covering the entire process from building container images across ISA nodes, converting test case templates into specialized test cases adapted for multiple heterogeneous ISA nodes, executing test cases in parallel on corresponding container instances across multiple heterogeneous ISA nodes, collecting and comparing test results, and generating a final test report. It can be understood that the embodiments of the present disclosure fully consider the needs of software compatibility testing in a cross-ISA environment, and achieve an efficient and reliable testing process through container technology and an automated tool chain.

[0102] Figure 3 FIG. 1 is a flow chart showing a compatibility testing method according to an embodiment of the present disclosure. Figure 3 As shown, the compatibility testing method of the embodiment of the present disclosure may include:

[0103] In step S310, a plurality of container images adapted to a plurality of heterogeneous instruction set architecture nodes are constructed for the application to be tested.

[0104] In step S320, the test case template for the application to be tested is converted into a plurality of dedicated test codes respectively adapted to the plurality of heterogeneous instruction set architecture nodes.

[0105] In step S330, the multiple container images and the multiple dedicated test codes are scheduled to the corresponding heterogeneous instruction set architecture node, so that the corresponding heterogeneous instruction set architecture node creates a container instance, runs the corresponding dedicated test code, and obtains the test result of the application to be tested on the corresponding heterogeneous instruction set architecture node.

[0106] In step S340, test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes are collected and compared to obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes.

[0107] Since the process of using the compatibility testing method of the embodiment of the present disclosure to test the software compatibility of cross-instruction set architecture nodes in a heterogeneous instruction set architecture environment has been described in detail in the device embodiment above, it will not be repeated here.

[0108] Figure 4 FIG. 1 is a schematic diagram showing a compatibility testing device provided according to an embodiment of the present disclosure. Figure 4 As shown, the compatibility testing apparatus 400 includes a container image builder 410 , a test case converter 420 , a distributed test execution engine 430 , and a result comparer 440 .

[0109] The container image builder 410 is used to build multiple container images that are compatible with multiple heterogeneous instruction set architecture nodes for the application to be tested.

[0110] The test case converter 420 is used to convert the test case template for the application to be tested into multiple dedicated test codes that are respectively adapted to the multiple heterogeneous instruction set architecture nodes.

[0111] The distributed test execution engine 430 is used to schedule the multiple container images and the multiple dedicated test codes to the corresponding heterogeneous instruction set architecture node, so that the corresponding heterogeneous instruction set architecture node creates a container instance, runs the corresponding dedicated test code, and obtains the test result of the application to be tested on the corresponding heterogeneous instruction set architecture node.

[0112] The result comparer 440 is used to collect and compare the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes to obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes.

[0113] Since the process of using the compatibility testing method of the embodiment of the present disclosure to test the software compatibility of cross-instruction set architecture nodes in a heterogeneous instruction set architecture environment has been described in detail in the device embodiment above, it will not be repeated here.

[0114] The present disclosure also provides an electronic device, such as Figure 5 As shown, it includes a memory 520, a processor 510, and a program stored in the memory 520 and executable on the processor 510. When the program is executed by the processor 510, the various processes of the various embodiments of the above-mentioned compatibility testing method can be implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0115] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be accomplished through instructions, or by controlling related hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, the present disclosure also provides a storage medium having a computer program or instructions stored thereon. When the computer program or instructions are executed by the processor, the various processes of the various embodiments of the compatibility testing method described above can be implemented.

[0116] Since the instructions stored in the storage medium can execute the steps of the compatibility testing method provided in the embodiment of the present disclosure, the beneficial effects that can be achieved by the compatibility testing method provided in the embodiment of the present disclosure can be achieved. Please refer to the previous embodiment for details and will not be repeated here. The specific implementation of each of the above operations can be referred to the previous embodiment and will not be repeated here.

[0117] In summary, the compatibility testing method provided by the present invention constructs multiple container images adapted to multiple heterogeneous instruction set architecture nodes for the application to be tested, converts the test case template for the application to be tested into multiple dedicated test codes adapted to the multiple heterogeneous instruction set architecture nodes respectively, schedules the multiple container images and the multiple dedicated test codes to the corresponding heterogeneous instruction set architecture nodes, so that the corresponding heterogeneous instruction set architecture nodes create container instances in parallel, run the corresponding dedicated test codes, obtain the test results of the application to be tested on the corresponding heterogeneous instruction set architecture nodes, collect and compare the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes, and obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes. In this way, through the application of container technology, efficient and reliable technical support is provided for the cross-instruction set architecture porting of software, and it can realize the rapid deployment and automated management of the test environment and test cases of the application in the heterogeneous instruction set architecture environment, greatly shortening the test cycle, improving the test efficiency, and reducing the dependence on the hardware platform, reducing the construction and maintenance cost of the test platform. By building different container images and dedicated test cases that are compatible with different heterogeneous instruction set architecture nodes, compatibility testing can be easily performed in a heterogeneous instruction set architecture environment, thereby improving test coverage.

[0118] Finally, it should be noted that the above embodiments are merely examples for the purpose of illustrating the present disclosure and are not intended to limit the embodiments. Those skilled in the art will readily appreciate that other variations or modifications based on the above descriptions are possible. It is not necessary and impossible to provide an exhaustive list of all embodiments. Obvious variations or modifications arising therefrom remain within the scope of protection of the present disclosure.

Claims

1. A compatibility testing method, comprising: For the application to be tested, build multiple container images that are compatible with multiple heterogeneous instruction set architecture nodes; Converting a test case template for the application to be tested into a plurality of dedicated test codes respectively adapted to the plurality of heterogeneous instruction set architecture nodes, including establishing a test case template for the application to be tested; parsing the test case template to obtain test information for performing a compatibility test on the application to be tested, the test information including at least a test case name, test steps, assertion conditions, and test parameters; Using a test code generator, based on the test information, to generate the plurality of dedicated test codes adapted to the plurality of heterogeneous instruction set architecture nodes; Dispatching the multiple container images and the multiple dedicated test codes to corresponding heterogeneous instruction set architecture nodes, so that the corresponding heterogeneous instruction set architecture nodes create container instances, run the corresponding dedicated test codes, and obtain test results of the application to be tested on the corresponding heterogeneous instruction set architecture nodes, including establishing a test execution task template for the application to be tested, the test execution task template including a running environment of the test execution task, configuration information of the container to be tested, and a test case name; Using a scheduler, based on the information in the test execution task template, the container image and the dedicated test code adapted to the heterogeneous instruction set architecture node are scheduled to the heterogeneous instruction set architecture node; wherein the heterogeneous instruction set architecture node uses the container image adapted to the heterogeneous instruction set architecture node to create a container instance of the application to be tested, executes the dedicated test code adapted to the heterogeneous instruction set architecture node, thereby testing the created container instance, and obtaining a test result of the application to be tested on the heterogeneous instruction set architecture node; The test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes are collected and compared to obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes.

2. The compatibility testing method according to claim 1, wherein: The step of constructing multiple container images adapted to multiple heterogeneous instruction set architecture nodes for the application to be tested includes: Establishing a construction environment for constructing the plurality of container images adapted to the plurality of heterogeneous instruction set architecture nodes; For the application to be tested, establish a configuration file template for building a container image; Using an automated build script, based on the configuration file template, the multiple container images adapted to the multiple heterogeneous instruction set architecture nodes are created.

3. The compatibility testing method according to claim 1, wherein: The collecting and comparing the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes to obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes includes: Define the data structure of the test results, which includes the test case name, instruction set architecture type, Execution status, execution time, and test logs; Collecting test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes; The test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes are compared across instruction set architectures, and the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes is obtained based on the comparison results.

4. The compatibility testing method according to claim 3, wherein: The step of comparing the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes across instruction set architectures, and obtaining the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes based on the comparison results, includes: Comparing the program execution time, resource utilization efficiency, and performance bottlenecks of the application to be tested on the multiple heterogeneous instruction set architecture nodes, and obtaining the performance compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes based on the comparison results; and / or Comparing the consistency of program execution behavior of the application to be tested on the multiple heterogeneous instruction set architecture nodes, as well as the correctness of instruction translation and simulation, and obtaining the instruction set compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes based on the comparison results; and / or Comparing the consistency of system call parameter passing and return values ​​of the application to be tested on the multiple heterogeneous instruction set architecture nodes, and the equivalence of system call behaviors, and obtaining the system call interface compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes based on the comparison results; and / or The function calling conventions, data type representations, memory alignment requirements, and exception handling mechanisms of the application to be tested in the multiple heterogeneous instruction set architecture nodes are compared, and the application binary interface compatibility of the application to be tested and the multiple heterogeneous instruction set architecture nodes is obtained based on the comparison results.

5. The compatibility testing method according to claim 1, wherein: After collecting and comparing the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes to obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes, the compatibility testing method further includes: Based on the comparison results of the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes, a compatibility test report of the multiple heterogeneous instruction set architecture nodes for the application to be tested is generated.

6. A compatibility testing device, comprising: A container image builder, used to build multiple container images adapted to multiple heterogeneous instruction set architecture nodes for the application under test; A test case converter, configured to convert a test case template for the application to be tested into a plurality of dedicated test codes respectively adapted to the plurality of heterogeneous instruction set architecture nodes, including establishing a test case template for the application to be tested; parsing the test case template to obtain test information for performing a compatibility test on the application to be tested, the test information including at least a test case name, test steps, assertion conditions, and test parameters; Using a test code generator, based on the test information, to generate the plurality of dedicated test codes adapted to the plurality of heterogeneous instruction set architecture nodes; A distributed test execution engine is configured to schedule the multiple container images and the multiple dedicated test codes to corresponding heterogeneous instruction set architecture nodes, so that the corresponding heterogeneous instruction set architecture nodes create container instances, run the corresponding dedicated test codes, and obtain test results of the application to be tested on the corresponding heterogeneous instruction set architecture nodes, including establishing a test execution task template for the application to be tested, the test execution task template including a running environment for the test execution task, configuration information of the container to be tested, and a test case name; Using a scheduler, based on the information in the test execution task template, the container image and the dedicated test code adapted to the heterogeneous instruction set architecture node are scheduled to the heterogeneous instruction set architecture node; wherein the heterogeneous instruction set architecture node uses the container image adapted to the heterogeneous instruction set architecture node to create a container instance of the application to be tested, executes the dedicated test code adapted to the heterogeneous instruction set architecture node, thereby testing the created container instance, and obtaining a test result of the application to be tested on the heterogeneous instruction set architecture node; A result comparer is used to collect and compare the test results of the application to be tested on the multiple heterogeneous instruction set architecture nodes to obtain the compatibility of the application to be tested with the multiple heterogeneous instruction set architecture nodes.

7. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, can implement the method according to any one of claims 1 to 5.

8. A storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions, when executed by a processor, implements the steps of the method according to any one of claims 1 to 5.

Images