Method and apparatus for testing instruction set ecosystem , and device and readable storage medium

Abstract

Provided in the embodiments of the present application are a method and apparatus for testing an instruction set ecosystem, and a device and a readable storage medium. The method comprises: determining at least one application object associated with an instruction set, wherein the instruction set comprises an instruction set to be tested and a reference instruction set, and the application object is used for operating on the basis of the instruction set to be tested or the reference instruction set; acquiring a first operating parameter when the application object operates on the basis of the instruction set to be tested, and a second operating parameter when the application object operates on the basis of the reference instruction set; on the basis of the first operating parameter and the second operating parameter, determining a parameter result of the application object, wherein the parameter result is used for representing a magnitude relationship between the first operating parameter and the second operating parameter; and on the basis of the parameter result of each application object, determining the ecosystem maturity of the instruction set to be tested. The present invention can determine the applicability of an instruction set to be tested.

Description

Instruction set ecosystem testing methods, apparatus, equipment and readable storage media

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411463336.1, filed on October 18, 2024, entitled "Instruction Set Ecosystem Testing Method, Apparatus, Device and Readable Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of computer technology, and in particular to an instruction set ecosystem testing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0004] Instruction sets are the fundamental language of computers. They specify the operations and commands that a processor can execute, thus defining the tasks and operations that the processor can perform. Different instruction set architectures have different levels of ecosystem maturity. Ecosystem maturity refers to the ecosystem built around the instruction set and its degree of perfection. Instruction sets with high ecosystem maturity can be widely used in various fields; therefore, it is necessary to clearly define the ecosystem maturity of an instruction set.

[0005] However, there is currently no solution to determine the ecological maturity of instruction sets, making it impossible to determine the applicability of instruction sets. Summary of the Invention

[0006] This application provides an instruction set ecosystem testing method, apparatus, electronic device, and readable storage medium, which can solve the problem of inability to determine the applicability of instruction sets in related technologies.

[0007] To address the aforementioned issues, this application discloses an instruction set ecosystem testing method, the method comprising:

[0008] Identify at least one application object associated with an instruction set; the instruction set includes a test instruction set and a reference instruction set; the application object is used to run based on the test instruction set or the reference instruction set;

[0009] Obtain the first running parameters of the application object when running based on the instruction set to be tested, and the second running parameters of the application object when running based on the reference instruction set;

[0010] Based on the first operating parameter and the second operating parameter, the parameter result of the application object is determined; the parameter result is used to represent the magnitude relationship between the first operating parameter and the second operating parameter.

[0011] Based on the parameter results for each application object, the ecological maturity of the instruction set to be tested is determined.

[0012] Optionally, there may be multiple application objects;

[0013] Following the step of determining at least one application object associated with the instruction set, the method further includes:

[0014] Based on the type of each application object, multiple application objects are classified to obtain a combination of objects of at least one type;

[0015] The instruction set's ecosystem is constructed based on at least one type of object combination; the ecosystem includes at least one application layer, with different application layers corresponding to different types of object combinations.

[0016] The ecological structure is used to indicate the order of application objects used when obtaining the first and second operating parameters of at least one application object.

[0017] Optionally, the step of determining the parameter result of the application object based on the first running parameters and the second running parameters includes:

[0018] Determine at least one first sub-parameter in the first running parameters and at least one second sub-parameter in the second running parameters; different first sub-parameters are used to describe different types of running conditions of the application object based on the instruction set to be tested; different second sub-parameters are used to describe different types of running conditions of the application object based on the reference instruction set.

[0019] Based on each first sub-parameter and its corresponding second sub-parameter, determine at least one sub-parameter result;

[0020] The parameter result of the application object is determined based on at least one sub-parameter result and the corresponding preset first weight combination; different sub-parameter results correspond to different weights in the preset first weight combination.

[0021] Optionally, the method further includes:

[0022] For each object combination corresponding to each application level, obtain the parameter results of each application object in the object combination to obtain the parameter result combination of the application level;

[0023] Based on at least one parameter result in the combination of parameter results, the ecological maturity corresponding to the application level is determined.

[0024] Optionally, the step of determining the ecological maturity of the instruction set to be tested based on the parameter results of each application object includes:

[0025] For each application layer, the ecosystem maturity of the application layer is determined based on the parameter results of the application objects in that application layer.

[0026] The ecological maturity of the instruction set is determined based on the ecological maturity of each application level and the corresponding preset second weight combination; different application levels correspond to different weights in the preset second weight combination.

[0027] Optionally, the step of obtaining the first runtime parameters of the application object based on the instruction set under test and the second runtime parameters of the application object based on the reference instruction set includes:

[0028] When the application object runs based on the instruction set to be tested, the first support parameter corresponding to the application object executing the preset test case set is obtained; the first support parameter represents the execution success rate corresponding to the instruction set to be tested.

[0029] When the application object runs based on the reference instruction set, obtain the second support parameter corresponding to the application object after executing the test case set; the second support parameter represents the execution success rate corresponding to the reference instruction set.

[0030] Optionally, the step of obtaining the first runtime parameters of the application object based on the instruction set under test and the second runtime parameters of the application object based on the reference instruction set includes:

[0031] When the application object runs based on the instruction set to be tested, the first execution time of the application object executing the preset test task is obtained; the first execution time represents the time required for the task corresponding to the instruction set to be tested to be completed.

[0032] When the application object runs based on the reference instruction set, the second execution time of the application object executing the preset test task is obtained; the second execution time represents the time required for the task corresponding to the reference instruction set to be completed.

[0033] On the other hand, embodiments of this application disclose an instruction set ecosystem testing device, the device comprising:

[0034] An application object module is used to determine at least one application object associated with an instruction set; the instruction set includes a test instruction set and a reference instruction set; the application object is used to run based on the test instruction set or the reference instruction set;

[0035] The runtime parameter module is used to obtain the first runtime parameters of the application object when running based on the instruction set under test, and the second runtime parameters of the application object when running based on the reference instruction set.

[0036] The parameter result module is used to determine the parameter result of the application object based on the first running parameter and the second running parameter; the parameter result is used to represent the magnitude relationship between the first running parameter and the second running parameter.

[0037] The ecosystem determination module is used to determine the ecosystem maturity of the instruction set to be tested based on the parameter results of each application object.

[0038] Optionally, there may be multiple application objects;

[0039] The device also includes:

[0040] The object composition module is used to classify multiple application objects according to the type of each application object to obtain an object composition of at least one type;

[0041] An ecosystem structure module is used to construct the ecosystem structure of the instruction set based on at least one type of object combination; the ecosystem structure includes at least one application layer, and different application layers correspond to different types of object combinations;

[0042] The ecological structure is used to indicate the order of application objects used when obtaining the first and second operating parameters of at least one application object.

[0043] Optional, the parameter results module includes:

[0044] The sub-parameter submodule is used to determine at least one first sub-parameter in the first running parameters and at least one second sub-parameter in the second running parameters; different first sub-parameters are used to describe different types of running conditions of the application object based on the instruction set under test; different second sub-parameters are used to describe different types of running conditions of the application object based on the reference instruction set.

[0045] The sub-result submodule is used to determine at least one sub-parameter result based on each first sub-parameter and its corresponding second sub-parameter;

[0046] The result weighting submodule is used to determine the parameter result of the application object based on at least one sub-parameter result and the corresponding preset first weight combination; different sub-parameter results correspond to different weights in the preset first weight combination.

[0047] Optionally, the device also includes:

[0048] The hierarchical object module is used to obtain the parameter results of each application object in the object combination corresponding to each application level, so as to obtain the parameter result combination of the application level.

[0049] The hierarchical ecosystem module is used to determine the ecological maturity corresponding to the application level based on at least one parameter result in the parameter result combination.

[0050] Optional, ecological determination modules include:

[0051] The hierarchical ecosystem submodule is used to determine the ecosystem maturity of each application layer based on the parameter results of the application objects in that application layer.

[0052] The instruction set ecosystem submodule is used to determine the ecosystem maturity of the instruction set based on the ecosystem maturity of each application level and the corresponding preset second weight combination; different application levels correspond to different weights in the preset second weight combination.

[0053] Optional, the runtime parameter module includes:

[0054] The first test case support submodule is used to obtain the first support parameter corresponding to the application object after executing a preset set of test cases when the application object runs based on the instruction set to be tested; the first support parameter represents the execution success rate corresponding to the instruction set to be tested.

[0055] The second test case support submodule is used to obtain the second support parameter corresponding to the application object after executing the test case set when the application object runs based on the reference instruction set; the second support parameter represents the execution success rate corresponding to the reference instruction set.

[0056] Optional, the runtime parameter module includes:

[0057] The first duration submodule is used to obtain the first execution duration of the application object executing a preset test task when the application object is running based on the instruction set to be tested; the first execution duration represents the time required for the task corresponding to the instruction set to be tested to be completed.

[0058] The second duration submodule is used to obtain the second execution duration of the application object executing a preset test task when the application object is running based on the reference instruction set; the second execution duration represents the time required for the task corresponding to the reference instruction set to be completed.

[0059] In another aspect, embodiments of this application also disclose an electronic device, which includes a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface communicate with each other through the communication bus. The memory is used to store executable instructions, which cause the processor to execute the aforementioned instruction set ecosystem testing method.

[0060] This application also discloses a readable storage medium, which, when the instructions in the readable storage medium are executed by the processor of an electronic device, enables the electronic device to execute the aforementioned instruction set ecosystem testing method.

[0061] The embodiments of this application include the following advantages: By identifying at least one application object associated with an instruction set, the instruction set including a reference instruction set and an instruction set to be tested, the application object is used to run based on the reference instruction set or the instruction set to be tested. First running parameters of the application object running based on the instruction set to be tested, and second running parameters of the application object running based on the reference instruction set are obtained. Based on the first and second running parameters, the parameter results of the application object are determined, and the parameter results are used to represent the magnitude relationship between the first and second running parameters. Based on the parameter results of each application object, the ecological maturity of the instruction set to be tested is determined. The first running parameters of the application object running based on the instruction set to be tested and the second running parameters of the application object running based on the reference instruction set can be used to jointly determine the parameter results, and then the ecological maturity of the instruction set can be obtained based on the parameter results of each application object, thus accurately measuring the applicability of the instruction set to be tested.

[0062] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 is a flowchart of the steps of an instruction set ecosystem testing method according to this application;

[0065] Figure 2 is a flowchart of another instruction set ecosystem testing method of this application;

[0066] Figure 3 is a structural block diagram of the instruction set ecology testing device provided in the example of this application;

[0067] Figure 4 is a structural block diagram of an electronic device for instruction set ecosystem testing provided in this application example. Specific Implementation

[0068] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0069] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and are not limited in number; for example, a first object can be one or more. Furthermore, the term "and / or" in the specification and claims is used to describe the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. In the embodiments of this application, the term "multiple" refers to two or more, and other quantifiers are similar.

[0070] Currently, the widely used instruction sets are the x86 architecture and the Advanced Reduced Instruction Set Computing (ARM) architecture. With technological advancements, the fifth generation of Reduced Instruction Set Computing (RISC), namely RISC-V, is gradually beginning to be applied.

[0071] RISC-V, x86, and ARM are three different architectures. From an instruction set perspective, x86 is a Complex Instruction Set Computing (CISC) architecture with a complex instruction set, supporting a large number of instructions, offering rich functionality, but also involving high complexity in instruction decoding. ARM is a Reduced Instruction Set Computing (RISC) architecture with a simpler instruction set than x86, fixed instruction lengths, and simplified instruction decoding and execution. RISC-V is an open-source RISC instruction set architecture with a concise instruction set that is easy to understand and implement, supports modular design, and allows selection of different instruction subsets based on specific needs. From an ecosystem maturity perspective, x86 has the most mature ecosystem, with extensive hardware support and widespread application in desktop computers, servers, and supercomputing, as well as abundant software and driver support. ARM is primarily used in mobile devices, widely found in smartphones and tablets. ARM architecture features low power consumption, while RISC-V, with its shorter development history, can be applied to embedded systems, the Internet of Things (IoT), and edge computing.

[0072] Method Implementation Examples

[0073] Referring to Figure 1, a flowchart illustrating the steps of an embodiment of an instruction set ecosystem testing method according to this application is shown. The method may specifically include the following steps:

[0074] Step 101: Determine at least one application object associated with the instruction set; the instruction set includes a test instruction set and a reference instruction set; the application object is used to run based on the test instruction set or the reference instruction set.

[0075] In this application embodiment, it is important to understand that the instruction set ecosystem refers to a comprehensive system built around a specific instruction set architecture, encompassing hardware, tools, operating systems, development environments, applications, services, community support, and the developer community. This includes microarchitectural design directly related to the instruction set hardware, as well as compilers, libraries, operating systems, development toolchains, debuggers, performance analyzers, middleware, application programming interfaces (APIs), standards and specifications, security frameworks, and multiple layers such as community contributions and training. These elements collectively form an interdependent and collaborative environment. The health and activity of the instruction set ecosystem directly impact the adoption rate, developer acceptance, and ultimate application extent of the architecture.

[0076] Instruction Set Ecosystem Maturity is a quantitative or qualitative parameter that describes the development level of an instruction set ecosystem. It is used to determine the overall performance of the instruction set in terms of technological maturity, market application, abundance of supporting resources, community activity, compatibility, sustainable development capabilities, and technological innovation.

[0077] Application objects can be applications, firmware, build toolchains and development toolkits, deep learning frameworks, deep learning computing platforms, and other objects, without limitation. Application objects associated with an instruction set can refer to application objects that can run on hardware based on that instruction set architecture.

[0078] Instruction sets can include reference instruction sets and instruction sets under test. Reference instruction sets can be instruction sets with high ecosystem maturity, such as x86 or ARM, while instruction sets under test can be instruction sets with unclear ecosystem maturity, such as RISC-V or other instruction sets that have emerged with technological development. There are no specific restrictions on reference instruction sets and instruction sets under test.

[0079] Step 102: Obtain the first running parameters of the application object based on the instruction set to be tested, and the second running parameters of the application object based on the reference instruction set.

[0080] In this embodiment of the application, running an application object based on the instruction set under test can refer to running hardware based on the architecture of the instruction set under test, obtaining the first running parameters corresponding to the instruction set under test, and the first running parameters representing the running status of the application object based on the instruction set under test.

[0081] Similarly, "application object running based on reference instruction set" can refer to hardware running based on the reference instruction set architecture, obtaining the second running parameters corresponding to the reference instruction set, and the second running parameters indicating the running status of the application object running based on the reference instruction set.

[0082] The first and second operating parameters can be one or a group of parameters, and the specific content and number of operating parameters are not limited here.

[0083] Step 103: Based on the first running parameters and the second running parameters, determine the parameter result of the application object; the parameter result is used to represent the size relationship between the first running parameters and the second running parameters.

[0084] In this embodiment of the application, the parameter result is determined based on the first and second running parameters of the application object. This can be achieved by subtracting or dividing the first and second running parameters or by performing other operations. The parameter result can represent the size relationship between the first and second running parameters.

[0085] There can be one or more application objects, so the corresponding parameter results can be determined for each application object. It can be understood that the parameter results can also serve as the ecological maturity of the application object. The parameter results of the application object can represent the relative relationship between the application object's performance based on the instruction set under test and its performance based on the reference instruction set.

[0086] Step 104: Determine the ecological maturity of the instruction set to be tested based on the parameter results of each application object.

[0087] In the embodiments of this application, the parameter results of each application object may be comprehensively statistically analyzed, such as adding the parameter results of each application object or calculating the average value, and other more complex calculations may be performed to obtain the ecological maturity of the instruction set to be tested, which is not limited here.

[0088] In summary, the embodiments of this application have the following advantages: By identifying at least one application object associated with an instruction set, the instruction set including a reference instruction set and an instruction set to be tested, the application object is used to run based on the reference instruction set or the instruction set to be tested. First running parameters of the application object running based on the instruction set to be tested, and second running parameters of the application object running based on the reference instruction set are obtained. Based on the first and second running parameters, the parameter results of the application object are determined, and the parameter results are used to represent the magnitude relationship between the first and second running parameters. Based on the parameter results of each application object, the ecological maturity of the instruction set to be tested is determined. The first running parameters of the application object running based on the instruction set to be tested and the second running parameters of the application object running based on the reference instruction set can be used to jointly determine the parameter results, and then the ecological maturity of the instruction set can be obtained based on the parameter results of each application object, thus accurately measuring the applicability of the instruction set to be tested.

[0089] Referring to Figure 2, a flowchart illustrating the steps of another embodiment of the instruction set ecosystem testing method of this application is shown. The method may specifically include the following steps:

[0090] Step 201: Determine at least one application object associated with the instruction set; the instruction set includes a test instruction set and a reference instruction set; the application object is used to run based on the test instruction set or the reference instruction set;

[0091] Step 202: Obtain the first running parameters of the application object based on the instruction set to be tested, and the second running parameters of the application object based on the reference instruction set.

[0092] Step 203: Based on the first running parameters and the second running parameters, determine the parameter result of the application object; the parameter result is used to represent the size relationship between the first running parameters and the second running parameters;

[0093] Step 204: Determine the ecological maturity of the instruction set to be tested based on the parameter results of each application object.

[0094] The contents of steps 201-204 above can be referred to the embodiment in Figure 1 above, and will not be repeated here.

[0095] Optionally, there may be multiple application objects;

[0096] Following the step of determining at least one application object associated with the instruction set, the method further includes:

[0097] Step A1: Based on the type of each application object, classify multiple application objects to obtain a combination of objects of at least one type;

[0098] Step A2: Construct the ecosystem structure of the instruction set based on at least one type of object combination; the ecosystem structure includes at least one application layer, with different application layers corresponding to different types of object combinations;

[0099] The ecological structure is used to indicate the order of application objects used when obtaining the first and second operating parameters of at least one application object.

[0100] In this embodiment of the application, when multiple application objects associated with the instruction set are identified, these application objects can be classified. This classification can be based on the functional purpose of the application objects. For example, the classification could be as follows:

[0101] For the cloud computing / cloud-native ecosystem, it can include the analytics and visualization platform Grafana, the container orchestration platform Kubernetes, the package manager Helm, the mesh service platform Istio, and the system monitoring and alerting toolkit Prometheus;

[0102] Open source projects, machine learning platforms, deep learning frameworks and libraries for video encoding / decoding and AI computing, and others, including PyTorch, MXNet, Caffe, Tensorflow, OpenCV, OpenBLAS, MNN, Ffmpeg, scikit-learn, Apache Spark Mllib, CNTK, Deeplearning4j, Apache Mahout, and ONNX.

[0103] Distributed system infrastructure, relational database management systems, distributed relational databases, and others used in the big data processing ecosystem may include: Hadoop, Spark, Flink, MySQL, GaussDB, Redis, and MariaDB.

[0104] Virtual machines used for virtualization and containerization, virtualization management platforms, container runtime daemons, application container engines, and others, may include kernel-based virtual machines (KVM), StartoVirt, iSulaD, docker;

[0105] Language-related technologies can include OpenJDK (an open-source implementation of the Java language) and Python (a computer programming language).

[0106] The compilation toolchain and development toolkit may include the GNU Compiler Collection (GCC), the Low Level Virtual Machine (LLVM), the Quick Emulator (QEMU), and the GNU symbolic debugger (GDB); where GNU is an operating system.

[0107] Operating systems and basic libraries may include OpenEuler (an open-source Linux operating system distribution), the Linux kernel, open-source cryptographic libraries (OpenSSL, Open Secure Sockets Layer), a C standard library implementation (musl libc), an open-source implementation of the SSH (Secure Shell) protocol (OpenSSH), and the C standard library (Glibc, GNU C Library).

[0108] The system firmware layer may include a unified extensible firmware interface (UEFI), an open-source secure boot implementation (OpenSBI), and an open-source baseboard management controller (OpenBMC).

[0109] Hardware support and acceleration can include machine learning units (MLUs), tensor processing units (TPUs), and others.

[0110] The above are some of the application objects and categories listed in this application. Different numbers of categories and different numbers of application objects can be set as needed.

[0111] Each category of objects can be considered a hierarchy, with each hierarchy containing one or more application objects. Further combining application objects from different hierarchical levels yields an ecosystem structure describing the entire instruction set. Each application object within this ecosystem can be used to determine the ecosystem maturity of the instruction set under test.

[0112] The ecosystem structure of the entire instruction set is as follows:

[0113] Where L1, L2, and others represent different application layers, L 1.1 L 1.2 ...This refers to application objects within application layer L1. Other application layers are similar and will not be elaborated further. Based on the ecosystem structure, runtime parameters can be obtained systematically for each application object.

[0114] In implementing embodiments of this application, multiple application objects are classified according to the type of each application object to obtain at least one type of object combination; based on the at least one type of object combination, the ecological structure of the instruction set is constructed. When determining the ecological maturity of the instruction set, the functions of different application objects can be clearly defined. Then, based on the ecological structure, the operating parameters can be obtained in an orderly manner for each application object, which is conducive to the clear and accurate determination of the ecological maturity of the instruction set under test.

[0115] Optionally, the step of determining the parameter result of the application object based on the first running parameters and the second running parameters includes:

[0116] Determine at least one first sub-parameter in the first running parameters and at least one second sub-parameter in the second running parameters; different first sub-parameters are used to describe different types of running conditions of the application object based on the instruction set to be tested; different second sub-parameters are used to describe different types of running conditions of the application object based on the reference instruction set.

[0117] Based on each first sub-parameter and its corresponding second sub-parameter, determine at least one sub-parameter result;

[0118] The parameter result of the application object is determined based on at least one sub-parameter result and the corresponding preset first weight combination; different sub-parameter results correspond to different weights in the preset first weight combination.

[0119] In this embodiment, the first running parameter can be a set of parameters, which may include at least one first sub-parameter. Different first sub-parameters are used to describe different types of running conditions of the application object based on the reference instruction set. For example, the first sub-parameter can be used to describe the running time, number of instructions, cache read status, and other types of running conditions of the application object based on the instruction set under test.

[0120] Since the first and second operating parameters differ only in their corresponding instruction sets, the second operating parameters and the second sub-parameters are similar and will not be elaborated upon here.

[0121] It is understandable that different sub-parameters are used to describe different types of operating conditions, and each sub-parameter can also be a set of sub-parameters, that is, a specific type of operating condition is described by a set of parameters.

[0122] For example, when testing the deep learning framework PyTorch to obtain the corresponding runtime parameters, PyTorch runtime parameters can include multiple sub-parameters.

[0123] PyTorch can include development projects, which can compare the number of instructions executed for a reference instruction set and the instruction set under test, as shown in the following formula:

[0124] Where, N K,RISC-V N represents the number of instructions in a single item within the first sub-parameter (based on the RISC-V instruction set). K,ARM RatioN represents the number of instructions (based on the ARM instruction set) for a single item in the second sub-parameter. K It is the speedup ratio of the number of instructions between the two.

[0125] However, PyTorch can include multiple projects to be tested (different projects can be used for different domains and applications). That is, each sub-parameter can also be a group of sub-parameters. Therefore, tests can be performed on each project to be tested to obtain the number of instructions executed, as shown in the following formula:

[0126] Among them, S N This represents the result of a sub-parameter when there are multiple projects, where q represents the number of projects, and RatioN... KThis is the instruction speedup ratio. The above formula allows us to average the instruction speedup ratios for multiple projects to obtain the final sub-parameter result. This sub-parameter result represents the instruction count of the instruction set under test (RISC-V) relative to the reference instruction set (ARM) across multiple projects. The instruction count corresponds to compilation quality; for example, fewer instructions generally result in higher compilation quality. N This can reflect the difference in compilation quality (CQ) between the instruction set under test and the reference instruction set.

[0127] For each project, another test could be the L1-dcache-load-misses, a metric used to measure the number of times the processor cannot find the required data in the cache and must therefore read the data from the slower main memory. Similarly, the following formula applies:

[0128] Among them, M K,RISC-V This represents the number of cache read misses for a single item in the first sub-parameter (based on the RISC-V instruction set), M. K,ARM RatioM represents the number of cache read misses for a single item in the second sub-parameter (based on the ARM instruction set). K This represents the speedup ratio for cache read misses between the two. Similarly, when there are multiple items, the following formula applies:

[0129] Among them, S M This represents the result of a sub-parameter when there are multiple projects, where q represents the number of projects, and RatioM K This is the cache read miss speedup ratio. The above formula allows us to average the cache read miss speedup ratios for multiple items to obtain the final sub-parameter result. This sub-parameter result represents the cache read miss count of the instruction set under test (RISC-V) relative to the reference instruction set (ARM) across multiple items. M It can represent the difference in memory performance (MP) between the instruction set under test and the reference instruction set.

[0130] The sub-parameter results obtained based on the above formula include, for example, S. N S M Each sub-parameter result can be combined with different weights in the preset first weight combination to determine the parameter result of the application object, i.e., the ecological maturity of the application object.

[0131] The embodiments of this application involve determining at least one first sub-parameter in the first operating parameters and at least one second sub-parameter in the second operating parameters; determining at least one sub-parameter result based on each first sub-parameter and its corresponding second sub-parameter; and determining the parameter result of the application object based on the at least one sub-parameter result and its corresponding preset first weight combination. This approach enables the determination of the application object's parameter result based on different aspects corresponding to different sub-parameters, thus improving the accuracy of the application object's parameter result.

[0132] Optionally, the step of obtaining the first runtime parameters of the application object based on the instruction set under test and the second runtime parameters of the application object based on the reference instruction set includes:

[0133] When the application object runs based on the instruction set to be tested, the first support parameter corresponding to the application object executing the preset test case set is obtained; the first support parameter represents the execution success rate corresponding to the instruction set to be tested.

[0134] When the application object runs based on the reference instruction set, obtain the second support parameter corresponding to the application object after executing the test case set; the second support parameter represents the execution success rate corresponding to the reference instruction set.

[0135] In this embodiment of the application, the running parameters may be support parameters for the test cases representing the instruction set, and the support parameters represent the execution success rate of the test cases.

[0136] When an application runs based on the instruction set under test, it can execute a pre-defined set of test cases. The first success rate is determined by the number of successfully executed test cases in the set. Successful test case execution can mean that the tested function or feature achieves the expected result. For example, in a calculation function test case, the result obtained from inputting specific data is accurate. It can also mean that no system crashes, error messages, or unexpected interruptions occur during test case execution, or that no data is lost, corrupted, or incorrectly modified, or any other issues arise.

[0137] Similarly, when the application object runs based on the reference instruction set, the second supporting parameter corresponding to the application object after executing the test case set is obtained.

[0138] Optionally, the step of obtaining the first runtime parameters of the application object based on the instruction set under test and the second runtime parameters of the application object based on the reference instruction set includes:

[0139] When the application object runs based on the instruction set to be tested, the first execution time of the application object executing the preset test task is obtained; the first execution time represents the time required for the task corresponding to the instruction set to be tested to be completed.

[0140] When the application object runs based on the reference instruction set, the second execution time of the application object executing the preset test task is obtained; the second execution time represents the time required for the task corresponding to the reference instruction set to be completed.

[0141] In this embodiment, the running parameters can represent the execution time of a preset test task. PyTorch can include development projects, which can correspond to test tasks. The execution time (on the wall) corresponding to the reference instruction set and the instruction set to be tested can be compared, as shown in the following formula:

[0142] Among them, T K,RISC-V T represents the first execution time (based on the RISC-V instruction set) of a single item in the first sub-parameter. K,ARM RatioT represents the second execution time (based on the ARM instruction set) of a single item in the second sub-parameter. K It represents the speedup ratio of execution time for both methods, corresponding to the sub-parameter results for a single project.

[0143] The corresponding sub-parameters can be determined based on the execution duration of multiple projects under test. When there are multiple projects, the formula is as follows:

[0144] Among them, S T This represents the result of a sub-parameter when there are multiple projects, where q represents the number of projects, and RatioT... K This is the execution time speedup ratio. The above formula allows us to average the execution time speedup ratios for multiple projects to obtain the final sub-parameter result. This sub-parameter result represents the execution time of the instruction set under test (RISC-V) relative to the reference instruction set (ARM) across multiple projects. T It can represent the difference in overall system performance between the instruction set under test and the reference instruction set, i.e., the difference in performance advancement (PA).

[0145] Besides S regarding execution time T Regarding the number of cache read misses (S) MBeyond test case support rates, other operational scenarios can also include the number of patches committed by software developers for different instruction sets, the support rate of different instruction set architecture versions of the same software for heterogeneous computing units, and the support rate of different instruction sets for specific industry business items. Heterogeneous computing units can include Central Processing Units (CPUs), Neural Processing Units (NPUs), Machine Learning Units (MLUs), TPUs, Single Instruction Multiple Data (SIMD) extended instruction sets NEO, RISC-V Vector Extensions (RVV), and others. These different types of operational scenarios can be understood as operational scenarios detected from different dimensions and perspectives.

[0146] Different weight allocations can be applied to different types of operating conditions, as shown by the following formula:

[0147] Where w1 is the sub-parameter result S related to execution time. T The weights;

[0148] w2 is the result of the sub-parameter S concerning the number of instructions. N The weights;

[0149] w3 is a subparameter result related to the number of cache read misses. M The weights;

[0150] w4 is the weight of the sub-parameter result regarding test case support rate. The sub-parameter result a2 / a1 can represent the difference in functional integrity (FI). a2 is the test case support rate of the instruction set under test, and a1 is the test case support rate of the reference instruction set.

[0151] w5 is the weight of the sub-parameter result regarding the number of software development patches submitted. The sub-parameter result b2 / b1 can represent the difference in the ecological prosperity of the instruction set under test and the reference instruction set, i.e., the open-source development activity level (ODAL). b2 is the number of software development patches submitted for the instruction set under test, and b1 is the number of software development patches submitted for the reference instruction set.

[0152] w6 is the weight of the sub-parameter result regarding the support rate of heterogeneous computing units. The sub-parameter result c2 / c1 can represent the difference in heterogeneous computing support level (HCSL). c2 is the support rate of heterogeneous computing units in the instruction set under test, and c1 is the support rate of heterogeneous computing units in the reference instruction set.

[0153] w7 is the weight of the sub-parameter result regarding the support rate of business items in a specific industry. The sub-parameter result d2 / d1 can represent the difference in Industry Application Support Level (IASL). d2 is the support rate of business items in a specific industry for the instruction set under test, and d1 is the support rate of business items in a specific industry for the reference instruction set.

[0154] In general, for a specific application, the overall parameter result of the application, i.e., the ecosystem maturity, can be represented by the following formula:

[0155] Among them, L ij This refers to the ecological maturity of the application object. For different types of operational scenarios, there is a maturity performance vector α = (S... T ,S N ,S M The maturity performance vector of the reference instruction set is β = (1, 1, 1, a2, b2, c2, d2), and the dimension weight vector is γ. k =(w1,w2,w3,w4,w5,w6,w7).

[0156] Optionally, the method further includes:

[0157] Step B1: For each application layer corresponding to the object combination, obtain the parameter results of each application object in the object combination to obtain the parameter result combination of the application layer;

[0158] Step B2: Determine the ecological maturity level corresponding to the application level based on at least one parameter result in the parameter result combination.

[0159] In the embodiments of this application, the ecological structure is composed of each level, and each level is also composed of at least one application object. Therefore, each application level actually corresponds to an object combination. Based on the parameter results of each application object in the object combination, the parameter result combination of the application level can be obtained.

[0160] Furthermore, the ecological maturity of the application layer can be determined based on the combination of parameter results. For example, this could involve adding the parameter results in the combination, or dividing the sum by the number of parameter results in the combination to obtain an average value, or other more complex calculation processes.

[0161] For example, the ecosystem maturity of an application layer can be determined using the following formula:

[0162] Among them, L i L represents the ecosystem maturity of the i-th application layer, n represents the number of application objects in the i-th application layer, and L represents the ecosystem maturity of the i-th application layer. ij This represents the ecological maturity of the application objects in the i-th application layer.

[0163] In implementing embodiments of this application, parameter results for each application object in the object combination corresponding to each application layer are obtained to obtain a parameter result combination for the application layer; based on at least one parameter result in the parameter result combination, the ecosystem maturity corresponding to the application layer is determined. This allows for the independent determination of the ecosystem maturity of the application layer, thereby revealing the usability of the instruction set under test for a specific purpose or function, further improving the granularity of ecosystem testing.

[0164] Optionally, the step of determining the ecological maturity of the instruction set to be tested based on the parameter results of each application object includes:

[0165] For each application layer, the ecosystem maturity of the application layer is determined based on the parameter results of the application objects in that application layer.

[0166] The ecological maturity of the instruction set is determined based on the ecological maturity of each application level and the corresponding preset second weight combination; different application levels correspond to different weights in the preset second weight combination.

[0167] In this embodiment, the ecological maturity of an application layer can be determined based on the ecological maturity of the parameter result set of each application object in each application layer.

[0168] Since the instruction set under test corresponds to the entire ecosystem, its ecosystem maturity can be determined based on the ecosystem maturity of each application layer within the ecosystem. Furthermore, each application layer can correspond to different preset weights. The ecosystem maturity of the instruction set under test can be determined using the following formula:

[0169] Among them, W iis the weight of the i-th layer in the ecological structure, m is the number of application layers, and the sum of the preset weights (i.e., the preset second weight combination) corresponding to all application layers is 1; L represents the overall ecological maturity of the ecological structure corresponding to the instruction set to be tested. i This represents the ecological maturity of the i-th application layer.

[0170] The embodiments of this application include the following advantages: By identifying at least one application object associated with an instruction set, the instruction set including a reference instruction set and an instruction set to be tested, the application object is used to run based on the reference instruction set or the instruction set to be tested. First running parameters of the application object running based on the instruction set to be tested, and second running parameters of the application object running based on the reference instruction set are obtained. Based on the first and second running parameters, the parameter results of the application object are determined, and the parameter results are used to represent the magnitude relationship between the first and second running parameters. Based on the parameter results of each application object, the ecological maturity of the instruction set to be tested is determined. The first running parameters of the application object running based on the instruction set to be tested and the second running parameters of the application object running based on the reference instruction set can be used to jointly determine the parameter results, and then the ecological maturity of the instruction set can be obtained based on the parameter results of each application object, thus accurately measuring the applicability of the instruction set to be tested.

[0171] It is understandable that the various operational parameters used to determine the ecological maturity can be updated periodically, the operational parameters can be collected automatically, and parameters such as preset weights can be updated in real time as needed.

[0172] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0173] Device Examples

[0174] Referring to Figure 3, a structural block diagram of an instruction set ecology testing device according to this application is shown. The device 30 includes:

[0175] Application object module 301 is used to determine at least one application object associated with an instruction set; the instruction set includes a test instruction set and a reference instruction set; the application object is used to run based on the test instruction set or the reference instruction set;

[0176] The runtime parameter module 302 is used to obtain the first runtime parameters of the application object running based on the instruction set to be tested, and the second runtime parameters of the application object running based on the reference instruction set.

[0177] The parameter result module 303 is used to determine the parameter result of the application object based on the first running parameter and the second running parameter; the parameter result is used to represent the magnitude relationship between the first running parameter and the second running parameter;

[0178] Ecosystem determination module 304 is used to determine the ecological maturity of the instruction set to be tested based on the parameter results of each application object.

[0179] Optionally, there are multiple application objects; the device further includes:

[0180] The object composition module is used to classify multiple application objects according to the type of each application object to obtain an object composition of at least one type;

[0181] An ecosystem structure module is used to construct the ecosystem structure of the instruction set based on at least one type of object combination; the ecosystem structure includes at least one application layer, and different application layers correspond to different types of object combinations;

[0182] The ecological structure is used to indicate the order of application objects used when obtaining the first and second operating parameters of at least one application object.

[0183] Optional, the parameter results module includes:

[0184] The sub-parameter submodule is used to determine at least one first sub-parameter in the first running parameters and at least one second sub-parameter in the second running parameters; different first sub-parameters are used to describe different types of running conditions of the application object based on the instruction set under test; different second sub-parameters are used to describe different types of running conditions of the application object based on the reference instruction set.

[0185] The sub-result submodule is used to determine at least one sub-parameter result based on each first sub-parameter and its corresponding second sub-parameter;

[0186] The result weighting submodule is used to determine the parameter result of the application object based on at least one sub-parameter result and the corresponding preset first weight combination; different sub-parameter results correspond to different weights in the preset first weight combination.

[0187] Optionally, the device also includes:

[0188] The hierarchical object module is used to obtain the parameter results of each application object in the object combination corresponding to each application level, so as to obtain the parameter result combination of the application level.

[0189] The hierarchical ecosystem module is used to determine the ecological maturity corresponding to the application level based on at least one parameter result in the parameter result combination.

[0190] Optional, ecological determination modules include:

[0191] The hierarchical ecosystem submodule is used to determine the ecosystem maturity of each application layer based on the parameter results of the application objects in that application layer.

[0192] The instruction set ecosystem submodule is used to determine the ecosystem maturity of the instruction set based on the ecosystem maturity of each application level and the corresponding preset second weight combination; different application levels correspond to different weights in the preset second weight combination.

[0193] Optional, the runtime parameter module includes:

[0194] The first test case support submodule is used to obtain the first support parameter corresponding to the application object after executing a preset test case set when the application object is running based on the reference instruction set; the first support parameter represents the execution success rate corresponding to the instruction set to be tested;

[0195] The second test case support submodule is used to obtain the second support parameter corresponding to the application object after executing the test case set when the application object runs based on the instruction set to be tested; the second support parameter represents the execution success rate corresponding to the reference instruction set.

[0196] Optional, the runtime parameter module includes:

[0197] The first duration submodule is used to obtain the first execution duration of the application object executing a preset test task when the application object is running based on the instruction set to be tested; the first execution duration represents the time required for the task corresponding to the instruction set to be tested to be completed.

[0198] The second duration submodule is used to obtain the second execution duration of the application object executing a preset test task when the application object is running based on the reference instruction set; the second execution duration represents the time required for the task corresponding to the reference instruction set to be completed.

[0199] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0200] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0201] Regarding the processor in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0202] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0203] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the computing processing device according to the embodiments of this application. This application can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0204] Referring to Figure 4, which is a structural block diagram of an electronic device for instruction set ecosystem testing according to an embodiment of this application, the electronic device includes: a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface communicate with each other through the communication bus. The memory stores executable instructions, which cause the processor to execute the instruction set ecosystem testing method of the aforementioned embodiment.

[0205] The processor can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable devices, transistor logic devices, hardware components, or any combination thereof. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, or a combination of a DSP and a microprocessor.

[0206] The communication bus may include a path for transmitting information between the memory and the communication interface. The communication bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus can be divided into an address bus, a data bus, and a control bus. For ease of illustration, only one line is used in Figure 4, but this does not indicate that there is only one bus or one type of bus.

[0207] The memory may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or it may be EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory), magnetic tape, floppy disk, and optical data storage devices.

[0208] This application also provides a non-transitory computer-readable storage medium, which, when the instructions in the storage medium are executed by the processor of an electronic device (server or terminal), enables the processor to execute the instruction set ecosystem testing method shown in Figures 1 and 2.

[0209] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0210] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including disk storage, CD-ROM, optical storage) containing computer-usable program code.

[0211] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0212] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a predictive manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0213] These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment to implement functional steps in a flowchart of one or more processes and / or a block diagram of one or more blocks.

[0214] While preferred embodiments of the embodiments described herein are presented, those skilled in the art will understand the basic inventive concept and can make other changes and modifications to these embodiments. The appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the embodiments of this application.

[0215] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0216] The terms "an embodiment," "embodiment," or "one or more embodiments" as used herein mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of this application. Furthermore, please note that the examples of the phrase "in one embodiment" do not necessarily all refer to the same embodiment.

[0217] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0218] In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0219] The foregoing has provided a detailed description of the instruction set ecosystem testing method, apparatus, electronic device, and readable storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method of ecosystem testing of instruction sets, wherein, The method includes: Identify at least one application object associated with an instruction set; the instruction set includes a test instruction set and a reference instruction set; the application object is used to run based on the test instruction set or the reference instruction set; Obtain the first running parameters of the application object when running based on the instruction set to be tested, and the second running parameters of the application object when running based on the reference instruction set; Based on the first operating parameter and the second operating parameter, the parameter result of the application object is determined; the parameter result is used to represent the magnitude relationship between the first operating parameter and the second operating parameter. Based on the parameter results of each application object, the ecological maturity of the instruction set to be tested is determined; There are multiple application objects; after the step of determining at least one application object associated with the instruction set, the method further includes: Based on the type of each application object, multiple application objects are classified to obtain a combination of objects of at least one type; The instruction set's ecosystem is constructed based on at least one type of object combination; the ecosystem includes at least one application layer, with different application layers corresponding to different types of object combinations. The ecological structure is used to indicate the order of application objects used when obtaining the first and second operating parameters of at least one application object.

2. The method of claim 1, wherein, The step of determining the parameter result of the application object based on the first operating parameter and the second operating parameter includes: Determine at least one first sub-parameter in the first running parameters and at least one second sub-parameter in the second running parameters; different first sub-parameters are used to describe different types of running conditions of the application object based on the instruction set to be tested; different second sub-parameters are used to describe different types of running conditions of the application object based on the reference instruction set. Based on each first sub-parameter and its corresponding second sub-parameter, determine at least one sub-parameter result; The parameter result of the application object is determined based on at least one sub-parameter result and the corresponding preset first weight combination; different sub-parameter results correspond to different weights in the preset first weight combination.

3. The method of claim 1, wherein, The method further includes: For each object combination corresponding to each application level, obtain the parameter results of each application object in the object combination to obtain the parameter result combination of the application level; Based on at least one parameter result in the combination of parameter results, the ecological maturity corresponding to the application level is determined.

4. The method of claim 1, wherein, The step of determining the ecological maturity of the instruction set to be tested based on the parameter results of each application object includes: For each application layer, the ecosystem maturity of the application layer is determined based on the parameter results of the application objects in that application layer. The ecological maturity of the instruction set is determined based on the ecological maturity of each application level and the corresponding preset second weight combination; different application levels correspond to different weights in the preset second weight combination.

5. The method of claim 1, wherein, The step of obtaining the first runtime parameters of the application object based on the instruction set under test and the second runtime parameters of the application object based on the reference instruction set includes: When the application object runs based on the instruction set to be tested, the first support parameter corresponding to the application object executing the preset test case set is obtained; the first support parameter represents the execution success rate corresponding to the instruction set to be tested. When the application object runs based on the reference instruction set, obtain the second support parameter corresponding to the application object after executing the test case set; the second support parameter represents the execution success rate corresponding to the reference instruction set.

6. The method of claim 1, wherein, The step of obtaining the first runtime parameters of the application object based on the instruction set under test and the second runtime parameters of the application object based on the reference instruction set includes: When the application object runs based on the instruction set to be tested, the first execution time of the application object executing the preset test task is obtained; the first execution time represents the time required for the task corresponding to the instruction set to be tested to be completed. When the application object runs based on the reference instruction set, the second execution time of the application object executing the preset test task is obtained; the second execution time represents the time required for the task corresponding to the reference instruction set to be completed.

7. An instruction set ecosystem testing apparatus, wherein, The device includes: An application object module is used to determine at least one application object associated with an instruction set; the instruction set includes a test instruction set and a reference instruction set; the application object is used to run based on the test instruction set or the reference instruction set; The runtime parameter module is used to obtain the first runtime parameters of the application object when running based on the instruction set under test, and the second runtime parameters of the application object when running based on the reference instruction set. The parameter result module is used to determine the parameter result of the application object based on the first running parameter and the second running parameter; the parameter result is used to represent the magnitude relationship between the first running parameter and the second running parameter. An ecosystem determination module is used to determine the ecosystem maturity of the instruction set to be tested based on the parameter results of each application object. The application objects are multiple; the device also includes: The object composition module is used to classify multiple application objects according to the type of each application object to obtain an object composition of at least one type; An ecosystem structure module is used to construct the ecosystem structure of the instruction set based on at least one type of object combination; the ecosystem structure includes at least one application layer, and different application layers correspond to different types of object combinations; The ecological structure is used to indicate the order of application objects used when obtaining the first and second operating parameters of at least one application object.

8. An electronic device, comprising: The electronic device includes a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface communicate with each other through the communication bus. The memory is used to store executable instructions, which cause the processor to execute the instruction set ecosystem testing method as described in any one of claims 1 to 6.

9. A readable storage medium, wherein, When the instructions in the readable storage medium are executed by the processor of the electronic device, the processor is able to perform the instruction set ecosystem testing method as described in any one of claims 1 to 6.

10. A computer program comprising computer-readable code, which, when executed on a computing processing device, causes the computing processing device to perform the instruction set ecosystem testing method according to any one of claims 1-6.

Images