Test method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of computers, in particular to a testing method and device, electronic equipment and a storage medium, and the method comprises the steps: collecting software version information and hardware configuration information, and respectively constructing a software version library and a hardware configuration library; based on the software version library and the hardware configuration library, analyzing the compatibility between the software version and the hardware configuration, and screening out at least one compatible software and hardware combination scheme; in a virtualized test environment, executing a test on the screened software and hardware combination scheme to obtain a virtual test result; and based on the virtual test result, selecting at least one target software and hardware combination scheme to perform a physical test, and generating an optimization suggestion according to the virtual test result and a physical test result. According to the embodiment of the invention, the multi-hardware configuration and multi-software version adaptability test efficiency can be improved, and the test resource consumption can be reduced.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a testing method and apparatus, electronic equipment and storage medium. Background Technology

[0002] In the testing field, especially in compatibility testing scenarios involving various hardware configurations and continuously iterating software versions, how to complete testing efficiently, comprehensively, and at low cost is a key issue. Typically, to evaluate the compatibility and performance of software on different hardware platforms, it is necessary to deploy the software to various physical devices for testing. Due to the diversity of hardware configurations (e.g., different CPUs, GPUs, memory, and operating systems) and frequent software version updates, the potential number of "hardware configuration-software version" combinations is enormous. Using traditional one-by-one physical testing methods would result in a huge workload, a long testing cycle, and consume a large amount of physical equipment resources, leading to low testing efficiency.

[0003] In related technologies, some testing solutions attempt to reduce manual intervention. For example, they control the testing process through scripts or utilize virtualization technology for preliminary verification. However, when dealing with the complex compatibility between multiple software versions and hardware configurations, these solutions still require improvement in terms of testing specificity and efficiency, and still consume a significant amount of physical equipment resources. Summary of the Invention

[0004] In view of this, this disclosure proposes a testing technology solution.

[0005] According to one aspect of this disclosure, a testing method is provided, comprising:

[0006] Collect software version information and hardware configuration information, and build software version repositories and hardware configuration repositories respectively;

[0007] Based on the software version library and hardware configuration library, analyze the compatibility between software versions and hardware configurations, and select at least one compatible software and hardware combination scheme.

[0008] In a virtualized testing environment, the selected software and hardware combination schemes are tested to obtain virtual test results;

[0009] Based on the virtual test results, at least one target software and hardware combination scheme is selected for physical testing, and optimization suggestions are generated based on the virtual test results and physical test results.

[0010] In one possible implementation, the software version information to be collected includes:

[0011] Parse the target version of the software package to obtain the first software bill of materials;

[0012] The binary file obtained by running the target version of the software in the executable environment is parsed to obtain the binary compatibility features required for the actual operation of the software, and a second software bill of materials is obtained. The binary compatibility features include at least one of the following: actual dependencies, instruction set and application binary interface (ABI) features.

[0013] Based on the first software bill of materials and the second software bill of materials, a trusted software bill of materials for the target version is determined and stored in the software version library.

[0014] In one possible implementation, the method further includes:

[0015] Compare the trusted software bill of materials for the target version with the trusted software bill of materials for the previous historical version;

[0016] If the compatibility features of the target version have changed compared to the historical version, the hardware configuration library is queried based on the new constraints resulting from the change.

[0017] Hardware configuration schemes in the hardware configuration library that do not meet the new constraints are marked as risky hardware configuration schemes corresponding to the target version.

[0018] In one possible implementation, the analysis of the compatibility between the software version and the hardware configuration, and the selection of at least one compatible software-hardware combination scheme, includes:

[0019] Based on graph neural network (GNN), the hardware interconnection topology of each hardware combination scheme in the hardware configuration library is analyzed to generate topology-aware features corresponding to each hardware combination scheme.

[0020] The topology-aware features are fused with the original hardware parameter features to obtain hardware features;

[0021] Based on the software version library, generate software features that include binary compatibility characteristics;

[0022] By matching the software features with the hardware features using compatibility rules, at least one compatible software-hardware combination scheme is selected.

[0023] In one possible implementation, the compatibility rule matching includes hard rule verification, which includes:

[0024] Remove risky hardware combinations from the hardware configuration library;

[0025] For the hardware combinations that were not eliminated, perform the following:

[0026] Determine whether the instruction set requirements in the software feature are supported by the hardware configuration corresponding to the hardware feature;

[0027] Determine whether the application binary interface (ABI) major version requirement in the software feature is compatible with the ABI version provided by the operating system environment corresponding to the hardware feature.

[0028] In one possible implementation, the software features are matched for compatibility with the hardware features to select at least one compatible software-hardware combination, including:

[0029] Hardware combination schemes that pass the hard rule verification in the hardware configuration library are selected as candidate hardware combination schemes.

[0030] Calculate the similarity between the software features of the target version and the hardware features of each of the candidate hardware combinations;

[0031] Candidate hardware combinations that meet the similarity criteria will be considered as hardware combinations compatible with the target software version.

[0032] In one possible implementation, the similarity between the software features of the target computation version and the hardware features of each of the candidate hardware combinations includes:

[0033] Based on the weight coefficients corresponding to different feature dimensions, the similarity value between the software features and the hardware features of each candidate hardware combination scheme is calculated.

[0034] The feature dimensions include at least one of the following: instruction set features, ABI features, and dependency features.

[0035] In one possible implementation, performing tests on the selected software-hardware combination scheme in a virtualized testing environment includes:

[0036] Based on the hardware characteristics of each selected software and hardware combination scheme, configure the corresponding virtualization test environment;

[0037] In the virtualized testing environment, virtual tests are run for the target version of the software, and virtual test results are generated.

[0038] In one possible implementation, the step of selecting at least one target software-hardware combination scheme for physical testing based on the virtual test results includes:

[0039] The test results in the virtual test results are compared with historical virtual test data to calculate the performance fluctuation value of each software and hardware combination scheme.

[0040] Based on the performance fluctuation value, a target solution is determined from the software and hardware combination scheme, and the target solution covers different performance ranges;

[0041] The target scheme is defined as at least one target software and hardware combination scheme for physical testing.

[0042] In one possible implementation, determining the target solution from the software-hardware combination scheme based on the performance fluctuation value includes:

[0043] Based on whether the performance fluctuation value exceeds a preset threshold, the corresponding software and hardware combination schemes are marked as high-risk combinations and low-risk combinations;

[0044] From the schemes marked as high-risk combinations, select the scheme with the best performance as the first target scheme, and select the scheme with the worst performance as the second target scheme.

[0045] From the options marked as low-risk combinations, select at least one option as the third target option.

[0046] In one possible implementation, generating optimization suggestions based on virtual test results and physical test results includes at least one of the following:

[0047] For the same software and hardware combination scheme, if the physical test results indicate that there is a problem with the software, a diagnostic report containing information on the problem reproduction environment will be generated.

[0048] For the same software and hardware combination scheme, if the physical test results are consistent with the virtual test results, the test data obtained from the test is stored to facilitate subsequent test calculations of performance fluctuation values.

[0049] For the same software and hardware combination scheme, if the physical test results and virtual test results are inconsistent, suggestions for adjusting virtualization platform parameters are generated, and / or the graph neural network is iteratively updated based on the differences between the physical test results and virtual test results.

[0050] According to another aspect of this disclosure, a testing apparatus is provided, comprising:

[0051] The module is used to collect software version information and hardware configuration information, and to build software version repositories and hardware configuration repositories respectively.

[0052] The filtering module is used to analyze the compatibility between software versions and hardware configurations based on the software version library and hardware configuration library, and to filter out at least one compatible software and hardware combination scheme.

[0053] The virtual testing module is used to perform tests on the selected software and hardware combination schemes in a virtualized testing environment and obtain virtual test results.

[0054] The physical testing module is used to select at least one target software and hardware combination scheme for physical testing based on the virtual test results, and generate optimization suggestions based on the virtual test results and the physical test results.

[0055] According to another aspect of this disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-described method.

[0056] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described method.

[0057] According to another aspect of this disclosure, a computer program product is provided, including a computer program or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program, when executed by a processor, implements the steps of the above-described method.

[0058] In this embodiment, software version information and hardware configuration information are collected, and software version libraries and hardware configuration libraries are constructed respectively. Then, based on these libraries, the compatibility between software versions and hardware configurations is analyzed, and at least one compatible software-hardware combination is selected. This allows for the pre-identification of potentially compatible software-hardware combinations before testing, reducing blind testing. In a virtualized testing environment, automated testing is performed on the selected software-hardware combination, yielding virtual test results. This enables rapid and parallel simulation verification of multiple combinations without consuming actual physical resources, reducing testing costs. Based on the virtual test results, at least one target software-hardware combination is selected for physical testing, and optimization suggestions are generated based on the virtual and physical test results. This combines virtual and physical testing, covering key scenarios with fewer actual tests, and continuously optimizing testing strategies and software-hardware compatibility through comparative analysis. Therefore, this testing method improves the efficiency of multi-hardware configuration and multi-software version compatibility testing and reduces testing resource consumption.

[0059] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0060] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0061] Figure 1 A flowchart illustrating a test method according to an embodiment of the present disclosure is shown.

[0062] Figure 2 A flowchart illustrating a test method according to an embodiment of the present disclosure is shown.

[0063] Figure 3 A flowchart illustrating hardware data acquisition according to an embodiment of the present disclosure is shown.

[0064] Figure 4 A flowchart illustrating software data acquisition according to an embodiment of the present disclosure is shown.

[0065] Figure 5 A flowchart illustrating a software and hardware combination scheme selected according to an embodiment of the present disclosure is shown.

[0066] Figure 6 A flowchart illustrating compatibility rule matching according to an embodiment of this disclosure is shown.

[0067] Figure 7 A flowchart illustrating a test method according to an embodiment of the present disclosure is shown.

[0068] Figure 8 A block diagram of a test apparatus according to an embodiment of the present disclosure is shown.

[0069] Figure 9 This is a block diagram illustrating a testing apparatus according to an exemplary embodiment. Detailed Implementation

[0070] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0071] As used herein, the terms “comprising,” “including,” “having,” or variations thereof are open-ended and include one or more of the stated features, integrals, elements, steps, components, or functions, but do not exclude the presence or addition of one or more other features, integrals, elements, steps, components, functions, or groups thereof.

[0072] When an element is referred to as “connected,” “coupled,” “responding,” or a variation thereof relative to another element, it may be directly connected, coupled, or responding to another element, or there may be an intermediate element present.

[0073] Although the terms first, second, third, etc., may be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another. Therefore, without departing from the teachings of the inventive concept, a first element / operation in some embodiments may be referred to as a second element / operation in other embodiments.

[0074] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0075] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0076] It should be noted that the information (including but not limited to user device information, user personal information, software version information, hardware configuration information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant regions.

[0077] As mentioned in the background section, conducting comprehensive compatibility testing presents challenges in terms of efficiency and resources when dealing with diverse hardware configurations and continuously iterating software versions. Existing testing methods may struggle to efficiently identify testing priorities and optimize resource allocation when handling massive hardware-software combinations.

[0078] To improve the above situation, this disclosure provides a testing scheme to improve testing efficiency and reduce the consumption of hardware resources. It should be understood that the embodiments presented in this disclosure are intended to provide illustrative examples of the technical ideas of this disclosure, and not to limit the scope of protection of this disclosure. Those skilled in the art, after reading this disclosure, may conceive of other variations or alternatives based on the concepts of this disclosure, and these should also be considered to fall within the scope of protection of this disclosure.

[0079] Figure 1 A flowchart illustrating a test method according to an embodiment of the present disclosure is shown. Figure 1 As shown, the method includes:

[0080] In step S11, software version information and hardware configuration information are collected to build software version libraries and hardware configuration libraries respectively.

[0081] Software version information describes the specific structure and characteristics of software at different iterations or release stages. This type of information includes not only the basic identifier of the software package itself, but also its internal components, the dependencies between components, and the underlying characteristics exhibited by its binary files when the software runs in a specific hardware environment.

[0082] For example, the system package manager can be used to query the list of installed software packages and their declared dependencies; at the same time, the executable file of the software can be analyzed to obtain details such as the dynamic link libraries required for its actual runtime, the supported instruction set architecture, or the Application Binary Interface (ABI) version.

[0083] Hardware configuration information describes the physical composition and interconnections of the computing system. This information can include detailed parameters, specifications, and interconnection topology of each hardware device. In one example, the model of the Central Processing Unit (CPU), the model and driver release date of the Graphics Processing Unit (GPU), the memory capacity, and the specific version number of the operating system can be obtained from the operating system. In another example, further information such as interconnections between devices, including connections via Peripheral Component Interconnect Express (PCIe) buses, memory channel configurations, or distances via Non-Uniform Memory Access (NUMA) architectures can be collected.

[0084] Regarding hardware configuration information, embodiments of this disclosure can parse the raw records obtained from various data sources, remove redundant formats, and convert them into structured data according to a preset data model. For example, script commands can be used to automatically collect hardware specifications from multiple devices, and then the cleaned data can be stored in a table in a database. This table can contain fields for uniquely identifying records, such as a CPU model field, a graphics processor model field, a memory size field, and an operating system version field.

[0085] For software version information, the declared dependency list obtained through package manager parsing can be compared with the actual runtime dependency list obtained through binary file analysis. Consistent and reliable software structure information, combined with its binary characteristics, is stored in a software repository. This repository can be managed using multiple related data tables; for example, one table records basic package information, another manages dependencies, and a dedicated table stores instruction set or application binary interface characteristics extracted from binary analysis.

[0086] For details on the specific data collection process, please refer to the possible implementation methods provided in this disclosure, which will not be elaborated here.

[0087] In step S12, based on the software version library and hardware configuration library, the compatibility between the software version and the hardware configuration is analyzed, and at least one compatible software and hardware combination scheme is selected.

[0088] In this disclosure, in order to efficiently identify solutions worthy of further verification from a massive number of potential combinations, targeted compatibility analysis can be performed based on the collection and construction of an information database.

[0089] Compatibility analysis assesses whether a specific version of software (hereinafter referred to as the target version) can run normally and stably on a given hardware configuration and achieve the expected performance. This analysis goes beyond surface parameter matching, delving into the impact of binary interfaces, instruction set support, system dependencies, and hardware topology on software operation.

[0090] For example, the analysis process can check whether the instruction set extensions targeted by the software during compilation are supported by the target CPU, or whether the versions of the dynamic link libraries that the software depends on are compatible with the library versions provided by the target operating system environment.

[0091] A software-hardware combination scheme represents the pairing relationship between a specific software version and a specific set of hardware configurations (hardware combination scheme). Subsequent testing phases will be performed on the software-hardware combination scheme. For example, for a newly released database software V2.1, analysis may reveal that it has high compatibility with hardware systems "configured with a specific model of CPU, a specific version of the graphics processor driver, and a specific capacity of memory".

[0092] Based on the data foundation built from software version libraries and hardware configuration libraries, embodiments of this disclosure can perform systematic compatibility analysis and identify at least one compatible software-hardware combination. Therefore, before investing actual testing resources, obviously incompatible or high-risk combinations can be eliminated in advance, thereby focusing on potentially usable combinations and saving physical testing resources.

[0093] Specifically, structured features can be extracted from the database. For hardware configurations, not only are their static parameters read, but the connection topology between hardware devices can also be further analyzed. For example, a graph structure can be used to represent the interconnection between multiple hardware devices (such as a central processing unit, a graphics processing unit, and a memory controller), and models such as graph neural networks (GNNs) can be used to learn the system-level features contained in this topology, generating a joint hardware representation that can integrate the original parameters and topological relationships.

[0094] For software versions, a trusted software material list can be extracted from the software version repository, such as the required instruction set, the major version number of the application binary interface, and key dependency library information. These features can then be organized into a software feature vector or matrix.

[0095] After obtaining the feature representations of the hardware and software, multi-level compatibility rule matching can be performed. First, rapid filtering using hard rules can be conducted, such as directly eliminating hardware configurations marked as having known conflicts with the software in the software version library, or checking whether the software's instruction set requirements are completely unsupported by a certain hardware configuration. Subsequently, for candidate combinations filtered through hard rules, the matching degree or similarity between their software and hardware features can be calculated. In one example, weights can be assigned to different dimensions such as instruction set matching degree and application binary interface version matching degree, and a comprehensive compatibility score can be obtained through weighted calculation.

[0096] Then, based on preset thresholds or filtering strategies (e.g., selecting the top few highest-scoring combinations), at least one software-hardware combination scheme deemed to have a high probability of compatibility can be output from all candidate "software version-hardware configuration" pairs. This serves as the initial set for the next stage of virtual testing, allowing testing resources to be focused on exploration directions more likely to succeed.

[0097] In step S13, the selected software and hardware combination scheme is tested in a virtualized test environment to obtain virtual test results;

[0098] A virtualized testing environment is a computing system instance created through software simulation, independent of the underlying physical hardware. This environment allows for the configuration of processor models, memory capacities, storage devices, operating systems, and even virtual connection topologies between devices, enabling flexible replication of software runtime scenarios with different hardware combinations. Multiple virtual machine instances can be created and run using a virtualization platform.

[0099] Then, testing is conducted in this virtualized testing environment. Within a pre-defined virtualized environment, the testing framework or scripts automatically drive and complete the software verification without manual intervention. Testing in a virtual testing environment is used to evaluate the software's functional correctness, performance, and stability under specific conditions. For example, smoke tests can be performed to check if the software's basic functions are working correctly; compatibility tests can be conducted to check the smooth interaction between the software and virtual hardware, operating system, and dependent libraries; and stress tests can be performed to apply high loads to the system to observe its response and resource consumption.

[0100] Based on one or more software and hardware combination schemes selected in step S12, these schemes can be tested in a virtualized test environment, and virtual test results can be obtained. This allows for rapid, parallel, and repeatable preliminary verification of multiple candidate schemes without occupying or interfering with actual physical equipment.

[0101] Specifically, the virtualization test environment can be dynamically configured based on the hardware characteristics of each candidate solution. For example, the joint representation vector of a hardware combination solution can be read, which encodes information such as CPU type, graphics processor model, memory size, and topology. Then, based on this information, a virtual machine instance with the corresponding virtual hardware configuration can be generated on the virtualization platform.

[0102] Next, in this virtual environment, the target version of the software can be deployed and a pre-defined test suite can be run. The test suite can cover multiple dimensions: functional tests verify whether the core features of the software work as expected; compatibility tests check for conflicts between the software and drivers, system services, etc., in the virtualized environment; and stress tests simulate high-concurrency users or large-scale data processing to evaluate the software's performance and stability boundaries. The entire testing process is controlled by an automated framework, which automatically executes test cases and collects logs and performance data.

[0103] After the test is completed, the collected raw data can be aggregated and analyzed to generate structured virtual test results. These results not only include whether each test passed or failed, but also the calculated baseline values ​​for key performance indicators.

[0104] In one example, all software and hardware combinations that successfully completed the test can be further categorized and labeled as "high-risk combinations" or "low-risk combinations" based on the magnitude of their performance baseline change relative to historical benchmark data (i.e., performance fluctuation values), providing a direct basis for subsequent physical testing and selection. Specific implementation methods are available in this disclosure and will not be elaborated upon here.

[0105] By using virtualization testing, the embodiments of this disclosure can conduct a centralized and rapid simulation verification and performance assessment of the selected solutions before investing in more costly physical testing, thereby effectively narrowing the testing scope and improving the efficiency of the overall testing process.

[0106] In step S14, based on the virtual test results, at least one target software and hardware combination scheme is selected for physical testing, and optimization suggestions are generated based on the virtual test results and the physical test results.

[0107] Based on the obtained virtual test results, at least one target software and hardware combination scheme can be selected for physical testing, and optimization suggestions can be generated based on the comparison of the two test results.

[0108] Specifically, the virtual test results are first analyzed, including the performance baseline, compatibility score, and performance fluctuation relative to historical data for each software and hardware combination. Based on this analysis, an intelligent screening strategy can be implemented to select a representative few combinations from all those that passed the virtual test as target software and hardware combinations for physical testing.

[0109] The target software-hardware combination scheme is a subset selected from the set of candidate schemes that have passed virtual testing for further verification on actual physical devices. These schemes are used to cover different testing concerns or risk levels, obtaining more comprehensive information with limited physical testing costs. For example, the schemes may include the combination that performed best in virtual testing, the combination that performed worst, and combinations with moderate or stable performance.

[0110] Once the target solution is determined, a test environment can be deployed on the corresponding real physical hardware to execute the same automated tests and collect physical test results. The physical test results and virtual test results should be aligned in terms of test items and metric types to facilitate subsequent comparison.

[0111] Physical testing is the process of deploying and running software on real, non-virtualized physical hardware to execute test cases and collect data. It captures behaviors that may not be fully simulated by a virtual environment when software interacts with real hardware drivers, firmware, physical buses, and actual thermal power consumption. In one example, the target version of the software can be installed on a physical server or workstation equipped with a specific CPU, GPU, memory modules, and operating system, running the same automated test suite as virtual testing.

[0112] Then, a thorough comparative analysis of the virtual and physical test results is conducted, generating optimization suggestions accordingly. This analysis can follow several logical paths: for example, for the same combination, if physical testing reveals functional faults or serious performance issues while virtual testing does not, the optimization suggestions could include providing detailed physical environment reproduction information to assist developers in diagnosing the problem. If the physical and virtual test results are largely consistent in performance trends, the test data can be included in a historical database to enrich the performance fluctuation calculation benchmark for subsequent version tests. If there are systematic deviations in key indicators, the optimization suggestions could point to adjusting the simulation parameters of the virtualization platform, or feeding the difference data back to the graph neural network model for iterative training to improve the accuracy of future predictions.

[0113] The embodiments disclosed herein can verify and correct the findings of virtual testing with a small amount of high-value physical testing. The resulting optimization suggestions not only help improve the current software version, but also continuously improve the accuracy and efficiency of the entire automated adaptability testing method itself.

[0114] In this embodiment, software version information and hardware configuration information are collected, and software version libraries and hardware configuration libraries are constructed respectively. Then, based on these libraries, the compatibility between software versions and hardware configurations is analyzed, and at least one compatible software-hardware combination is selected. This allows for the pre-identification of potentially compatible software-hardware combinations before testing, reducing blind testing. In a virtualized testing environment, automated testing is performed on the selected software-hardware combination, yielding virtual test results. This enables rapid and parallel simulation verification of multiple combinations without consuming actual physical resources, reducing testing costs. Based on the virtual test results, at least one target software-hardware combination is selected for physical testing, and optimization suggestions are generated based on the virtual and physical test results. This combines virtual and physical testing, covering key scenarios with fewer actual tests, and continuously optimizing testing strategies and software-hardware compatibility through comparative analysis. Therefore, this testing method improves the efficiency of multi-hardware configuration and multi-software version compatibility testing and reduces testing resource consumption.

[0115] In one possible implementation, the software version information to be collected includes:

[0116] Parse the target version of the software package to obtain the first software bill of materials;

[0117] The binary file obtained by running the target version of the software in the executable environment is parsed to obtain the binary compatibility features required for the actual operation of the software, and a second software bill of materials is obtained. The binary compatibility features include at least one of the following: actual dependencies, instruction set and application binary interface (ABI) features.

[0118] Based on the first software bill of materials and the second software bill of materials, a trusted software bill of materials for the target version is determined and stored in the software version library.

[0119] A software package is typically the basic unit for software distribution and deployment. It contains the program's source code, pre-compiled binaries, installation scripts, and metadata files that describe its dependencies.

[0120] During package parsing, the system reads and interprets structured information from the software distribution format. Different operating systems or distribution architectures typically employ specific package management formats; for example, RPM packages are used on Red Hat-based systems, while DEB packages are used on Debian-based systems. These packages contain not only executable files but also metadata files describing their contents. By parsing this metadata, the system can obtain the components declared by the package author and their dependencies.

[0121] The information obtained from parsing the software package can be saved as a first software bill of materials (SBOM). A SBOM is a structured list that details all components that make up a software version and their relationships. It can include component version information, source, and dependencies between components.

[0122] Package parsing can be performed by invoking or simulating the operating system's package manager commands. For example, the command `rpm -qa --provides` can be executed to read information from an RPM package, or the command `dpkg -l` can be executed to obtain a list of DEB packages. By parsing metadata such as spec files or control files within the package, the list of publicly declared files, required dependencies, and their version ranges can be extracted, thereby generating a first software bill of materials.

[0123] Binary compatibility features are low-level properties inherent in software executables or dynamic link libraries that determine whether they can function correctly in a hardware environment. These features typically do not change with software configuration but are determined by the compilation environment and the linked libraries. Binary compatibility features can include at least one of the following:

[0124] Actual dependencies, namely the shared library files and their specific versions loaded by the dynamic linker at runtime;

[0125] Instruction set, i.e., the central processing unit instruction set architecture and extensions targeted at the time of compilation, such as whether support for SSE4.2 or AVX-512 instructions is required;

[0126] Application binary interface (ABI) characteristics, such as symbol versions exported from shared libraries, conventions for system calls, or memory layouts of specific data structures, need to be consistent with the interfaces provided by the operating system runtime environment.

[0127] To determine the actual requirements of the software in a real-world environment, the binary files of the target version of the software can be parsed. For example, this can be done using binary analysis tools to analyze the header information, dynamic segments, and symbol tables of executable files or core dynamic libraries. This reveals the shared libraries actually linked by the software (potentially including libraries not listed in the declaration), the specific instruction set targeted by the file's compilation, and the major version numbers of the ABIs it depends on. Based on these analysis results, a second software bill of materials is generated, which more directly reflects the actual state of the software during runtime.

[0128] Subsequently, the first software bill of materials and the second software bill of materials can be cross-validated by comparing the first software bill of materials derived from the software declaration with the second software bill of materials derived from actual operation to identify any discrepancies between them.

[0129] This discrepancy could include: the binary file actually depends on a library, but that dependency is not declared in the package metadata; or, the declared version of the dependent library conflicts with the version of the library actually linked by the binary file.

[0130] If inconsistencies are found during the comparison, they can be recorded as abnormal events, and corresponding prompts and repair suggestions can be generated to indicate potential software packaging problems or missing dependencies.

[0131] Based on the first and second software bills of materials, a comprehensive judgment is made to determine the final trustworthy software bill of materials. For example, it can be verified whether declared dependencies match the actual linked libraries, and whether there are any undeclared dependencies or version conflicts. Consistent information can be directly adopted; for discrepancies, a decision can be made using rules or strategies, such as basing decisions on actual analysis results, i.e., only saving the software bill of materials that pass cross-validation, or marking them as items requiring manual review. This final trustworthy software bill of materials integrates declared information with empirical analysis, resulting in high accuracy.

[0132] In this embodiment, a first software bill of materials (BOM) is obtained by parsing the target version's software package, and the software's binary file is parsed to obtain the binary compatibility characteristics required for its actual operation, thereby obtaining a second software bill of materials (BOM). Then, based on the first and second software BOMs, a trusted software BOM for the target version is determined and stored in a software version repository. This integrates the software's theoretical declaration information with its actual operational characteristics. By comparing and synthesizing the two, it can effectively identify and correct incomplete or biased information problems that may arise from relying solely on package manager metadata. The constructed software version repository can provide high-quality compatibility characteristic data that more closely reflects the actual operational requirements of the software, laying a more reliable foundation for subsequent accurate analysis of the software-hardware compatibility relationship. This helps reduce compatibility misjudgments caused by inaccurate dependency information and improves the efficiency and accuracy of the entire automated testing process.

[0133] In another possible implementation, the software bill of materials that matches the comparison can be stored as a trusted software bill of materials in the software version library. Using a high-quality trusted software bill of materials as input can provide a more solid and accurate data foundation for subsequent software-hardware compatibility analysis, reduce false screening or omissions caused by information distortion, and thus improve the accuracy and reliability of the entire multi-hardware configuration compatibility testing process.

[0134] In one possible implementation, the method further includes:

[0135] Compare the trusted software bill of materials for the target version with the trusted software bill of materials for the previous historical version;

[0136] If the compatibility features of the target version have changed compared to the historical version, the hardware configuration library is queried based on the new constraints resulting from the change.

[0137] Hardware configuration schemes in the hardware configuration library that do not meet the new constraints are marked as risky hardware configuration schemes corresponding to the target version.

[0138] Historical versions can be one or more previously released versions of the target software prior to this update. Typically, you can choose the version immediately preceding the target version for comparison to clearly identify the impact of the most recent change.

[0139] Specifically, the trusted software bill of materials (BOM) of the target version is compared with that of the previous historical version to quantify the differences in compatibility features between the two versions. This involves examining characteristic fields such as instruction set requirements, ABI major version number, and dependency library list item by item to calculate the degree of difference or directly identify specific changes. For example, the comparison might reveal that the target version has added support for the "AVX-512" instruction set, or that its dependent "glibc" library version has been upgraded from 2.17 to 2.28.

[0140] If the comparison results indicate that the compatibility characteristics of the target version have changed compared to previous versions, then the hardware configuration library is queried based on the new constraints arising from these changes. Changes in compatibility characteristics can refer to alterations in the binary compatibility features of software as it evolves from one version to another. These changes can manifest at multiple levels; for example, a new version might require support for a newer processor instruction set, upgrade the major version number of the application binary interface it depends on, or add or remove certain critical runtime dependencies. These changes directly constitute new constraints on the software's runtime environment.

[0141] New constraints can be hardware or system environment requirements directly derived from changes. For example, "requires support for the AVX-512 instruction set" is a new constraint for the central processing unit; "requires glibc version >= 2.28" is a new constraint for the operating system environment. These conditions can be converted into database query statements.

[0142] Then, a query is performed in the hardware configuration library to find hardware combinations that do not meet the new constraints. For example, all hardware combinations configured with CPUs that do not support the AVX-512 instruction set, or all hardware combinations whose operating system versions are below a certain threshold and cannot provide the required ABI environment, are filtered out.

[0143] These retrieved hardware combinations that do not meet the new constraints are marked as risky hardware combinations corresponding to the target version. This mark can be a status tag stored in a database, used for rapid identification and filtering in subsequent steps such as compatibility rule matching.

[0144] In this embodiment, by comparing the trusted software bill of materials (BOM) of the target version with that of the previous historical version, compatibility feature changes introduced by software iterations can be identified. By querying the hardware configuration library based on the new constraints resulting from these changes, software-level changes can be mapped to specific hardware environment requirements. By marking hardware combinations in the hardware configuration library that do not meet the new constraints as risky hardware combinations, hardware settings that may become incompatible due to software upgrades can be predicted in advance. Therefore, this implementation can proactively and accurately identify compatibility risks by utilizing version difference analysis, allowing testing resources to focus more on combinations that are truly likely to cause problems. This avoids wasting virtual and physical testing resources on known incompatible or high-risk configurations, thereby further improving the targeting and efficiency of the entire automated testing process.

[0145] In one possible implementation, the analysis of the compatibility between the software version and the hardware configuration, and the selection of at least one compatible software-hardware combination scheme, includes:

[0146] Based on graph neural network (GNN), the hardware interconnection topology of each hardware combination scheme in the hardware configuration library is analyzed to generate topology-aware features corresponding to each hardware combination scheme.

[0147] The topology-aware features are fused with the original hardware parameter features to obtain hardware features;

[0148] Based on the software version library, generate software features that include binary compatibility characteristics;

[0149] By matching the software features with the hardware features using compatibility rules, at least one compatible software-hardware combination scheme is selected.

[0150] Hardware interconnect topology describes the physical connections and communication relationships between various hardware components (such as central processing unit, graphics processing unit, memory, and storage controller) within a computing system, and can characterize the path and latency characteristics of data flow. For example, the topology of a hardware combination may include a central processing unit connected to multiple graphics processing units through multiple PCIe lanes, and memory interconnected to the central processing unit and graphics processing units through specific channels, forming a complex graph structure.

[0151] Then, based on a Graph Neural Network (GNN), the interconnect topology of each hardware configuration scheme in the hardware configuration library can be analyzed. Each hardware device can be regarded as a node in a graph, and the physical connections (such as PCIe connections and memory channels) and logical relationships (such as NUMA distance) between devices can be regarded as edges, thereby constructing a topology graph representing the entire hardware system.

[0152] To learn meaningful topology-aware features from the graph, a specific graph neural network model can be implemented. For example, a model containing a three-layer Graph Attention Network (GATv2) can be constructed, with output dimensions of 128, 256, and 256 for each layer, respectively. The original hardware parameters of this model serve as input. Through multiple layers of attention convolution operations, the model learns the weights of relationships between nodes and aggregates neighborhood information. Ultimately, the interconnection features of the entire system or key devices within the system (such as PCIe connections, NUMA distance, and memory channel topology) are quantized into a 256-dimensional graph embedding vector, thus generating the topology-aware features.

[0153] Clearly, topology-aware features are high-dimensional vector representations extracted from the aforementioned hardware interconnection topology, capable of characterizing the overall structural characteristics and internal communication patterns of the system. They capture potential patterns of collaboration between components, such as whether there is a high-speed direct connection between a graphics processor and a central processing unit, or system-level attributes like non-uniform latency in memory access.

[0154] The learned topology-aware features (such as the 256-dimensional graph embedding vector mentioned above) can be fused with the original hardware parameter features of the corresponding hardware combination. For example, the topology-aware feature vector representing a specific CPU node can be concatenated with parameter vectors such as the CPU's model, clock speed, and cache size to obtain a comprehensive hardware feature vector that reflects both the individual performance of the device and its role in a specific system topology. This fused feature provides a more comprehensive and in-depth description of the hardware configuration.

[0155] Raw hardware parameters are the static specifications of each hardware component, such as the CPU's clock speed and number of cores, the GPU's computing power and video memory size, and the RAM's capacity and frequency. These parameters describe the individual performance of a single device. The dimensions of raw hardware parameters and topology-aware features can be the same, or if they are different, the dimensions of the raw hardware parameters can be converted to match those of the topology-aware features.

[0156] Furthermore, based on the software version library, binary compatibility features of the target software version can be extracted and organized into a structured software feature vector or matrix.

[0157] Through a pre-defined compatibility rule matching process, the software features are matched and calculated with the fused hardware features of each hardware combination, and at least one software-hardware combination scheme is selected as having a high probability of compatibility. For specific details, please refer to the possible implementation methods provided in this disclosure, which will not be elaborated here.

[0158] In this embodiment, a topology-aware feature is generated by analyzing the hardware interconnection topology of various hardware combinations in the hardware configuration library based on a graph neural network. This feature is then fused with the original hardware parameter features to obtain hardware features. Simultaneously, software features containing binary compatibility features are generated based on a software version library. Furthermore, the software features and hardware features are matched and filtered through compatibility rule matching, enabling a multi-dimensional, system-level compatibility assessment. This approach considers not only the individual performance parameters of hardware components but also captures the systemic impact of the interconnection structure between components on software operation, resulting in a more comprehensive and profound description of the hardware configuration. Matching these enhanced hardware features with precise software requirement features improves the accuracy and granularity of compatibility judgment, helping to more effectively filter out truly suitable software-hardware combinations from a vast number of combinations. This provides higher-quality input for subsequent virtual testing, thereby improving the accuracy and efficiency of the entire automated compatibility testing process.

[0159] In one possible implementation, the compatibility rule matching includes hard rule verification, which includes:

[0160] Remove risky hardware combinations from the hardware configuration library;

[0161] For the hardware combinations that were not eliminated, perform the following:

[0162] Determine whether the instruction set requirements in the software feature are supported by the hardware configuration corresponding to the hardware feature;

[0163] Determine whether the application binary interface (ABI) major version requirement in the software feature is compatible with the ABI version provided by the operating system environment corresponding to the hardware feature.

[0164] The core operations of hard rule validation mainly include the following two aspects:

[0165] First, solutions with hardware combinations already marked as risky in the hardware configuration library are directly removed from the current candidate list. These risk markers originate from previous analysis steps, such as during software version iteration comparisons, where certain hardware configurations were identified as failing to meet the requirements of newly introduced instruction sets or ABIs in the software.

[0166] Secondly, for the remaining hardware combinations that were not eliminated, two compatibility checks can be performed on each one. The first check includes instruction set support. The instruction sets required by the target software can be extracted from the software features; for example, it may need to support specific extended instruction sets such as "AVX2" or "SSE4.2". Next, the specific hardware configuration corresponding to the hardware features is queried to confirm whether its included CPU actually supports these instruction set requirements. If the software requires a certain instruction set that the hardware does not support, then the combination cannot pass the verification.

[0167] The second criterion is a compatibility check at the application binary interface level. This involves checking the ABI major version requirements specified in the software characteristics; for example, the software might require running on an ABI environment of "GLIBC_2.17" or higher. Then, verify the ABI version provided by the operating system environment associated with the hardware characteristics (whose version information is usually recorded as part of the hardware configuration). This compatibility condition is met only if the ABI major version provided by the operating system environment is greater than or equal to the major version required by the software. ABI major version incompatibility usually means that the library functions called by the software or the layout of the data structures used are incompatible with the system environment, leading to runtime errors.

[0168] For example, for a software version that requires the use of the "AVX-512" instruction set for accelerated computing, hard rule validation will filter out all hardware configurations equipped with older CPUs that do not support this instruction set. As another example, software built using a newer version of the GCC compiler might require an ABI version of "GLIBC_2.28," in which case hard rule validation will exclude all hardware configurations with operating systems (such as some older Linux distributions) whose system library versions are lower than this requirement.

[0169] In this embodiment, by removing risky hardware combinations marked in the hardware configuration library, known incompatible combinations can be proactively avoided from entering the subsequent time-consuming evaluation process by utilizing historical analysis conclusions. For each hardware combination that has not been removed, the system checks whether the instruction set requirements in the software features are supported by the hardware, and whether the ABI major version requirements in the software features are compatible with the ABI version provided by the operating system environment. This allows for rapid and deterministic screening from the two fundamental levels of instruction execution capability and binary interface specifications. Therefore, this hard rule verification can quickly eliminate a large number of obviously unqualified hardware combinations in the early stages of compatibility analysis, significantly reducing the size of the candidate set that needs to enter subsequent complex similarity calculations. This effectively reduces the overall computational overhead of compatibility analysis, improves screening efficiency, and ensures that the candidate solutions that pass this stage at least meet the basic necessary conditions for software operation.

[0170] In one possible implementation, the software features are matched for compatibility with the hardware features to select at least one compatible software-hardware combination, including:

[0171] Hardware combination schemes that pass the hard rule verification in the hardware configuration library are selected as candidate hardware combination schemes.

[0172] Calculate the similarity between the software features of the target version and the hardware features of each of the candidate hardware combinations;

[0173] Candidate hardware combinations that meet the similarity criteria will be considered as hardware combinations compatible with the target software version.

[0174] All hardware combinations that successfully pass the hard rule verification in the hardware configuration library are identified as candidate hardware combinations for the current round. These candidate hardware combinations meet fundamental and deterministic requirements such as instruction set support and ABI major version compatibility.

[0175] Then, the similarity between the software features of the target version and the hardware features of each candidate hardware combination is calculated. This similarity is used to quantify the degree of closeness between two feature vectors (i.e., the software feature vector and the hardware feature vector) in a multidimensional feature space. A higher similarity value generally indicates that the software requirements are more closely matched with the environmental characteristics provided by the hardware. In one implementation, a weighted cosine similarity algorithm can be used; for details, please refer to the possible implementations provided in this disclosure, which will not be elaborated here.

[0176] After determining the similarity, candidate solutions can be filtered based on preset similarity criteria. Similarity criteria are one or more preset rules or thresholds regarding similarity values ​​used to determine compatibility. For example, an absolute threshold can be set to select only solutions with similarity values ​​higher than that value; or a relative strategy can be used to select the top N solutions in terms of similarity.

[0177] For example, a dynamic threshold can be set, which can be adjusted based on the distribution of historical test data or the resource budget for the current test. All candidate hardware combinations with similarity scores that reach or exceed this threshold will be ultimately determined as hardware combinations compatible with the target software version. If the similarity condition is set to select the top few best options, then the similarity scores can be sorted in descending order, and a specified number of the top-ranked options can be selected.

[0178] In this embodiment, by using hardware combinations verified through hard rules in the hardware configuration library as candidate solutions, subsequent evaluations are ensured to focus on configurations with basic operating conditions. By calculating the similarity between the software features of the target version and the hardware features of each candidate hardware combination, a multi-dimensional quantitative evaluation of compatibility can be performed, rather than simply a binary yes / no judgment. By using candidate solutions that meet the similarity criteria as compatible solutions, a final selection decision can be made based on an adjustable and comprehensive metric. Therefore, more refined and flexible compatibility matching can be achieved on top of rapid hard filtering. It not only considers absolute compatibility requirements but also evaluates the matching degree between software and hardware across multiple feature dimensions, thereby helping to further select software and hardware combinations with better adaptability and greater performance potential from those that have passed basic verification. This provides more valuable input for subsequent virtual and physical testing, optimizing the allocation efficiency of test resources.

[0179] In one possible implementation, the similarity between the software features of the target computation version and the hardware features of each of the candidate hardware combinations includes:

[0180] Based on the weight coefficients corresponding to different feature dimensions, the similarity value between the software features and the hardware features of each candidate hardware combination scheme is calculated.

[0181] The feature dimensions include at least one of the following: instruction set features, ABI features, and dependency features.

[0182] In some possible implementations of this disclosure, the process of calculating the similarity between software features and hardware features can employ a configurable weighted evaluation strategy to quantify the degree of matching between the two in greater detail. This strategy involves the application of feature dimensions and their weight coefficients.

[0183] Feature dimensions refer to independently meaningful evaluation aspects or attribute categories considered when assessing software and hardware compatibility. Each dimension focuses on a specific perspective of compatibility. In this implementation, the feature dimensions considered mainly include instruction set features, application binary interface features, and dependency features.

[0184] Among them, instruction set features are related to the matching relationship between the processor instruction set targeted by the software compilation and the instruction set actually supported by the hardware; ABI features are related to the compatibility relationship between the binary interface specification followed by the software and the interface version provided by the operating system runtime environment; and dependency features are related to the satisfaction relationship between the library files required for the software to run and the libraries that are installed or available in the target hardware system environment.

[0185] When calculating similarity, appropriate weight coefficients can be assigned to each feature dimension involved in the calculation.

[0186] This weighting coefficient is a numerical parameter associated with a specific feature dimension, used to characterize the proportion or importance of that dimension in the overall similarity calculation. Different weighting coefficients allow the system to flexibly adjust the evaluation criteria to suit the focus of different software versions or testing scenarios.

[0187] The weighting coefficients can be preset fixed values ​​or dynamically adjusted based on the software type, version iteration frequency, or historical testing experience. For example, for scientific computing software that heavily relies on specific processor instruction set extensions for high-performance computing, a relatively high weight can be assigned to the "instruction set features" dimension; while for middleware software that relies more on the stability of specific system library versions, a higher weight can be assigned to "dependency features" or "ABI features".

[0188] It can calculate the matching degree or similarity sub-score of software features and hardware features on each feature dimension. For example, in the instruction set dimension, it can calculate the overlap between the list of instruction sets required by the software and the list of instruction sets supported by the hardware; in the ABI dimension, it can calculate the compatibility based on version number comparison; in the dependency dimension, it can check whether all the libraries required by the software exist and whether their versions meet the requirements, and calculate a satisfaction ratio.

[0189] Then, based on preset weight coefficients, the sub-scores of all dimensions are weighted and aggregated. For example, a weighted average or weighted cosine similarity formula can be used to multiply the sub-score of each dimension by its corresponding weight coefficient, and then sum (or average) all the weighted results to obtain a comprehensive similarity value between 0 and 1 (or other scales). This value comprehensively reflects the overall matching degree between software requirements and hardware configuration across multiple key dimensions, while a higher value indicates theoretically better compatibility potential.

[0190] In this embodiment, by calculating the similarity value between software features and the hardware features of each candidate hardware combination scheme according to the weight coefficients corresponding to different feature dimensions, wherein the feature dimensions include at least one of instruction set features, ABI features, and dependency features, a configurable, multi-dimensional comprehensive compatibility quantitative assessment can be achieved. By assigning differentiated weights to different dimensions, the compatibility concerns of different types of software can be flexibly adapted, making the assessment results more targeted and instructive. Therefore, the scheme screening based on this weighted similarity value can not only identify available compatible combinations, but also help to prioritize those hardware configuration schemes that match better in the dimensions most valued by the software, thereby potentially bringing better runtime performance or higher stability. This allows subsequent testing and verification to focus on more promising candidate targets, improving the intelligence and result quality of the entire compatibility testing process.

[0191] In one possible implementation, performing tests on the selected software-hardware combination scheme in a virtualized testing environment includes:

[0192] Based on the hardware characteristics of each selected software and hardware combination scheme, configure the corresponding virtualization test environment;

[0193] In the virtualized testing environment, virtual tests are run for the target version of the software, and virtual test results are generated.

[0194] As mentioned earlier, a virtualized testing environment refers to a flexibly customizable computer system simulation instance created through software virtualization technology. It can simulate a complete operating environment with specific hardware configurations (such as a specific model of central processing unit, a certain amount of memory, and a specific type of graphics processor) and software stacks (such as a specific version of operating system), providing an isolated and reproducible testing sandbox for software.

[0195] Correspondingly, virtual testing, conducted within the aforementioned virtualized environment, involves a series of verification activities performed on the deployed target software using test scripts or programs. These tests are used to evaluate the software's behavior and performance from different dimensions, and may specifically include functional testing, compatibility testing, and stress testing.

[0196] Functional testing verifies whether the core features and business logic of the software execute correctly as expected; compatibility testing focuses on checking whether there are conflicts or anomalies between the software and system components, drivers and dependent libraries in the virtual environment; stress testing evaluates the software's performance, stability and resource consumption by simulating high load or extreme usage scenarios.

[0197] When performing virtual testing, the corresponding virtualization test environment can be dynamically configured based on the hardware characteristics of each selected software-hardware combination. These hardware characteristics include key hardware parameters and topology information for the solution. Based on this information, the virtualization management platform's interface is invoked to automatically create or adjust the configuration parameters of virtual machine instances. For example, the CPU parameters of the virtual machine are set according to the CPU model and number of cores specified in the hardware characteristics; virtual memory is allocated according to the memory size; and whether GPU virtualization (such as using GPU passthrough or virtual GPU technology) is enabled is determined based on the presence of a specific graphics processor. In this way, each selected hardware combination can obtain a highly realistic virtual test environment.

[0198] In a configured virtualized testing environment, the target version of the software can be automatically deployed and a pre-set, comprehensive automated test suite can be run. This test suite covers multiple aspects, including the aforementioned functional testing, compatibility testing, and stress testing. The entire testing process is driven by an automation framework, automatically executing test cases, monitoring the software's running status, and collecting system logs and performance counter data.

[0199] Finally, the testing framework aggregates, analyzes, and formats all collected raw data to generate structured virtual test results. These results not only include the pass / fail status of each test case but also calculate a comprehensive compatibility score and extract key performance baseline metrics (such as average frame rate, maximum memory usage, and peak CPU utilization). These results can be saved in report form and associated with specific software and hardware combinations, providing direct evidence for subsequent physical testing selection and comparative analysis.

[0200] In this embodiment, by configuring corresponding virtualized test environments based on the hardware characteristics of each selected software-hardware combination scheme, a highly approximate simulation platform can be quickly and accurately constructed for each candidate scheme. By running virtual tests (including functional tests, compatibility tests, and stress tests) against the target software version in the virtualized test environment, the correctness, adaptability, and robustness of the software in the simulated environment can be systematically verified in multiple dimensions. By generating virtual test results containing compatibility scores and performance baselines, test findings can be transformed into quantifiable and comparable structured data. Therefore, leveraging the flexibility and replicability of virtualization technology, multiple potential compatible schemes selected in the early stages can be efficiently, parallelly, and standardized for preliminary verification and performance assessment without the need for actual physical hardware. This significantly narrows the scope requiring more costly physical testing phases and allows for the early detection of potential software defects or compatibility issues, greatly improving the efficiency and resource utilization of the entire compatibility testing process.

[0201] In one possible implementation, the step of selecting at least one target software-hardware combination scheme for physical testing based on the virtual test results includes:

[0202] The test results in the virtual test results are compared with historical virtual test data to calculate the performance fluctuation value of each software and hardware combination scheme.

[0203] Based on the performance fluctuation value, a target solution is determined from the software and hardware combination scheme, and the target solution covers different performance ranges;

[0204] The target scheme is defined as at least one target software and hardware combination scheme for physical testing.

[0205] The performance metrics obtained from the virtual test results are compared with historical virtual test data stored in the database. For each software and hardware combination that has completed the virtual test, the deviation of its key performance metrics relative to the historical baseline value is calculated, i.e., the performance fluctuation value. The performance fluctuation value is used to quantify the degree of change of the current test result relative to the historical baseline data, and is usually expressed as a percentage or absolute difference to represent the magnitude of performance improvement or decline. For example, the percentage change in the current test's graphics rendering frame rate compared to the frame rate obtained by the previous version of the software under the same virtual hardware configuration can be calculated.

[0206] Based on the calculated performance fluctuation values, all solutions can be analyzed and grouped to determine the final target solution, ensuring that the selected set of solutions covers different performance ranges. For example, based on the distribution of performance fluctuation values, several typical groups can be identified, including those with the most significant performance improvements, the most obvious performance degradation, and relatively stable performance. Then, at least one representative solution can be selected from each group. This selection strategy helps to gain a comprehensive understanding of the actual performance spectrum of the new software version on different types of hardware configurations through limited physical testing.

[0207] These identified target solutions will be used as at least one target software and hardware combination solution that requires physical testing in the next step, and will be used for subsequent actual hardware device testing.

[0208] In this embodiment, by comparing virtual test results with historical virtual test data to calculate the performance fluctuation values ​​of each software-hardware combination scheme, the current test can be quantitatively evaluated within the context of version iteration, identifying trends in software behavior. By determining target schemes covering different performance ranges from the schemes based on performance fluctuation values, it ensures that the physical test samples are representative and diverse, rather than blindly selected. By identifying the target schemes as the software-hardware combination schemes for physical testing, the findings of virtual testing can be effectively transformed into precise physical verification instructions. Therefore, the data generated by virtual testing can guide the precise allocation of physical testing resources, ensuring that the maximum information gain regarding the performance of the new software version on different hardware configurations is obtained at the lowest physical testing cost, thereby significantly improving the efficiency of the entire testing process and the comprehensiveness of the conclusions.

[0209] In one possible implementation, determining the target solution from the software-hardware combination scheme based on the performance fluctuation value includes:

[0210] Based on whether the performance fluctuation value exceeds a preset threshold, the corresponding software and hardware combination schemes are marked as high-risk combinations and low-risk combinations;

[0211] From the schemes marked as high-risk combinations, select the scheme with the best performance as the first target scheme, and select the scheme with the worst performance as the second target scheme.

[0212] From the options marked as low-risk combinations, select at least one option as the third target option.

[0213] In the aforementioned implementation, the specific strategy for determining the target scheme based on performance fluctuation values ​​can be further refined to ensure that the selected samples can effectively reveal the behavioral boundaries and trends of the software on different hardware.

[0214] A classification threshold can be set based on the magnitude of performance fluctuation. This threshold can be determined according to the actual situation. For example, solutions with an absolute performance fluctuation exceeding 15% can be classified as "high-risk combinations," which represent significant performance gains or losses that may result from software version iterations; while solutions with fluctuation values ​​below this threshold can be classified as "low-risk combinations," which represent relatively stable performance adaptation.

[0215] Based on the classification, representative solutions are selected from different categories. For example, from the "high-risk combination," the solution with the most significant performance improvement can be selected as the "first target solution" to verify the optimal potential performance release; simultaneously, the solution with the most significant performance degradation can be selected as the "second target solution" to identify the most severe performance regression issues. From the "low-risk combination," a solution can be randomly selected or based on other strategies (such as the prevalence of hardware configurations) as the "third target solution" to represent the mainstream adaptation situation with stable performance. Through this sampling method from different risk and performance ranges, the final set of target solutions can construct a multi-dimensional observation matrix.

[0216] Then, this set of selected target schemes is determined as the target software and hardware combination schemes for entering the physical testing phase, thereby transforming the quantitative findings in the virtual environment into precise action instructions to guide actual physical verification.

[0217] In this embodiment, by comparing virtual test results with historical data to calculate performance fluctuation values, the performance change spectrum brought about by the new software version can be quantitatively identified. By determining target solutions from different performance ranges based on performance fluctuation values, limited physical testing resources can be focused on the most informationally valuable high-risk scenarios and stable benchmark scenarios. By identifying this target solution as the physical testing object, the analysis conclusions of virtual testing can be efficiently transformed into actual verification tasks. Thus, a decision-making process from large-scale virtual simulation to precise physical verification with small samples is achieved. The breadth analysis of virtual testing is used to intelligently define the depth boundary of physical testing, thereby ensuring a comprehensive evaluation of software compatibility while greatly optimizing the investment of physical testing resources and significantly improving the overall efficiency of testing.

[0218] In one possible implementation, generating optimization suggestions based on virtual test results and physical test results includes at least one of the following:

[0219] For the same software and hardware combination scheme, if the physical test results indicate that there is a problem with the software, a diagnostic report containing information on the problem reproduction environment will be generated.

[0220] For the same software and hardware combination scheme, if the physical test results are consistent with the virtual test results, the test data obtained from the test is stored to facilitate subsequent test calculations of performance fluctuation values.

[0221] For the same software and hardware combination scheme, if the physical test results and virtual test results are inconsistent, suggestions for adjusting virtualization platform parameters are generated, and / or the graph neural network is iteratively updated based on the differences between the physical test results and virtual test results.

[0222] After obtaining the physical and virtual test results, the virtual and physical test results of the same software and hardware combination scheme can be compared item by item, and different processing logic can be executed according to the comparison conclusion. Specifically:

[0223] If the physical test results indicate that the software has problems, a diagnostic report can be generated based on a combination of solutions that clearly indicate functional failures or performance issues in the software. The diagnostic report is a document containing detailed information about the problem reproduction environment, including a list of specific hardware configurations, operating system and driver versions, software deployment paths, and the test steps that triggered the problem. This information is used to assist developers in accurately locating and fixing defects.

[0224] If the physical test results are consistent with the virtual test results, and if the physical test results and virtual test results are consistent in terms of functional status and core performance trends, and the calculated performance indicators change more than the historical baseline value than the preset performance change threshold, then a data storage operation is performed to store the detailed performance data obtained in this test into the historical database to update the performance baseline under this hardware configuration, so that subsequent tests can calculate performance fluctuation values ​​based on the latest data.

[0225] If the physical test results and virtual test results are inconsistent, and if there are inconsistencies or systematic deviations in key indicators, then the simulation of the virtual simulation environment under this configuration is determined to have optimizable differences. Two types of suggestions may be generated accordingly:

[0226] First, we generate suggestions for adjusting virtualization platform parameters, namely, suggestions for improving virtual machine configurations (such as CPU emulation mode and device emulation accuracy) to improve simulation fidelity.

[0227] Secondly, the physical test results are used as real label data, which, together with the corresponding virtual test results and hardware features, constitute training data. This data is used to iteratively update the graph neural network model that has undergone compatibility analysis in the early stage, so that the model's future predictions are closer to physical reality.

[0228] In this embodiment, by generating a diagnostic report containing complete information on the reproducible environment for scenarios where problems are discovered during physical testing, the testing and development / debugging phases can be efficiently integrated, accelerating the repair of software defects. By storing test data based on performance change thresholds to update historical benchmarks when virtual and physical test results are consistent and performance changes are significant, performance fluctuation analysis is always based on the latest valid data. Furthermore, by generating virtual platform parameter adjustment suggestions or triggering model iteration updates when test results are inconsistent, the testing system itself can continuously optimize the accuracy of simulation and prediction. Therefore, not only can specific optimization suggestions be provided for the current testing cycle, but also the long-term accuracy and reliability of the entire automated adaptability testing system can be continuously improved through data accumulation and model learning.

[0229] The following describes in detail several specific embodiments provided in this disclosure with reference to the accompanying drawings.

[0230] Figure 2 A flowchart illustrating a test method according to an embodiment of the present disclosure is shown. Figure 2 As shown, the method includes: obtaining basic data from the hardware configuration library and the software configuration library, then analyzing it in the AI ​​core layer; after entering the AI ​​core layer, two tasks are processed in parallel:

[0231] A hardware knowledge graph is built based on a hardware configuration library. This involves analyzing the hardware interconnect topology and generating topology-aware features through a graph neural network (GNN) to model the parameters, specifications, and connection relationships of hardware devices.

[0232] A software dependency graph is constructed based on a software configuration library, which involves parsing the software version and generating software features containing information such as binary compatibility characteristics, in order to analyze the internal structure and dependencies of the software.

[0233] Subsequently, the two are comprehensively analyzed through pre-set compatibility rule matching (including hard rule verification and similarity calculation), thereby outputting an adapted hardware model (i.e., the selected software and hardware combination scheme). This model will serve as the configuration basis for the virtualization testing phase, where the selected scheme is tested in a virtual environment. After the test is deployed and run in the virtual environment, the performance indicators obtained from the test are compared with the preset threshold to determine whether they exceed the threshold. The corresponding test scheme is then marked as a high-risk specimen (i.e., a high-risk combination) or a normal version (i.e., a low-risk combination), and the target scheme is selected.

[0234] Then, the physical testing scheduling phase begins. For the selected target solutions, compatibility testing, stress testing, and performance testing are sequentially performed on actual physical devices, and a physical test report is generated. Finally, the physical test report is compared and analyzed with the previous virtualization test results. This involves analyzing the differences between the virtual and physical test results, comprehensively evaluating these differences, and generating optimization suggestions based on these differences. These suggestions may include problem diagnosis reports, virtual platform parameter adjustments, or model iteration recommendations to guide improvements in software and hardware compatibility and adjustments to testing strategies.

[0235] Figure 3 A flowchart illustrating hardware data acquisition according to an embodiment of this disclosure is shown. Figure 3 As shown, the hardware data acquisition system is responsible for comprehensively acquiring hardware configuration information from the target computing device. This system operates through four parallel subsystems: the memory subsystem uses PowerShell scripts to obtain basic memory information; the GPU subsystem similarly uses PowerShell to obtain basic graphics processor information; the CPU subsystem uses PowerShell to obtain basic central processing unit (CPU) characteristics; and the operating system (OS) information subsystem uses PowerShell to obtain basic operating system characteristics. The raw data collected by each subsystem is then aggregated and sent to the data cleaning stage, where redundant formats are removed, data standards are unified, and normalization is performed. The cleaned data undergoes structured storage operations, transforming it into records with consistent fields and types according to a preset data model, and is finally persistently stored in the hardware configuration repository, providing a structured hardware configuration data foundation for subsequent compatibility analysis and testing.

[0236] Figure 4 A flowchart illustrating software data acquisition according to an embodiment of this disclosure is shown. Figure 4 In the software version information collection and processing flow, after collection begins, the form of the target software package is used to determine its nature: if it is a source code package, the package manager parsing step is executed, which involves parsing the package metadata to extract its declared dependencies and obtain a file list; if it is not a source code package (i.e., a binary file), the binary analysis step is executed, which involves analyzing the executable file to monitor the dependencies required for the actual operation of the software and extracting its Application Binary Interface (ABI) characteristics. An initial SBOM can be obtained in both environments.

[0237] Cross-validation is then performed, comparing the declaration information parsed from the package manager in the initial SBOM with the actual runtime requirements obtained from binary analysis. If the validation fails, repair suggestions for package declaration errors are generated; if the validation passes, a trusted SBOM is generated.

[0238] The process then proceeds with version comparison, comparing the current version of the SBOM with historical versions, focusing on whether its ABI characteristics have changed. If the ABI has not changed, the process ends directly; if the ABI has changed, this version is marked as a major update, and the hardware configuration library is referenced and queried based on the new constraints resulting from this change, thereby determining and inputting the hardware combination scheme that needs to be avoided.

[0239] Figure 5 A flowchart illustrating a software and hardware combination scheme selected according to an embodiment of the present disclosure is shown. Figure 5 As shown, software data is collected for the new software version. This collected data is processed, organized, and stored in a software database (i.e., a software version repository). Next, a SBOM comparison is performed, comparing the newly generated Trusted Software Material List (SBOM) with historical versions. Branches are generated based on the comparison results: if significant changes in compatibility characteristics are found, the software version is marked as a risky version, and the hardware database is queried based on the new constraints to determine the hardware configurations that need to be avoided; if the changes are not significant, it is treated as a normal version and the subsequent process continues. Simultaneously, key information is extracted from the software database to generate a software feature matrix, which includes binary compatibility characteristics such as instruction sets, ABI, and dependencies.

[0240] Hardware data is collected for the target hardware environment. This hardware data is used to build a hardware database (i.e., a hardware configuration library). Further, based on the hardware devices (such as CPU, GPU, memory, and OS) and their connections in the library, a topology graph is constructed. Each hardware component is treated as a node, and their interconnections as edges, forming a system-level topology graph. Along the hardware data processing path, a graph neural network (GNN) is used to encode the constructed hardware topology graph, learning topology-aware features that reflect the system's interconnections. Subsequently, these topology-aware features are concatenated with the original static parameters of each hardware device to form a comprehensive hardware group joint representation describing the hardware configuration.

[0241] Finally, the software feature matrix and the joint representation of hardware groups are input into the compatibility rule matching module. This module performs multi-level analysis, including hard rule verification and similarity calculation, to select highly compatible solutions from all hardware combinations and generate test hardware group configurations (the selected software and hardware combination solutions), providing clear input targets for subsequent virtualization testing.

[0242] Figure 6 A flowchart illustrating compatibility rule matching according to an embodiment of this disclosure is shown. Figure 6As shown, software feature vectors extracted from the software version control library (i.e., software features containing binary compatibility features) and hardware grouping features obtained from the hardware configuration library and GNN analysis (i.e., joint hardware grouping representations that integrate topology-aware features and original parameters) are loaded synchronously. Then, a hard rule check is performed. If the check fails, the hardware group is directly discarded (i.e., removed from the candidate set); if the check passes, the fine-grained evaluation stage is entered, where software similarity is calculated, that is, the similarity value between the software features and each hardware feature is calculated based on the weight coefficients.

[0243] Based on the calculated similarity values, all hardware combinations that pass the hard check are sorted into compatible hardware groups. Then, a similarity threshold (0.8 in the diagram) is set for filtering: if the similarity of a hardware group does not meet this threshold (i.e., similarity ≤ 0.8), it is discarded; if it meets the threshold (i.e., similarity > 0.8), it is used as a test group. This group represents at least one compatible software and hardware combination scheme selected for subsequent virtual testing, and then proceeds to the next stage of virtualization test environment configuration.

[0244] Figure 7 A flowchart illustrating a test method according to an embodiment of the present disclosure is shown. Figure 7 As shown, the performance data obtained from testing the new software version in a virtual simulation environment is first compared with historical data (i.e., historical virtual test data) stored in the database to calculate performance fluctuations and obtain performance fluctuation values. Then, based on a preset threshold (15% as an example in the figure), a judgment is made: if the performance fluctuation is greater than 15%, the software and hardware combination scheme is marked as a high-risk group; then, the direction of the fluctuation is further determined: if it is a performance improvement, the test group with the largest improvement is selected as the representative within this group; if it is a performance decline, the test group with the worst performance is selected as the representative. If the performance fluctuation is no more than 15%, the combination scheme is marked as a low-risk group, and a performance-balanced group is randomly selected as the representative from this group. Finally, these selected representative high-risk and low-risk group schemes together constitute the target set for subsequent physical testing.

[0245] Figure 8 A block diagram of a testing apparatus according to an embodiment of the present disclosure is shown, such as Figure 8 As shown, the image processing device 20 includes:

[0246] According to another aspect of this disclosure, a testing apparatus is provided, comprising:

[0247] Module 21 is used to collect software version information and hardware configuration information to build software version repositories and hardware configuration repositories respectively.

[0248] The filtering module 22 is used to analyze the compatibility between software versions and hardware configurations based on the software version library and hardware configuration library, and to filter out at least one compatible software and hardware combination scheme.

[0249] Virtual testing module 23 is used to perform tests on the selected software and hardware combination schemes in a virtualized testing environment and obtain virtual test results;

[0250] The physical testing module 24 is used to select at least one target software and hardware combination scheme for physical testing based on the virtual test results, and generate optimization suggestions based on the virtual test results and the physical test results.

[0251] In one possible implementation, the building module 21 is configured to:

[0252] Parse the target version of the software package to obtain the first software bill of materials;

[0253] The binary file obtained by running the target version of the software in the executable environment is parsed to obtain the binary compatibility features required for the actual operation of the software, and a second software bill of materials is obtained. The binary compatibility features include at least one of the following: actual dependencies, instruction set and application binary interface (ABI) features.

[0254] Based on the first software bill of materials and the second software bill of materials, a trusted software bill of materials for the target version is determined and stored in the software version library.

[0255] In one possible implementation, the apparatus further includes a risk combination determination module, for:

[0256] Compare the trusted software bill of materials for the target version with the trusted software bill of materials for the previous historical version;

[0257] If the compatibility features of the target version have changed compared to the historical version, the hardware configuration library is queried based on the new constraints resulting from the change.

[0258] Hardware configuration schemes in the hardware configuration library that do not meet the new constraints are marked as risky hardware configuration schemes corresponding to the target version.

[0259] In one possible implementation, the filtering module 22 is used to:

[0260] Based on graph neural network (GNN), the hardware interconnection topology of each hardware combination scheme in the hardware configuration library is analyzed to generate topology-aware features corresponding to each hardware combination scheme.

[0261] The topology-aware features are fused with the original hardware parameter features to obtain hardware features;

[0262] Based on the software version library, generate software features that include binary compatibility characteristics;

[0263] By matching the software features with the hardware features using compatibility rules, at least one compatible software-hardware combination scheme is selected.

[0264] In one possible implementation, the compatibility rule matching includes hard rule verification, which includes:

[0265] Remove risky hardware combinations from the hardware configuration library;

[0266] For the hardware combinations that were not eliminated, perform the following:

[0267] Determine whether the instruction set requirements in the software feature are supported by the hardware configuration corresponding to the hardware feature;

[0268] Determine whether the application binary interface (ABI) major version requirement in the software feature is compatible with the ABI version provided by the operating system environment corresponding to the hardware feature.

[0269] In one possible implementation, the filtering module 22 is used to:

[0270] Hardware combination schemes that pass the hard rule verification in the hardware configuration library are selected as candidate hardware combination schemes.

[0271] Calculate the similarity between the software features of the target version and the hardware features of each of the candidate hardware combinations;

[0272] Candidate hardware combinations that meet the similarity criteria will be considered as hardware combinations compatible with the target software version.

[0273] In one possible implementation, the filtering module 22 is used to:

[0274] Based on the weight coefficients corresponding to different feature dimensions, the similarity value between the software features and the hardware features of each candidate hardware combination scheme is calculated.

[0275] The feature dimensions include at least one of the following: instruction set features, ABI features, and dependency features.

[0276] In one possible implementation, the virtual test module 23 is used for:

[0277] Based on the hardware characteristics of each selected software and hardware combination scheme, configure the corresponding virtualization test environment;

[0278] In the virtualized testing environment, virtual tests are run for the target version of the software, and virtual test results are generated.

[0279] In one possible implementation, the physical testing module 24 is used for:

[0280] The test results in the virtual test results are compared with historical virtual test data to calculate the performance fluctuation value of each software and hardware combination scheme.

[0281] Based on the performance fluctuation value, a target solution is determined from the software and hardware combination scheme, and the target solution covers different performance ranges;

[0282] The target scheme is defined as at least one target software and hardware combination scheme for physical testing.

[0283] In one possible implementation, the physical testing module 24 is used for:

[0284] Based on whether the performance fluctuation value exceeds a preset threshold, the corresponding software and hardware combination schemes are marked as high-risk combinations and low-risk combinations;

[0285] From the schemes marked as high-risk combinations, select the scheme with the best performance as the first target scheme, and select the scheme with the worst performance as the second target scheme.

[0286] From the options marked as low-risk combinations, select at least one option as the third target option.

[0287] In one possible implementation, the physical testing module 24 is configured to perform at least one of the following:

[0288] For the same software and hardware combination scheme, if the physical test results indicate that there is a problem with the software, a diagnostic report containing information on the problem reproduction environment will be generated.

[0289] For the same software and hardware combination scheme, if the physical test results are consistent with the virtual test results, the test data obtained from the test is stored to facilitate subsequent test calculations of performance fluctuation values.

[0290] For the same software and hardware combination scheme, if the physical test results and virtual test results are inconsistent, suggestions for adjusting virtualization platform parameters are generated, and / or the graph neural network is iteratively updated based on the differences between the physical test results and virtual test results.

[0291] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0292] This disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0293] This disclosure also provides a non-volatile computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0294] This disclosure also provides a computer program product, including a computer program or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program, when executed by a processor, implements the steps of the above method.

[0295] Figure 9 This is a block diagram illustrating a testing apparatus according to an exemplary embodiment. For example, apparatus 1900 may be provided as a server or terminal device. (Refer to...) Figure 9 The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0296] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output interface 1958 (I / O interface). Device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM macOS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0297] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of the device 1900 to perform the above-described method.

[0298] Computer-readable storage media can be tangible devices capable of holding and storing programs / instructions used by instruction execution devices. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0299] The computer program (or computer-readable program instructions) described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage medium in the respective computing / processing device.

[0300] The computer program (or computer program instructions) used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions to implement various aspects of this disclosure.

[0301] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0302] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0303] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0304] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0305] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A testing method, characterized in that, include: Collect software version information and hardware configuration information, and build software version repositories and hardware configuration repositories respectively; Based on the software version library and hardware configuration library, analyze the compatibility between software versions and hardware configurations, and select at least one compatible software and hardware combination scheme. In a virtualized testing environment, the selected software and hardware combination schemes are tested to obtain virtual test results; Based on the virtual test results, at least one target software and hardware combination scheme is selected for physical testing, and optimization suggestions are generated based on the virtual test results and physical test results.

2. The method according to claim 1, characterized in that, The software version information includes: Parse the target version of the software package to obtain the first software bill of materials; The binary file obtained by running the target version of the software in the executable environment is parsed to obtain the binary compatibility features required for the actual operation of the software, and a second software bill of materials is obtained. The binary compatibility features include at least one of the following: actual dependencies, instruction set and application binary interface (ABI) features. Based on the first software bill of materials and the second software bill of materials, a trusted software bill of materials for the target version is determined and stored in the software version library.

3. The method according to claim 2, characterized in that, The method further includes: Compare the trusted software bill of materials for the target version with the trusted software bill of materials for the previous historical version; If the compatibility features of the target version have changed compared to the historical version, the hardware configuration library is queried based on the new constraints resulting from the change. Hardware configuration schemes in the hardware configuration library that do not meet the new constraints are marked as risky hardware configuration schemes corresponding to the target version.

4. The method according to claim 1, characterized in that, The analysis examines the compatibility between the software version and the hardware configuration, and identifies at least one compatible software-hardware combination, including: Based on graph neural network (GNN), the hardware interconnection topology of each hardware combination scheme in the hardware configuration library is analyzed to generate topology-aware features corresponding to each hardware combination scheme. The topology-aware features are fused with the original hardware parameter features to obtain hardware features; Based on the software version library, generate software features that include binary compatibility characteristics; By matching the software features with the hardware features using compatibility rules, at least one compatible software-hardware combination scheme is selected.

5. The method according to claim 4, characterized in that, The compatibility rule matching includes hard rule verification, which includes: Remove risky hardware combinations from the hardware configuration library; For the hardware combinations that were not eliminated, perform the following: Determine whether the instruction set requirements in the software feature are supported by the hardware configuration corresponding to the hardware feature; Determine whether the application binary interface (ABI) major version requirement in the software feature is compatible with the ABI version provided by the operating system environment corresponding to the hardware feature.

6. The method according to claim 5, characterized in that, The software features are matched with the hardware features for compatibility, and at least one compatible software-hardware combination scheme is selected, including: Hardware combination schemes that pass the hard rule verification in the hardware configuration library are selected as candidate hardware combination schemes. Calculate the similarity between the software features of the target version and the hardware features of each of the candidate hardware combinations; Candidate hardware combinations that meet the similarity criteria will be considered as hardware combinations compatible with the target software version.

7. The method according to claim 6, characterized in that, The similarity between the software features of the target version and the hardware features of each of the candidate hardware combinations includes: Based on the weight coefficients corresponding to different feature dimensions, the similarity value between the software features and the hardware features of each candidate hardware combination scheme is calculated. The feature dimensions include at least one of the following: instruction set features, ABI features, and dependency features.

8. The method according to claim 1, characterized in that, The step of performing tests on the selected software and hardware combination schemes in a virtualized testing environment includes: Based on the hardware characteristics of each selected software and hardware combination scheme, configure the corresponding virtualization test environment; In the virtualized testing environment, virtual tests are run for the target version of the software, and virtual test results are generated.

9. The method according to claim 1, characterized in that, Based on the virtual test results, at least one target software and hardware combination scheme is selected for physical testing, including: The test results in the virtual test results are compared with historical virtual test data to calculate the performance fluctuation value of each software and hardware combination scheme. Based on the performance fluctuation value, a target solution is determined from the software and hardware combination scheme, and the target solution covers different performance ranges; The target scheme is defined as at least one target software and hardware combination scheme for physical testing.

10. The method according to claim 9, characterized in that, The step of determining the target solution from the software and hardware combination scheme based on the performance fluctuation value includes: Based on whether the performance fluctuation value exceeds a preset threshold, the corresponding software and hardware combination schemes are marked as high-risk combinations and low-risk combinations; From the schemes marked as high-risk combinations, select the scheme with the best performance as the first target scheme, and select the scheme with the worst performance as the second target scheme. From the options marked as low-risk combinations, select at least one option as the third target option.

11. The method according to claim 1, characterized in that, The process of generating optimization suggestions based on virtual test results and physical test results includes at least one of the following: For the same software and hardware combination scheme, if the physical test results indicate that there is a problem with the software, a diagnostic report containing information on the problem reproduction environment will be generated. For the same software and hardware combination scheme, if the physical test results are consistent with the virtual test results, the test data obtained from the test is stored to facilitate subsequent test calculations of performance fluctuation values. For the same software and hardware combination scheme, if the physical test results and virtual test results are inconsistent, suggestions for adjusting virtualization platform parameters are generated, and / or the graph neural network is iteratively updated based on the differences between the physical test results and virtual test results.

12. A testing device, characterized in that, include: The module is used to collect software version information and hardware configuration information, and to build software version repositories and hardware configuration repositories respectively. The filtering module is used to analyze the compatibility between software versions and hardware configurations based on the software version library and hardware configuration library, and to filter out at least one compatible software and hardware combination scheme. The virtual testing module is used to perform tests on the selected software and hardware combination schemes in a virtualized testing environment and obtain virtual test results. The physical testing module is used to select at least one target software and hardware combination scheme for physical testing based on the virtual test results, and generate optimization suggestions based on the virtual test results and physical test results.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 11.

14. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program, or a non-volatile computer-readable storage medium carrying a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images