Test method, server and storage medium

Abstract

The invention provides a testing method, a server and a storage medium, and relates to the technical field of testing. The test method is applied to a server, a control module and a tested system module are deployed on the server, communication connection is established between the control module and the tested system module, the current operation environment of the server is a pre-silicon environment, and the test method can comprise the following steps: adopting the control module to test a target main program of a test tool through the target main program; test task information is sent to the tested system module, and the main program is used for shielding the peripheral interface and providing simulation test data; the tested system module is adopted to perform test operation based on the test task information to obtain a test result, and the test result is sent to the control module; and receiving a test result by using a control module. The control module deployed on the server based on the pre-silicon environment and the tested system model are tested, so that the dependence on an external control terminal is thoroughly eliminated, the server of the pre-silicon environment is independently operated and tested, and a result is obtained.

Description

Technical Field

[0001] This application relates to the field of testing technology, and more specifically, to a testing method, a server, and a storage medium. Background Technology

[0002] Pre-silicon (PS) refers to all stages and related work in the chip manufacturing process, from chip design to tape-out. The PS environment is primarily used in the chip design verification stage, before the chip is actually manufactured; it typically runs in a simulator or simulation environment. The PS environment is usually a relatively independent verification platform, limited by factors such as hardware cost, space, and verification complexity, making it difficult to configure multiple auxiliary and external devices as easily as the post-silicon environment. Therefore, testing solutions for the PS environment have become a hot research topic.

[0003] In related technologies, existing testing solutions for pre-silicon environments mainly focus on optimizing simulation algorithms and tools, adopting a B (test control terminal) / S (test server) architecture. The control mode consists of a web control terminal and a SUT server. Task configuration and test execution are required on the web page of the test control terminal, and it is not possible to run tests and obtain results independently on the server side. Summary of the Invention

[0004] The purpose of this application is to provide a testing method, server, and storage medium to address the shortcomings of the prior art and solve the aforementioned technical problems.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide a testing method applied to a server, wherein a control module and a system under test module are deployed on the server, a communication connection is established between the control module and the system under test module, and the current operating environment of the server is a pre-silicon environment, the method comprising: The control module sends test task information to the system module under test through the target main program of the test tool. The main program is used to shield peripheral interfaces and provide simulated test data. The system module under test is tested based on the test task information to obtain test results, and the test results are sent to the control module. The control module receives the test results.

[0006] Optionally, before the control module sends test task information to the system under test module through the target main program of the test tool, the method further includes: Using the control module, add environment files corresponding to the architecture to the development toolkit directory of the testing tool, and modify the peripheral interface file code of the testing tool to shield the peripheral interface, thereby obtaining the target main program.

[0007] Optionally, the step of performing test operations on the tested system module based on the test task information, obtaining test results, and sending the test results to the control module includes: The tested system module is started by a preset script; The system under test module is initialized, and a communication connection is established between the system under test module and the control module. Create an initial test task, and fill in the task parameters of the initial test task according to the test task information to obtain the test task; The test module of the system under test is tested according to the test task, the test results are obtained, and the test results are sent to the control module.

[0008] Optionally, initializing the tested system module includes performing the following steps respectively: The system eliminates preset checks and conditional branches by directly executing the startup command of the tested system module through the preset script, eliminates decompression overhead by using pre-installed software packages, and locates the program code of the target programming language using preset system variables; wherein, the system variables are system variables used to specify the search path of the executable program; Monitor whether the ports of the system module under test are available, and obtain the port detection results; Record the port detection results and the startup status of the server during the startup process of the system module under test.

[0009] Optionally, establishing a communication connection between the system under test module and the control module includes: A full-duplex communication connection is established between the system under test module and the control module through a preset interface; The control module sends a formatted message to the system under test module via the full-duplex communication connection. The formatted message includes the main program requirements of the test tool.

[0010] Optionally, the method further includes: A check request is sent to the system module under test to check the device list, evaluation type, and version information of the system module under test, and to determine whether the system module under test is in a ready state.

[0011] Optionally, the step of creating an initial test task and filling in the task parameters of the initial test task according to the test task information to obtain the test task includes: The control module sends a test task creation request to the system under test module based on preset program code, so that the system under test module creates the initial test task. The control module receives a test task identifier sent by the system under test module in response to the test task creation request. The control module configures at least one of the following for the initial test task based on the test task information: test task name, software parameters of the system module under test, hardware parameters of the system module under test, Internet protocol address, test task identifier, local area network Internet protocol, and Internet protocol of the system module under test.

[0012] Optionally, performing test operations on the system module under test according to the test task, obtaining the test results, and sending the test results to the control module includes: The control module is used to initiate the test task to perform test operations on the system module under test, and the execution status of the test task is monitored. If the execution status of the test task indicates that the test task has been completed, then the test result is obtained.

[0013] Optionally, obtaining the test results includes: Obtain the test result identifier returned by the system module under test; The test result identifier is queried at preset intervals using a polling mechanism, and the test result is obtained. Download the test results according to the test result identifier.

[0014] Optionally, after performing test operations on the system module under test according to the test task and obtaining the test results, the method further includes: The process of the control module is cleaned up and its resources are released. Key events during the cleanup process are recorded, including process lookup results and cleanup status.

[0015] Optionally, the method further includes: The operating system type and file system type are detected to determine that the current running environment is a pre-silicon environment. Analyze the target module in the target main program to locate the disk space verification logic; The disk space verification logic is based on a preset data serialization format configuration to inject negative values ​​to trigger the verification, thereby circumventing physical disk limitations.

[0016] Secondly, embodiments of this application also provide a server, including: a memory and a processor, wherein the memory stores a computer program executable by the processor, and the processor executes the computer program to implement the method described in any of the first aspects above.

[0017] Thirdly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when read and executed, implements the method described in any of the first aspects above.

[0018] The beneficial effects of this application are as follows: This application provides a testing method applied to a server. The server is equipped with a control module and a system under test (SUT) module. A communication connection is established between the control module and the SUT module. The server's current operating environment is a pre-silicon environment. This testing method may include: the control module sending test task information to the SUT module through the target main program of a testing tool, wherein the main program is used to shield peripheral interfaces and provide simulated test data; the SUT module performing test operations based on the test task information, obtaining test results, and sending the test results to the control module; and the control module receiving the test results. Testing based on the control module and SUT model deployed on the server in the pre-silicon environment completely eliminates the dependence on external control terminals, enabling the server in the pre-silicon environment to run tests independently and obtain results. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic diagram of the architecture of a testing system provided in an embodiment of this application; Figure 2 A flowchart illustrating a testing method provided in this application embodiment. Figure 1 ; Figure 3 A flowchart illustrating a testing method provided in this application embodiment. Figure 2 ; Figure 4 A flowchart illustrating a testing method provided in this application embodiment. Figure 3 ; Figure 5 A flowchart illustrating a testing method provided in this application embodiment. Figure 4 ; Figure 6A flowchart illustrating a testing method provided in this application embodiment. Figure 5 ; Figure 7 A flowchart illustrating a testing method provided in this application embodiment. Figure 6 ; Figure 8 A flowchart illustrating a testing method provided in this application embodiment. Figure 7 ; Figure 9 A flowchart illustrating a testing method provided in this application embodiment. Figure 8 ; Figure 10 This application provides a schematic diagram of the structure of a server according to an embodiment of the present application. Figure 11 This is a schematic diagram of a server deployment architecture provided in an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0022] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0023] In the description of this application, it should be noted that if the terms "upper", "lower", etc. appear to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship that the product of this application is usually placed in, it is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0024] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] It should be noted that, where there is no conflict, the features in the embodiments of this application can be combined with each other.

[0026] This application provides a testing system. Figure 1 This application provides a schematic diagram of the architecture of a testing system, as shown in the embodiments. Figure 1 As shown, the test system includes a control module and a system under test (SUT). The test system runs on a server, and the server's operating system can be Linux or Windows.

[0027] The control module includes an instrument data and management module, a controller, a graphical user interface module (front-end GUI module), an embedded database, an energy efficiency load set, and a communication module; the system under test module includes a load implementation and operation module, an energy efficiency load set, a communication module, and a processing module (SUT dispatch).

[0028] It should be noted that instrument data is provided through virtual devices. The instrument data and management module interacts with the controller via function calls and sockets. The controller manages load testing tasks, load pushing and operation monitoring, benchmark testing, and test report management. The graphical user interface module communicates with the controller via HTTP (Hypertext Transfer Protocol) and WebSocket (a network communication protocol). Data can be transferred between the controller and the embedded database. The controller interacts with the energy efficiency load set on the control module via file push and with the communication module on the control module via function calls.

[0029] The graphical user interface module refers to the module corresponding to the graphical interface in the original testing tool (BenchSEE). In fact, the graphical user interface module does not have a visual graphical interface.

[0030] In this embodiment, the processing module on the system under test is responsible for load operation and scheduling, as well as information collection. The processing module interacts with the communication module and the energy efficiency load set on the system under test through function calls. The processing module interacts with the load instance running module through sockets. The energy efficiency load set can create instances and run instances on the load instance running module.

[0031] The energy efficiency load set on the control module and the energy efficiency load set on the system under test module interact via SFTP (a network protocol) or a local area network sharing protocol; the communication module on the control module and the communication module on the system under test module transmit information via socket.

[0032] This application also provides a testing method applied to a server. The server has a control module and a system under test module deployed on it. A communication connection is established between the control module and the system under test module. Specifically, the control module and the system under test module communicate via the WebSocket protocol. The current operating environment of the server is a pre-silicon environment.

[0033] A RESTful API is a web service interface based on the REST (Representational State Transfer) architectural style. Its core idea is to abstract entities in the network as "resources" and manipulate these resources using standard HTTP methods. Operations on resources include retrieving, creating, modifying, and deleting, which correspond to the HTTP protocol's GET, POST, PUT, and DELETE methods. It provides a way for clients and servers to communicate through standardized operations and resource access patterns. Specifically, the RESTful API interface is built using JakartaWebSocket (a standard Java application programming interface) and FastJSON (an open-source library).

[0034] Web interfaces based on the HTTP protocol are a common type of web service interface. Besides these, there are other service interfaces based on other protocols, such as the common WebSocket, WebService, Dubbo (a high-performance remote procedure call framework), and MQTT (a lightweight message queue / message passing protocol).

[0035] In this embodiment, a RESTful API interface is called to drive the modified test tool project, integrating the control module and the system under test module into the server. The control logic bypasses the graphical interface and web page, configuring test preconditions, controlling test startup, and collecting test results through the program and web interface.

[0036] The following explains a test method provided in an embodiment of this application.

[0037] Figure 2 A flowchart illustrating a testing method provided in this application embodiment. Figure 1 ,like Figure 2 As shown, this method, applied to the aforementioned server, may include: S101. The control module sends test task information to the system module under test through the target main program of the test tool.

[0038] The main program is used to shield peripheral interfaces and provide simulated test data.

[0039] In this embodiment, a control module is used to directly call a preset interface via code to provide service incentives and send test task information, including test instructions, to the system module under test. This differs from related technologies where users manually operate through a web browser, representing an interactive testing method based on a web browser.

[0040] It should be noted that, compared with the client and server testing architecture in related technologies, the embodiment of this application is a single-machine closed-loop architecture, which is tested through the control module and the system under test module deployed on the server. The system under test module can also be referred to as SUT (System Under Test).

[0041] S102. Perform test operations on the system module under test based on the test task information, obtain the test results, and send the test results to the control module.

[0042] In some implementations, a control module initiates a test task via HTTP PUT (a request method), performs test operations on the system module under test based on the test task information, monitors the execution status of the test operations in real time, and obtains the test results after the test operations are completed.

[0043] If the test operation is successfully completed, the test result can be a test report; if the test operation fails, the test result can be an error feedback.

[0044] S103, The control module receives the test results.

[0045] It is worth noting that the control module uses a programmatic interface to collect and store test results. If the test result is a test report, it supports storage in multiple formats, including, for example, PDF (Portable Document Format), TXT (Text File), and EXCEL (spreadsheet).

[0046] In addition, the test report supports multiple report formats, ensuring the completeness and readability of the test reports. Users can choose the appropriate test report format according to their needs, improving the practicality of the test reports and allowing for the display of test results.

[0047] In this embodiment, the test client on the control module integrates the Log4j2 logging framework and the Hutool utility library, employing a multi-threaded collaborative mechanism to automate the orchestration of test tasks. This mechanism dynamically manages concurrent tasks through a thread pool, intelligently assigning and executing test cases in parallel based on preset strategies (such as task type, priority, or resource dependencies). Each worker thread independently runs a specific test task, and state synchronization and result aggregation are achieved through shared data structures or message queues. This design significantly improves test execution throughput and effectively shortens the overall test cycle.

[0048] In summary, this application provides a testing method applied to a server. The server deploys a control module and a system under test (SUT) module. A communication connection is established between the control module and the SUT module. The server's current operating environment is a pre-silicon environment. This testing method includes: the control module sending test task information to the SUT module via the target main program of a testing tool. The main program is used to shield peripheral interfaces and provide simulated test data. The SUT module performs test operations based on the test task information, obtains test results, and sends the test results to the control module. The control module receives the test results. Testing based on a control module and SUT model deployed on a server in a pre-silicon environment completely eliminates dependence on external control terminals, enabling the server in the pre-silicon environment to run tests independently and obtain results.

[0049] Moreover, compared to existing technologies that rely on browser-based user operations for testing (which involves page rendering, DOM (Document Object Model) parsing, and event triggering), this application effectively avoids the significant latency and resource consumption issues caused by browser rendering in related technologies. More importantly, by reducing unnecessary steps from the stimulus source (script) to the server (such as the browser engine in the evaluation control terminal), it significantly reduces signal attenuation along the transmission path, thereby ensuring high consistency of test stimulus parameters and accuracy of measurement results.

[0050] This application significantly shortens the testing time in the pre-silicon environment and improves testing efficiency while ensuring the accuracy of test results. It also reduces the overall testing time while maintaining minimal changes in test results.

[0051] Optionally, before the control module sends test task information to the system under test module through the target main program of the test tool, the following steps are also included: Add the corresponding architecture environment file to the development toolkit directory of the testing tool, and modify the peripheral interface file code of the testing tool to hide the peripheral interface, thus obtaining the target main program.

[0052] Alternatively, the testing tool can be BenchSEE (Benchmark of Server Energy Efficiency).

[0053] In this embodiment of the application, the development kit (JDK) runtime environment is modified. Specifically, JDK environment files corresponding to the architecture are added to the development kit (JDK) directory to adapt to the hardware architecture.

[0054] Furthermore, since the pre-silicon system is only concerned with raw performance and does not care about power consumption and temperature data, and in this embodiment, in order to modify the test architecture, the host computer and external measurement equipment were eliminated, leaving only the server. This application modifies the peripheral interface file code of BenchSEE, and comments out the code of the power meter and thermometer coupled to the BenchSEE main program, so that the server does not necessarily need to physically connect to the power meter and thermometer. This allows the physical power meter and thermometer to be transformed into virtual devices using code, which is verified by full version / trimmed version experimental data.

[0055] Of course, the power consumption meter and thermometer mentioned above are just examples, and other external devices can also be used. This application does not impose specific limitations on them.

[0056] In summary, eliminating reliance on external devices (such as power meters / thermometers) avoids manual intervention, improves testing efficiency, and ensures the accuracy and efficiency of task configuration. It can operate independently in the pre-silicon environment and autonomously complete testing tasks, enhancing the flexibility and operability of pre-silicon testing.

[0057] Optionally, Figure 3 A flowchart illustrating a testing method provided in this application embodiment. Figure 2 ,like Figure 3 As shown, the process in S102 above, which involves performing test operations on the tested system module based on test task information, obtaining test results, and sending the test results to the control module, may include: S201. Start the system module under test through a preset script.

[0058] Among them, after the system module under test is started, the startup status of the system module under test can be dynamically monitored.

[0059] S202. Initialize the system module under test and establish a communication connection between the system module under test and the control module.

[0060] In some implementations, the target main program in the control module is used to initialize the system module under test, specifically including service initialization, port listening of the system module under test, log recording during the startup process of the system module under test, and status feedback.

[0061] It should be noted that the communication between the system under test module and the control module is established by using the target main program in the control module as real-time communication, and the control module is used to simulate client behavior.

[0062] S203. Create an initial test task and fill in the task parameters of the initial test task according to the test task information to obtain the test task.

[0063] The initial test task is created by the module under test using an HTTP POST request (an important method for sending data to a server).

[0064] In addition, some task parameters of the initial test task are populated based on the test task information, and some parameters are dynamically populated.

[0065] S204. Perform test operations on the system module under test according to the test task, obtain the test results, and send the test results to the control module.

[0066] In some implementations, the control module initiates a test task via an HTTP PUT request, performs test operations on the system module under test based on the test task information, monitors the execution status in real time, obtains the test results, and the system module under test sends the test results to the control module, which then receives the test results.

[0067] Optionally, Figure 4 A flowchart illustrating a testing method provided in this application embodiment. Figure 3 ,like Figure 4 As shown, the process of initializing the system module under test in S202 above may include performing the following steps respectively: S301, Eliminate preset checks and conditional branches: Directly execute the startup command of the tested system module through preset scripts, eliminate decompression overhead by using pre-installed software packages, and locate the program code of the target programming language by using preset system variables.

[0068] System variables are used to specify the search path for executable programs. The default script is a shell script (a scripting program used to automate the execution of operating system commands). The pre-installed package can be a JRE (Java Runtime Environment).

[0069] It's worth noting that by launching the shell script using ProcessBuilder (a Java class used to create and manage operating system processes), execution permissions are dynamically set. This completely skips the environment initialization process, directly calling the java command from the system environment variables, eliminating all environment preparation steps and reducing pre-startup operations by 90%.

[0070] In some implementations, the control module eliminates preset checks and conditional branches, rendering no conditional judgments and directly executing the startup command via a shell script. This avoids the overhead of prediction and file system operations. The system's already installed JRE is used directly, bypassing compressed file decompression operations and eliminating decompression overhead. A preset system variable is used to locate the target programming language's code, saving the file search process. This preset system variable can be PATH (a system environment variable), and the target programming language can be Java.

[0071] S302. Monitor whether the port of the system module under test is available and obtain the port detection result.

[0072] In some implementations, the control module uses a socket (a core interface for inter-process communication in computer networks) to poll and detect whether the port of the system module under test is available, obtains the port detection result, and then uses an exponential backoff algorithm to verify service availability.

[0073] The port of the system module under test can be CP_PORT:8080.

[0074] In this embodiment of the application, it is necessary to ensure that the system module under test starts successfully. If the port of the system module under test is unavailable, it will continue to retry until the system module under test starts successfully.

[0075] S303. Record the port detection results during the startup process of the tested system module and the startup status of the server.

[0076] In some implementations, a control module is used to record key events during the startup process of the system module under test. These key events include port detection results and server startup status. The port detection results and server startup status facilitate debugging and monitoring.

[0077] It should be noted that the control module is used to initialize the module under test, that is, the control module executes the above S301 to S303 processes.

[0078] In summary, the control plane is initiated via scripts, and the status of the tested system modules is dynamically monitored. Socket polling is used to check port availability, ensuring successful server startup. This automated mechanism avoids manual intervention and improves testing efficiency.

[0079] Optionally, Figure 5 A flowchart illustrating a testing method provided in this application embodiment. Figure 4 ,like Figure 5 As shown, establishing a communication connection between the tested system module and the control module in S202 includes: S401. Establish a full-duplex communication connection between the system module under test and the control module through a preset interface.

[0080] The default interface is WebSocketContainer. Full-duplex communication can be WebSocket (a full-duplex communication protocol based on TCP (Transmission Control Protocol)).

[0081] In some implementations, the control module establishes a WebSocket connection between the system under test module and the control module through a WebSocketContainer.

[0082] S402. The control module sends formatted messages to the system module under test via a full-duplex communication connection.

[0083] The formatted messages include: the main program requirements of the testing tool.

[0084] It should be noted that the main program requirements of the testing tool refer to the requirements of the original Benchsee testing program. For example, it can be nav=2×tamp=xxx&lang=zh_cn, using a control module to simulate client behavior.

[0085] In summary, the tested system module and the control module establish real-time communication via the WebSocket protocol to simulate client behavior. This bidirectional communication mechanism ensures real-time monitoring and status feedback during task execution.

[0086] Optionally, the method further includes: Send an inspection request to the system module under test to check the device list, evaluation type, and version information of the system module under test, and determine whether the system module under test is in a ready state.

[0087] In this embodiment, the inspection request is an HTTP request, which is sent by the control module to the system module under test. The HTTP request satisfies the requirements of the original BenchSEE test program.

[0088] Optionally, Figure 6 A flowchart illustrating a testing method provided in this application embodiment. Figure 5 ,like Figure 6 As shown, the process of creating an initial test task in S203 above, and filling in the task parameters of the initial test task according to the test task information to obtain the test task, may include: S501. The control module sends a test task creation request to the system module under test based on the preset program code, so that the system module under test can create an initial test task.

[0089] In some implementations, a control module is used to send a test task creation request to the system under test module via an HTTP POST request based on preset program code. Correspondingly, the system under test module receives the test task creation request and creates an initial test task according to the test task creation request.

[0090] S502. A control module is used to receive the test task identifier sent by the module under test for creating a test task.

[0091] Specifically, a control module receives a JSON (JavaScript Object Notation, a lightweight data interchange format) response from the module under test during the creation of the initial test task, which contains a test task identifier. This test task identifier can be referred to as the task ID (Identity).

[0092] S503. The control module configures at least one of the following for the initial test task based on the test task information: test task name, software parameters of the system module under test, hardware parameters of the system module under test, Internet protocol address, test task identifier, local area network Internet protocol, and Internet protocol of the system module under test.

[0093] It should be noted that after configuring the initial test task, the test task is obtained.

[0094] In some implementations, the control module dynamically obtains the local IP address (Internet Protocol) via an HTTP GET request. This IP address includes the server's Ethernet address and loopback address. This action is temporarily set via the shell during the startup and initialization phase of the system under test module and configured in the initial test task to ensure that the test task can correctly identify the local network environment.

[0095] In addition, configure the connection information of the system under test in the initial test task, including IP address, username and password, to ensure successful connection to the system under test.

[0096] It should be noted that the task is created via an HTTP request and the local IP address is dynamically obtained, automatically configuring the connection information of the tested system modules. This automated process ensures the accuracy and efficiency of task configuration.

[0097] In this embodiment, task parameters are dynamically generated based on test task information, supporting the generation of random task names to avoid conflicts. Simultaneously, task configurations are automatically populated based on hardware information (such as processor, memory, disk, etc.) to ensure the flexibility and environmental adaptability of task parameters. The load level is also set at this stage. The specific implementation is as follows: The JSON configuration file contains the following parameters: taskSutList and sutIp.

[0098] taskSutList: Contains detailed configuration information for tasks, such as task name and IP address. The code reads taskSutList from the config_stream.json file, combines it with myIp obtained from the configuration file and taskId obtained from the hardware information, to form a complete task configuration.

[0099] sutIp (hardware parameters of the system module under test): The code reads the taskSutList from the config_stream.json file, and then sets the sutIp of each taskSut to the SUT_IP obtained from the hardware information in the loop.

[0100] The parameters implemented through a Java program are as follows: taskName (Test Task Name): Generates a random combination of 5 letters and 5 numbers using the generateRandomCode method, then shuffles the result using a shuffle algorithm to generate a random task name, which is then set as the task name.

[0101] This feature supports the random generation of task names, avoiding name conflicts. The random names consist of a combination of letters and numbers, and a shuffling algorithm ensures their randomness, further improving the reliability of task management.

[0102] The software parameters of the tested system module (e.g., OS, CPU, architecture, cores, processor, chips, frequency, memory, disk): After extracting information such as OS and CPUInfo from the configuration file through code, this information is checked for null values. If valid values ​​exist, they are set into the corresponding fields in the task configuration. CPUInfo is a nested JSON object containing information such as CPU, architecture, cores, processor, chips, and frequency.

[0103] taskId (Test Task Identifier): A new SUT task is created by sending a POST request to the server. The server returns a task ID, which is extracted and set into the taskId field in the task configuration.

[0104] myIp (Local Area Network Internet Protocol): Obtain the local IP address by sending a GET request to the server and set it in the myIp field of the task configuration.

[0105] sutIp (implementation method of the system module under test): By looping through taskSutList, the sutIp of each taskSut is set to the SUT_IP obtained from the hardware information to ensure that the obtained SUT_IP corresponds one-to-one with the parameters in the JSON configuration file.

[0106] Among them, JSON Schema dynamic binding technology (such as config_aes.json) is used, combined with runtime hardware identification algorithms (CPU architecture / memory topology) and environment-aware configuration adapters (OS kernel version detection) to dynamically generate task parameters.

[0107] It's worth noting that the hardware parameters in the task configuration are dynamically populated based on system information (such as CPU, memory, and disk) obtained at runtime. Automatic filling of information such as CPU name, architecture, number of cores, number of threads, and frequency is supported, enhancing the flexibility and environmental adaptability of task configuration. This dynamic configuration mechanism allows the system to adapt to different testing environments, improving system flexibility. Random task name generation avoids name conflicts, enhancing the reliability of task management.

[0108] It should be noted that the original program of the testing tool requires certain fields to be filled in, which are all compiled into JSON templates. If the information cannot be extracted automatically, the corresponding field information is extracted from the JSON template.

[0109] Optionally, Figure 7 A flowchart illustrating a testing method provided in this application embodiment. Figure 6 ,like Figure 7 As shown, the process in S204 above, which involves performing test operations on the system module under test according to the test task, obtaining test results, and sending the test results to the control module, may include: S601. The control module is used to start the test task to perform test operations on the system module under test, and the execution status of the test task is monitored.

[0110] In some implementations, a control module initiates a test task via an HTTP PUT request, performs test operations on the system module under test according to the test task, and monitors the execution status of the test task. During the test task initiation process, a JSON response indicating the execution status of the test task is returned.

[0111] It should be noted that the test task is started via an HTTP PUT request, and the task execution status is monitored in real time. During the task startup process, the system module under test will return a JSON response indicating the task execution status, ensuring the transparency and controllability of the task execution.

[0112] S602. If the execution status of the test task indicates that the test task has been completed, then obtain the test result.

[0113] If the execution status of the test task indicates successful completion, a test report is obtained; if the execution status indicates failure, an error message is obtained. The test result is either a test report or an error message. The system module under test can send test results to the control module, and the control module can receive the test results.

[0114] In summary, the bidirectional communication mechanism between the control module and the system under test (SUT) module ensures real-time monitoring and status feedback during test task execution, improving the transparency of test task execution. Real-time monitoring of the test task execution status allows users to understand the test task's progress promptly, enhancing system controllability.

[0115] Optionally, Figure 8 A flowchart illustrating a testing method provided in this application embodiment. Figure 7 ,like Figure 8 As shown, obtaining the test results in S602 above may include: S701. Obtain the test result identifier returned by the module under test.

[0116] S702. Through a polling mechanism, the test result identifier is queried at preset intervals, and the test result is obtained.

[0117] The test results are presented in the test report, and the test result identifier is presented in the test report identifier.

[0118] In some implementations, a control module polling mechanism is used to periodically check the test report identifier returned by the system under test (SUT) module, ensuring that the test report is retrieved immediately after its generation. During the polling process, the SUT module returns a JSON response containing the test report identifier. This mechanism ensures minimal delay in test report retrieval and improves the efficiency of test report management.

[0119] S703. Download the test results according to the test result identifier.

[0120] In this embodiment of the application, after downloading the test results, the control module can save the test results to a local file.

[0121] Optionally, after performing test operations on the system module under test according to the test task in S204 and obtaining the test results, the method may further include: Clean up the control module's processes and release resources, and record key events during the cleanup process, including process lookup results and cleanup status.

[0122] In some implementations, a PID-level process tracing algorithm is used to automatically clean up the control module's processes and release resources. The control module's processes are located using the `ps -ef` command via command-line tools, and then forcibly terminated using the `kill -9` command to release resources. A resource reclamation priority queue is used to find and forcibly terminate target processes, ensuring the reliability of resource release.

[0123] The automatic cleanup mechanism ensures the timely release of system resources, preventing resource hoarding and performance degradation. Furthermore, recording key events during the cleanup process ensures the reliability of resource release.

[0124] This application provides a testing method that supports dynamically acquiring and populating task parameters, including hardware information (such as CPU, memory, disk, etc.) and network information (such as local IP address). This dynamic configuration mechanism enhances the system's flexibility and adaptability. A polling mechanism periodically checks the test result identifier of the test task and can download the test results to a local file, supporting multiple report formats to ensure the completeness and readability of the test results. After the test task is completed, the process of the control module on the control terminal is automatically cleaned up, releasing system resources. The process of the control module can be located and forcibly terminated using a command-line tool to ensure the reliability of resource release. The control module establishes real-time communication with the system module under test via the WebSocket protocol to simulate client behavior. The task execution status is monitored in real time via HTTP requests to ensure the transparency and controllability of the test task execution.

[0125] Moreover, the reconstructed test system can independently complete the entire process of stimulus signal generation, test execution, result recording and data storage on the server, significantly improving the autonomy and robustness of the test environment.

[0126] Furthermore, this application simplifies test configuration and operation processes, reduces the professional skills required of testers, enabling even non-professional testers to quickly get started and efficiently complete pre-silicon testing tasks, minimizing the impact of human error on test results. An automated configuration tool is designed to generate an adapted code framework and complete environment deployment with a single click.

[0127] In practical applications, experimental verification shows that, while maintaining the incentive score fluctuation within ±1%, the average execution time of the test method provided in this application embodiment is reduced by 70%, effectively solving the resource bottleneck problem in high-frequency testing scenarios. Furthermore, the single-machine closed-loop architecture supports multi-threaded concurrent testing, further improving the throughput of large-scale testing tasks and providing an efficient and accurate quantitative testing platform for server energy efficiency evaluation.

[0128] It should be noted that the processes S801 to S803 described below propose a dynamic bypass technique for disk space detection, which can solve the problem in existing technologies where the absence of a physical disk in the pre-silicon environment prevents test tasks from executing normally. By automatically detecting the pre-silicon environment, reverse engineering and exploiting vulnerabilities, and configuring dynamic parameters, S801 to S803 can effectively circumvent disk space limitations, ensuring the smooth execution of test tasks in the pre-silicon environment.

[0129] Optionally, Figure 9 A flowchart illustrating a testing method provided in this application embodiment. Figure 8 ,like Figure 9 As shown, the method may further include: S801 detects the operating system type and file system type, and determines that the current running environment is a pre-silicon environment.

[0130] It should be noted that in the current pre-silicon initramfs file system environment, the disk space status needs to be manually detected and confirmed, which is a cumbersome and error-prone process (such as the system prompting "SUT disk remaining space detection failed").

[0131] In some implementations, the S801 process is executed automatically during the initialization phase. Pre-defined detection code detects the operating system type, file system mount point, and memory usage characteristics (checking whether / proc / mounts contains the initramfs identifier), determines that the current running environment is a pre-silicon environment, and triggers a dynamic bypass mechanism.

[0132] Compared to existing technologies that rely on manual environmental assessment, leading to high test interruption rates (e.g., manual adjustments are needed when disk space is insufficient), this application employs an environment-aware algorithm (e.g., through System.getProperty("os.name") and file system scanning) to achieve fully automated identification and response, eliminating the need for manual intervention. This increases the success rate of test tasks in pre-silicon environments to over 95%, avoids a test failure rate of over 30%, and reduces environment configuration time by over 70%.

[0133] S802. Analyze the target module in the target main program and locate the disk space verification logic.

[0134] The target main program is the main program of BenchSEE, and the target module can be the model that is mainly responsible for generating and running the load pressure. For example, the target module is SutServiceImpl.class.

[0135] In some implementations, decompilation techniques are used to analyze SutServiceImpl.class in the BenchSEE server to locate the disk space verification logic.

[0136] S803: Based on the disk space verification logic, negative values ​​are injected through the preset data serialization format configuration to circumvent physical disk limitations.

[0137] The default data serialization format can be YAML.

[0138] It is worth noting that existing technologies cannot bypass disk space detection mechanisms (such as BenchSEE requiring 400MB of disk space), making testing infeasible in pre-silicon environments. In this application embodiment, by exploiting the negative value verification defect of sutWorkspaceMB in the key method (SutServiceImpl.isDiskEnough()), a negative value trigger is injected using YAML configuration, thereby successfully circumventing the physical disk limitation.

[0139] The `sutWorkspaceMB` method is used to calculate physical disk space limit detection. Since this is done before the silicon process, there is no physical hard drive, so this software method for detecting the lack of a physical hard drive must be bypassed. The `sutWorkspaceMB` method is in a closed-source JAR file. For example, using YAML configuration to inject a negative value trigger can be: `bench.sut-workspace-MB: -100) triggers Assert.isTrue(needDiskSpace ≤ 0)`.

[0140] Compared to existing technologies that rely solely on static configurations (such as modifying physical disks), which lack flexibility, this solution introduces a dynamic parameter injection engine. This engine supports runtime modification of YAML files (such as config_aes.json), allowing parameter values ​​to be dynamically adjusted according to testing needs (e.g., configurable negative value ranges). This enables a 50% reduction in test task execution time and a 40% reduction in resource consumption in a pre-silicon environment. Furthermore, experimental verification through the methodology framework shows that performance deviation is controlled within ±1%.

[0141] It is worth noting that the dynamic bypass technology for disk space detection in S801 to S803 described above has the following technical effects: Automated Environment Awareness and Adaptation: Precise Environment Identification: The system can automatically detect whether the current operating environment is a pre-silicon environment. By identifying key characteristics such as operating system type and file system type, it accurately determines whether it is in the initramfs file system. This avoids the tedious manual environment assessment and potential human error, providing accurate environmental information for subsequent disk space detection bypass operations. Intelligent Adaptation Process: Once a pre-silicon environment is identified, the system automatically triggers a dynamic disk space detection bypass mechanism. The corresponding optimization process can be initiated without manual intervention, ensuring that testing tasks can be performed smoothly in the pre-silicon environment, greatly improving the automation and efficiency of the testing process.

[0142] Flexible parameter configuration and task-driven approach: Dynamic parameter adaptation: When using YAML configuration to inject negative values ​​to circumvent disk space limitations, the system can flexibly adjust the injected parameter values ​​according to the needs of different test tasks. This dynamic parameter configuration method enables the system to adapt to various test scenarios and requirements, enhancing the system's flexibility and scalability. Task-driven execution: This technology is tightly integrated with the creation and execution of test tasks. Dynamic bypass technology for disk space detection is automatically detected and applied only when test tasks are created and executed. This ensures the system's normal operation in non-test scenarios while fully leveraging its optimization capabilities during test task execution, ensuring efficient completion of test tasks.

[0143] Efficient Resource Utilization and Performance Optimization: Reduced Resource Consumption: By accurately identifying the pre-silicon environment and dynamically bypassing disk space detection, frequent test failures and duplicate tests caused by insufficient disk space are avoided, reducing resource consumption on invalid tests. Simultaneously, this technology requires no additional hardware support, further reducing the hardware resource requirements of the testing environment and improving resource utilization efficiency. Enhanced Test Performance: In the pre-silicon environment, the execution efficiency of test tasks is often limited by various factors. The dynamic bypass technology for disk space detection eliminates this potential performance bottleneck, enabling test tasks to execute more smoothly, effectively shortening test time, improving test performance, and thus accelerating the entire pre-silicon verification process.

[0144] Stable and Reliable Testing Assurance: Enhanced System Stability: This technology, through a deep understanding and utilization of server-side logic, employs secure and reliable vulnerability exploitation methods to dynamically bypass disk space detection, avoiding negative impacts on system stability. Throughout the testing process, the system maintains stable operation, reducing the occurrence of system crashes or anomalies due to technical implementation issues. Guaranteed Data Integrity: While bypassing disk space detection, the system still ensures the normal execution of test tasks and the complete collection of test data. The accuracy and completeness of test results are effectively guaranteed, providing reliable data support for subsequent analysis and decision-making.

[0145] Easy to maintain and upgrade: Centralized configuration management: Relevant parameter configurations are centralized in a YAML file, allowing testers or maintenance personnel to manage and modify them uniformly as needed. When testing requirements change or the bypass mechanism needs adjustment, only the corresponding YAML configuration file needs to be modified, without modifying a large amount of code, reducing maintenance costs and error risks. Good compatibility and scalability: This technology fully considers compatibility with existing testing systems during implementation and can be seamlessly integrated with other functional modules of the system. At the same time, its flexible architecture design also facilitates future functional expansion and upgrades, such as easily supporting new test task types or adapting to different pre-silicon environments.

[0146] The above technical effects demonstrate the advantages and value of dynamic bypass technology based on disk space detection in pre-silicon testing scenarios from multiple perspectives. This technology can effectively solve the problems caused by disk space limitations in existing pre-silicon testing, improve testing efficiency and quality, and reduce costs and risks.

[0147] It should be noted that the embodiments of this application can be extended to various hardware platforms and operating systems. By abstracting hardware interfaces and operating system interfaces, the system can adapt to different hardware and operating systems. It can also be integrated into a cloud platform to support server energy efficiency testing in a cloud environment. It can utilize the cloud platform's API (Application Programming Interface) for resource management and task scheduling, and combine with existing systems to achieve cloud testing.

[0148] This application embodiment can combine artificial intelligence technology, using machine learning algorithms to analyze test data, predict energy efficiency trends, and optimize test parameters to achieve intelligent optimization and prediction of energy efficiency. This application embodiment can also support energy efficiency testing of edge computing devices, improving device performance and energy efficiency, and optimizing system architecture to adapt to the low power consumption and high performance requirements of edge computing devices. This application embodiment can also support multi-threaded concurrent testing, further improving the throughput of large-scale test tasks. In implementation, thread pool management and task scheduling are used to achieve multi-threaded concurrent execution of test tasks.

[0149] Furthermore, embodiments of this application can also provide an intuitive operation interface and real-time monitoring functions based on the developed graphical user interface, enhancing the user experience. This application also supports multiple languages ​​to meet the needs of international users. Specifically, multilingual support is achieved through resource files and an internationalization framework. This application can employ encryption algorithms to protect data transmission and storage, and implement access control and authentication mechanisms to enhance system security and protect test data and configuration information.

[0150] Figure 10This application provides a schematic diagram of the structure of a server, as shown in the embodiment of the present application. Figure 10 As shown, the server includes: processor 101 and memory 102.

[0151] The memory 102 is used to store programs, and the processor 101 calls the programs stored in the memory 102 to execute the above method embodiments. The specific implementation and technical effects are similar, and will not be described in detail here.

[0152] Figure 11 This is a schematic diagram of a server deployment architecture provided in an embodiment of this application, as shown below. Figure 11 As shown, the server is equipped with a control module and a system under test module. The server has an eth0 Ethernet address and a lo loopback address network.

[0153] The control module interacts with the system under test through an interface. The control module sends requests to the system under test to perform tests, and the system under test can return the test results to the control module through the interface.

[0154] Optionally, this application also provides a program product, such as a computer-readable storage medium, including a program that, when executed by a processor, performs the above-described method embodiments.

[0155] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A testing method, characterized in that, Applied to a server, the server is equipped with a control module and a system under test (SUT) module, and a communication connection is established between the control module and the SUT module. The current operating environment of the server is a pre-silicon environment. The method includes: The control module sends test task information to the system module under test through the target main program of the test tool. The main program is used to shield peripheral interfaces and provide simulated test data. The system module under test is tested based on the test task information to obtain test results, and the test results are sent to the control module. The control module receives the test results.

2. The method according to claim 1, characterized in that, Before the control module sends test task information to the system under test module through the target main program of the test tool, the method further includes: Using the control module, add environment files corresponding to the architecture to the development toolkit directory of the testing tool, and modify the peripheral interface file code of the testing tool to shield the peripheral interface, thereby obtaining the target main program.

3. The method according to claim 1, characterized in that, The step of performing test operations on the tested system module based on the test task information, obtaining test results, and sending the test results to the control module includes: The tested system module is started by a preset script; The system under test module is initialized, and a communication connection is established between the system under test module and the control module. Create an initial test task, and fill in the task parameters of the initial test task according to the test task information to obtain the test task; The test module of the system under test is tested according to the test task, the test results are obtained, and the test results are sent to the control module.

4. The method according to claim 3, characterized in that, The initialization of the system module under test includes performing the following steps respectively: The system eliminates preset checks and conditional branches by directly executing the startup command of the tested system module through the preset script, eliminates decompression overhead by using pre-installed software packages, and locates the program code of the target programming language using preset system variables; wherein, the system variables are system variables used to specify the search path of the executable program; Monitor whether the ports of the system module under test are available, and obtain the port detection results; Record the port detection results and the startup status of the server during the startup process of the system module under test.

5. The method according to claim 3, characterized in that, Establishing a communication connection between the system under test module and the control module includes: A full-duplex communication connection is established between the system under test module and the control module through a preset interface; The control module sends a formatted message to the system under test module via the full-duplex communication connection. The formatted message includes the main program requirements of the test tool.

6. The method according to claim 5, characterized in that, The method further includes: A check request is sent to the system module under test to check the device list, evaluation type, and version information of the system module under test, and to determine whether the system module under test is in a ready state.

7. The method according to claim 3, characterized in that, The process of creating an initial test task and filling in the task parameters of the initial test task according to the test task information to obtain the test task includes: The control module sends a test task creation request to the system under test module based on preset program code, so that the system under test module creates the initial test task. The control module receives a test task identifier sent by the system under test module in response to the test task creation request. The control module configures at least one of the following for the initial test task based on the test task information: test task name, software parameters of the system module under test, hardware parameters of the system module under test, Internet protocol address, test task identifier, local area network Internet protocol, and Internet protocol of the system module under test.

8. The method according to claim 3, characterized in that, Perform test operations on the system module under test according to the test task, obtain the test results, and send the test results to the control module, including: The control module is used to initiate the test task to perform test operations on the system module under test, and the execution status of the test task is monitored. If the execution status of the test task indicates that the test task has been completed, then the test result is obtained.

9. The method according to claim 8, characterized in that, Obtaining the test results includes: Obtain the test result identifier returned by the system module under test; The test result identifier is queried at preset intervals using a polling mechanism, and the test result is obtained. Download the test results according to the test result identifier.

10. The method according to claim 3, characterized in that, After performing test operations on the system module under test according to the test task and obtaining the test results, the method further includes: The process of the control module is cleaned up and its resources are released. Key events during the cleanup process are recorded, including process lookup results and cleanup status.

11. The method according to any one of claims 1-10, characterized in that, The method further includes: The operating system type and file system type are detected to determine that the current running environment is a pre-silicon environment. Analyze the target module in the target main program to locate the disk space verification logic; The disk space verification logic is based on a preset data serialization format configuration to inject negative values ​​to trigger the verification, thereby circumventing physical disk limitations.

12. A server, characterized in that, include: A memory and a processor, the memory storing a computer program executable by the processor, the processor executing the computer program to implement the method according to any one of claims 1-11.

13. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when read and executed, implements the method described in any one of claims 1-11.

Images