A memory-based disc-ripplin g whole-machine test method

Abstract

The present application relates to the technical field of computer system testing, in particular to a discless whole machine testing method based on memory, comprising: a server constructs a resource pool containing a multi-system image and a testing program module; a network starts a machine to be tested, so that it loads a memory-based minimum kernel to run; a virtual execution environment and a temporary file system are established in the memory, which are independent of local storage; a module loading sequence is generated according to a testing task, corresponding modules are dynamically obtained from the server and injected into the memory for execution, and active modules are cached based on usage frequency; hardware data is collected in real time during testing, packaged into standard logs and returned to the server; and the machine to be tested is restarted after testing, so that the memory data is completely emptied. Through the memory-based diskless testing environment and the module dynamic loading caching mechanism, the method realizes fast and clean testing deployment and flexible scheduling of testing logic, and avoids the dependence of traditional testing on hard disks and the problem of data residue.

Description

Technical Field

[0001] This invention relates to the field of computer system testing technology, and in particular to a memory-based whole-system testing method without a master disk. Background Technology

[0002] Traditional whole-machine testing methods heavily rely on the local physical storage media of the machine under test (DUT), such as hard drives or solid-state drives (SSDs). Before testing, a complete operating system and a large test suite need to be installed on the DUT, or a dedicated test boot disk needs to be created. All temporary data, logs, and intermediate results generated during the test are also stored locally. After the test, tedious disk cleanup or system restore operations must be performed to eliminate the impact of residual test data on subsequent production processes. This method results in long test environment preparation and recovery times, wears down local storage devices during the testing process, and poses a security risk of data residue.

[0003] Existing testing technologies typically deploy test programs in a monolithic, static manner. During testing, regardless of the specific functionalities required by the test task, the complete test program package usually needs to be loaded into the machine under test. This not only consumes significant memory and storage resources but also solidifies the testing process, making it impossible to flexibly combine test items based on different hardware configurations or testing objectives. For scenarios requiring updates or additions to test modules, it is often necessary to rebuild and redeploy the entire test image, which is inefficient and makes it difficult to achieve fine-grained management and dynamic scheduling of test resources. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a memory-based whole-system testing method that eliminates the need for a master disk.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a memory-based whole-system testing method without master disk, comprising:

[0006] On the server side, a resource pool containing multiple operating system images and test packages is built, with each test package consisting of several independent functional modules;

[0007] The system provides boot information to the machine under test (DUT) through the network boot protocol service, enabling the DUT to obtain a minimal operating system kernel from the server and load it into memory for execution during the boot phase.

[0008] A virtual execution environment is created in memory, which provides a temporary file system independent of local storage to hold temporary data generated during the test.

[0009] A module loading sequence is generated according to the test task requirements, and the corresponding test program module is dynamically obtained from the server's resource pool according to the module loading sequence and injected into the virtual execution environment.

[0010] Establish a memory cache management mechanism to monitor the usage frequency of test program modules in memory and maintain a set of active modules according to a preset replacement strategy;

[0011] During the execution of the test task, the hardware performance data of the machine under test is captured in real time, and the hardware performance data is encapsulated into a standardized log stream and uploaded to the server;

[0012] After completing all test tasks, a clear command is sent to the device under test to restart the device and clear all temporary data in memory.

[0013] As a further aspect of the present invention, the step of providing boot information to the machine under test via the network boot protocol service, enabling the machine under test to obtain a minimized operating system kernel from the server and load it into memory for execution during the boot phase, includes:

[0014] Configure the Dynamic Host Configuration Protocol service on the server side to assign a network address to the machine under test and specify the location information of the Network Startup Protocol server.

[0015] The initial bootloader is transmitted to the machine under test via Simple File Transfer Protocol Service. The initial bootloader contains the acquisition path and verification information of the operating system kernel.

[0016] After the machine under test executes the initial bootloader, it downloads a compressed, minimal operating system kernel image file from the server. The minimal operating system kernel contains only the basic drivers and system services required to run the test.

[0017] Extract the downloaded minimal operating system kernel image file to a specified area of ​​memory, and jump to the kernel entry point to start the operating system;

[0018] During the operating system startup process, the test program package directory stored on the server is mounted via the network file system protocol, forming a virtual read-only file system view.

[0019] As a further aspect of the present invention, the step of decompressing the downloaded minimized operating system kernel image file to a designated area of ​​memory includes:

[0020] A contiguous storage space in memory is allocated as the kernel running area, the size of which is preset according to the uncompressed size of the minimized operating system kernel image file;

[0021] A streaming decompression algorithm is used to download and decompress simultaneously, and the decompressed kernel data is written into the kernel execution area in order of address offset.

[0022] During the decompression process, the checksum of the kernel data is calculated and compared with the checksum pre-calculated on the server side to verify data integrity.

[0023] After decompression and verification are completed, memory protection attributes are set to mark the kernel runtime area as an executable-only area to prevent the test program from accidentally modifying the kernel code.

[0024] As a further aspect of the present invention, the step of generating a module loading sequence according to the test task requirements, and dynamically obtaining the corresponding test program module from the server's resource pool according to the module loading sequence and injecting it into the virtual execution environment includes:

[0025] Parse the test task configuration file, extract the test items to be executed and their dependencies, and generate a module dependency graph in the form of a directed acyclic graph;

[0026] The module dependency graph is topologically sorted to obtain a linear module loading sequence, ensuring that dependent modules are loaded before the modules they depend on.

[0027] According to the order of the module loading sequence, a Hypertext Transfer Protocol request is sent to the server to obtain the binary data of the corresponding test program module;

[0028] In the memory space of the virtual execution environment, an independent load address range is allocated for each module, and address references within the module are relocated;

[0029] Register the loaded module information into the module management table, recording the module's entry point, size, and status information.

[0030] As a further aspect of the present invention, the step of initiating a Hypertext Transfer Protocol request to the server in the order of the module loading sequence to obtain the binary data of the corresponding test program module includes:

[0031] Generate a unique identifier for each test program module based on its content hash value, and use the unique identifier as the cache lookup key;

[0032] Before initiating a Hypertext Transfer Protocol request, first check if a copy of the module with the same unique identifier exists in the local memory cache;

[0033] If a valid copy exists in the memory cache, the module data in the cache is used directly, skipping the network transmission step;

[0034] If a valid copy is not found in the memory cache, a request is sent to the server, and the hash value of the received module data is verified to be consistent with the expected unique identifier.

[0035] The newly acquired module data is stored in the memory cache, and the access timestamp and reference count in the cache metadata are updated.

[0036] As a further aspect of the present invention, the establishment of a memory cache management mechanism, which monitors the usage frequency of test program modules in memory and maintains an active module set according to a preset replacement strategy, includes:

[0037] Maintain access history for each cached test program module, recording the most recent access time and the number of historical accesses;

[0038] Periodically scan the access history of all cache modules and calculate the overall popularity score of each module, which is based on a weighted combination of access time and access frequency;

[0039] Determine the amount of cache space that needs to be released based on the current available memory space and the preset memory usage threshold;

[0040] Candidate modules are selected and eliminated in order of their overall popularity score from low to high until the released cache space meets the demand.

[0041] For modules selected for elimination, if they are not currently referenced by any test task, the memory they occupy will be released immediately; otherwise, they will be marked as pending release and released after the reference is removed.

[0042] As a further aspect of the present invention, the step of capturing hardware performance data of the machine under test in real time during the execution of the test task, and encapsulating the hardware performance data into a standardized log stream and uploading it to the server, includes:

[0043] A lightweight data acquisition agent is deployed in a virtual execution environment, which obtains raw performance data by directly reading hardware performance counters and system status registers;

[0044] The collected raw performance data is preprocessed, including unit standardization, outlier filtering, and timestamp alignment.

[0045] The preprocessed performance data is encapsulated according to a preset log format. Each log entry contains a test case identifier, timestamp, performance metric type, and value.

[0046] The encapsulated log entries are streamed to the log collector on the server side using a persistent transmission protocol.

[0047] The server parses and reassembles the received log stream to restore the complete test execution trajectory and performance profile.

[0048] As a further aspect of the present invention, the preprocessing of the collected raw performance data includes unit standardization, outlier filtering, and timestamp alignment, including:

[0049] Identify the original units of the raw performance data and convert them to standard International System of Units (SI) representation according to predefined conversion rules;

[0050] An anomaly detection algorithm based on statistical distribution is applied to calculate the moving average and standard deviation of performance data values, and to filter out data points that exceed the reasonable range.

[0051] A high-precision clock source is used to stamp each data point with a monotonically increasing timestamp to ensure time consistency of data from different hardware counters.

[0052] Interpolation of timestamps is performed to compensate for time deviations caused by differences in collection frequency, generating time series data with equal intervals.

[0053] The processed performance data is associated with the corresponding test phase metadata to form a structured performance record with semantic tags.

[0054] As a further aspect of the present invention, after all test tasks are completed, a clear command is sent to the device under test (DUT) to restart the DUT and clear all temporary data in its memory, including:

[0055] The test scheduler monitors the execution status of all test tasks and triggers a cleanup process when it detects that the last test task has been completed.

[0056] Send a safe shutdown request to the machine under test, the request carrying identifiers for immediate restart and memory clearing;

[0057] After receiving a safe shutdown request, the machine under test suspends all running processes and synchronizes the log data in memory to the server.

[0058] The system calls the underlying interface to send a clear command to the memory controller, causing the memory controller to perform a memory fill operation and write all memory units into a specific pattern.

[0059] The hardware reset signal is triggered, causing the device under test to restart. During the startup process, the original data in the memory is automatically cleared by the hardware.

[0060] As a further aspect of the present invention, the step of calling the system's low-level interface to send a clear command to the memory controller, causing the memory controller to perform a memory filling operation, includes:

[0061] Obtain the address space mapping information of physical memory, determine the memory range that needs to be cleared, and exclude the basic input / output system area reserved for hardware;

[0062] Construct a special memory fill command, specifying the fill mode and target address range, wherein the fill mode is zero fill or random number fill;

[0063] The fill command is written to the command register of the memory controller using memory-mapped input / output.

[0064] Monitor the status register of the memory controller and wait for a confirmation signal that the fill operation is complete;

[0065] To verify the filling effect, randomly sample a portion of memory addresses and check if their contents have changed to the specified filling pattern to confirm that the clearing operation was successfully executed.

[0066] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0067] By constructing a virtual execution environment and temporary file system completely independent of local physical storage in the memory of the machine under test (DUT), the runtime environment, programs, and generated process data required for testing are not written to the hard drive. This eliminates the dependence of traditional testing on fixed storage media, achieving "instant startup" and "instant destruction" of the test environment. No pre-installation of the system is required before testing, and no complex erasure is needed after testing; the memory data is completely erased after the DUT restarts, eliminating the risk of data residue. This method shortens the preparation and cleanup time of the test environment and avoids wear and tear on the hard drive during testing, making it particularly suitable for rapid quality inspection and production scenarios with high requirements for data security and testing efficiency.

[0068] By dynamically generating module loading sequences based on test tasks and establishing a memory caching management strategy based on usage frequency, the system achieves on-demand, phased loading and intelligent caching of test program modules. The system only loads and executes modules necessary for the current test, and retains frequently used active modules in memory according to preset strategies. This changes the static deployment model that requires loading the entire test package at once. It reduces the peak memory usage of a single test task, allowing limited memory resources to support more complex test combinations. Simultaneously, the dynamic module loading mechanism makes test case updates, rollbacks, and combinations extremely flexible. After the server updates a module, subsequent tests can automatically obtain the new version, achieving seamless hot upgrades of testing capabilities and fine-grained resource scheduling. Attached Figure Description

[0069] Figure 1 This is a flowchart of the memory-based whole-system testing method for removing the master disk as described in this invention;

[0070] Figure 2 A flowchart for network boot and minimized kernel loading;

[0071] Figure 3 A flowchart for dynamically loading test program modules;

[0072] Figure 4 Box plot distribution of CPU utilization in test phases 1-5;

[0073] Figure 5 The curves showing the relationship between the number of samples, success rate, and confidence level during the memory filling effect verification phase. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0075] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0076] See Figure 1A centrally managed resource pool is built on the server side. This pool stores multiple operating system images customized for different hardware platforms and testing needs, as well as test program packages composed of several independent functional modules. A network boot protocol service provides boot information to the machine under test (DUT), guiding it to obtain a streamlined, minimal operating system kernel from the server during the initial boot process. This kernel is directly loaded into the DUT's memory for execution, eliminating reliance on the local hard drive. Within the in-memory operating system, a virtual execution environment isolated from local storage is created. This environment provides a temporary, non-existent file system to support necessary temporary data storage during testing. Test task execution is driven by a module loading sequence, generated according to specific test requirements. The system dynamically retrieves the corresponding test program modules from the server's resource pool on demand based on this sequence and injects them into the in-memory virtual execution environment for execution. To optimize module loading efficiency and manage limited memory resources, a memory cache management mechanism is established. This mechanism continuously monitors metrics such as the usage frequency of each test program module in memory and dynamically maintains an active module set according to a preset strategy to improve access speed. During test task execution, agents deployed within the virtual execution environment capture various hardware performance data of the machine under test in real time. This data is immediately encapsulated into standard format log streams and continuously uploaded to the server for centralized storage and analysis. Once all preset test tasks are completed, the system sends a clear command to the machine under test, triggering a restart. During this process, all temporary data in memory, including the operating system kernel, test modules, and runtime data, is completely cleared, restoring the machine under test to its original state.

[0077] See Figure 2In one embodiment of the present invention, a Dynamic Host Configuration Protocol (DHCP) service is deployed and runs on the server side. After the device under test (DUT) is powered on, the DHCP service responds to its network boot request, assigns a temporary Internet Protocol (IP) address to the DUT, and explicitly points to the address of the Network Boot Protocol (NBPPC) server providing the boot file in its reply message. In a specific implementation, the server side also runs a Simple File Transfer Protocol (SLTP) service, which stores the initial bootloader. When allocating network parameters, the DHCP service specifies the filename of the initial bootloader, and the DUT downloads the initial bootloader from the SLTP server based on this information. The initial bootloader contains the acquisition path and verification information of the operating system kernel. The acquisition path is a Uniform Resource Locator (URL), and the verification information is the hash value of the operating system kernel image file. After executing the initial bootloader, the DUT downloads a compressed, minimal operating system kernel image file from the server based on the acquisition path. The minimal operating system kernel contains only the basic drivers and system services required for running the test. The basic drivers include network card drivers and memory controller drivers, and the system services include process scheduling, memory management, and network protocol stack initialization.

[0078] In practice, the downloaded minimal operating system kernel image file is decompressed to a designated area of ​​memory. Specifically, a contiguous block of memory is allocated as the kernel runtime area. The size of the kernel runtime area is pre-set based on the uncompressed size of the minimal operating system kernel image file, with the rule being the uncompressed size plus a fixed proportion of reserved space. A streaming decompression algorithm is used to download and decompress simultaneously. This algorithm reads the network data stream, decompresses data blocks in real time, and writes the decompressed kernel data into the kernel runtime area in address offset order. During decompression, a checksum of the kernel data is calculated using a bitwise-based rolling checksum algorithm. The rolling checksum algorithm formula is:

[0079]

[0080] Where: characters Represents the updated checksum value, characters This represents the checksum value before the update, in characters. Indicates the checksum bit width, character The integer value representing the newly input data byte, sign Represents a circular left shift operation, symbol Represents a circular right shift operation, symbol This represents a bitwise XOR operation. The calculated checksum is compared with the pre-calculated checksum on the server side after decompression to verify the integrity of data transmission and decompression. After decompression and verification, memory protection attributes are set. By modifying the page table entries of the memory management unit, the kernel runtime area is marked as an executable region. The executable region attribute prevents the test program from writing to the kernel code area, thus preventing the test program from accidentally modifying the kernel code. In some embodiments, the system jumps to the kernel entry point to start the operating system. During the operating system startup process, the operating system kernel initiates a network file system protocol mount request. The network file system protocol mounts the test program package directory stored on the server. The test program package directory on the server side is exported through the network file system protocol. After mounting, a virtual read-only file system view is formed in the operating system memory of the machine under test.

[0081] In some embodiments, the minimized operating system kernel image file uses a specific compression format, and a streaming decompression algorithm performs real-time decompression for that specific compression format. Optionally, the starting address of the kernel runtime area is aligned to the memory page size, which is 4096 bytes. Optionally, the verification information includes a digital signature in addition to the hash value, and the initial bootloader needs to verify the digital signature using a pre-set public key. After the kernel is loaded and started, the operating system kernel initializes the network file system protocol client. The network file system protocol client connects to the network file system protocol server on the server side according to the configuration and mounts the specified remote directory. It can be understood that the directory mounted by the network file system protocol appears as a directory under the root file system, and files in the test package directory are accessed in read-only mode in this directory. In specific implementations, the read-only attribute is set through mount parameters, and the operating system kernel prevents any write system calls to the network file system protocol mount point. It can be understood that the complete content of the network file system protocol directory is not stored in memory; file data is only retrieved from the server on demand via the network file system protocol and cached in memory during access.

[0082] See Figure 3In one embodiment of the present invention, the generation of the module loading sequence begins with parsing the test task configuration file. The test task configuration file, written in Extensible Markup Language (EXPLAIN) or JSON format, explicitly lists the list of test items to be executed and the dependency declarations between the items. The test task configuration file is parsed to extract the test program module identifier corresponding to each test item, and a module dependency graph is constructed based on the dependency declarations. The module dependency graph is represented in memory as a directed acyclic graph, where nodes are test program modules, and directed edges point from dependent modules to dependent modules. The module dependency graph is topologically sorted, with the sorting algorithm iteratively removing nodes starting from those with an in-degree of zero, resulting in a linear module loading sequence. This sequence ensures that nodes of dependent modules are always listed before nodes of dependent modules. The module loading sequence is stored as an ordered list, where each item contains a unique identifier for the test program module and a set of identifiers for its dependent modules.

[0083] In implementation, following the module loading sequence, the system initiates a Hypertext Transfer Protocol (HTTP) request to the server to retrieve the binary data of the corresponding test program module. A unique identifier based on the hash value of its content is generated for each test program module. This generation process involves applying a secure hash algorithm to the complete binary content of the test program module, outputting a fixed-length hexadecimal string as the unique identifier. This unique identifier serves as the cache lookup key. Before initiating the actual HTTP request, the system first checks its local memory cache for a matching key-value entry. The local memory cache is a hash table-based data structure, where the key is the unique identifier and the value is the binary data block and metadata of the test program module. If a valid copy exists in the memory cache, the corresponding binary data block is read directly from memory, skipping the network transmission step. If no valid copy exists in the memory cache, a HTTP GET request is initiated to the server. The Uniform Resource Locator (URL) of the request is formed by concatenating the server's base address and the unique identifier. When receiving module data, the system calculates the hash value of the received data stream in real time and compares the calculation result with the unique identifier used when initiating the request. This consistency verification ensures the integrity and correctness of the data. Newly acquired module data is stored in a memory cache. This storage operation involves creating new key-value pairs in a hash table and updating the associated cache metadata, which includes a timestamp of the most recent access time and a reference count initially set to one. In some embodiments, the generation of the module loading sequence considers the estimated loading cost of the modules and employs a scheduling algorithm based on dependency depth and estimated weights to determine the final order. The scheduling algorithm calculates the priority value of each module using the following formula:

[0084]

[0085] Where: characters This indicates the loading priority value of the test program module m; a higher value indicates a higher priority. This represents the maximum dependency depth of test program module m in the module dependency graph; character This represents the weighting of the estimated binary data acquisition and loading time for test program module m; character It is an adjustment coefficient between zero and one, used to balance the impact of dependency depth and time weight. Modules are ranked according to their priority values. Arrange the modules in descending order to form an optimized module loading sequence.

[0086] In some embodiments, an independent load address range is allocated for each module in the memory space of the virtual execution environment, and the virtual execution environment maintains a global memory address allocation table. When allocating an independent load address range, allocation is performed on demand within a pre-defined contiguous virtual address space based on the size requirements declared in the module's binary file header, ensuring that the allocated address range does not overlap with other loaded modules. After address allocation, address references within the module are relocated. The relocation process resolves the module's relocation section and corrects all absolute address references based on the module's actual load base address. Loaded module information is registered in the module management table, a kernel data structure that records the module's entry point address, memory size, current state, and its unique identifier. Optionally, the module loading process supports lazy loading, i.e., only the module's metadata and necessary code segments are loaded, while large data segments within the module are loaded on demand when actually accessed. Optionally, for modules with the same unique identifier but different versions, the server resource pool stores multiple versions, and the version number parameter can be carried in the Hypertext Transfer Protocol request to specify the version to be retrieved. It's understandable that the lifespan of the memory cache is limited to a single test session, and all cached content naturally becomes invalid after the machine under test restarts. It's also understandable that the dynamic acquisition and injection mechanism of modules eliminates the need for the test system to persistently store any test programs locally on the machine under test, achieving true master disk-free testing.

[0087] In one embodiment of the present invention, an access history is maintained for each test program module cached in memory. The access history is a data structure that records the system clock timestamp of the most recent access to the test program module and an integer counter representing the cumulative number of accesses. The access history is updated each time a test program module is loaded or executed. The system periodically scans the access history of all cached modules using a background daemon thread. The scanning period is configurable, and the scan operation reads the most recent access timestamp and historical access count for each test program module.

[0088] In practice, a comprehensive popularity score is calculated for each test module during periodic scanning. This score is based on a weighted combination of access time and access frequency. The calculation of the comprehensive popularity score employs a decay-weighted model that considers both time decay and access frequency. The specific formula is as follows:

[0089]

[0090] Where: characters This represents the overall popularity score of the test program module; character Indicates the historical access count of the test program module; character Indicates the system clock timestamp during this scan; character Indicates the system clock timestamp of the most recent access to the test program module; character It is a positive frequency weighting coefficient; logarithmic function Used to normalize time differences and produce a decay effect. Overall popularity score. The lower the value, the less active the test module has been recently. Based on the total available memory space of the current system and a preset memory usage watermark threshold, the amount of cache space that needs to be released is dynamically calculated. The memory usage watermark threshold is a percentage value; cleanup is triggered when the proportion of used cache memory to total cache capacity exceeds this threshold. The formula for calculating the amount of cache space that needs to be released is: the current amount of used cache memory minus the product of the total cache capacity and the memory usage watermark threshold.

[0091] In practice, after determining the space that needs to be released, the memory cache management mechanism sorts all cache modules in ascending order of their overall popularity score. This sorting generates a candidate elimination list, starting with the test module with the lowest overall popularity score, and calculates the cumulative memory size occupied by each module sequentially. When the cumulative value reaches or exceeds the calculated cache space size that needs to be released, the traversal stops, and these traversed test modules constitute the candidate elimination module set. For each test module in the candidate elimination module set, the memory cache management mechanism checks its reference count, which records how many active test tasks are currently using this test module. If the reference count of a test module is zero, it means that it is not referenced by any test task, and the memory release function is immediately called to reclaim all the memory resources it occupies and remove it from the cache hash table. If the reference count of a test module is greater than zero, it cannot be released immediately; the memory cache management mechanism marks its status as "pending release" and updates the cache metadata. In some embodiments, the memory cache management mechanism records the load timestamp and last access timestamp of the test module, and the calculation of the overall popularity score can incorporate more factors to form a multi-dimensional popularity assessment. Optionally, the memory usage watermark can be set with two thresholds: a high watermark and a low watermark. When the cache exceeds the high watermark, aggressive cleanup is triggered until memory usage falls below the low watermark.

[0092] In some embodiments, for test modules marked as "pending release," the memory cache management mechanism continuously monitors changes in their reference counts. It is understood that whenever a test task completes and releases its reference to a test module, the reference count of that test module decreases. The memory cache management mechanism registers callback functions that are triggered when the reference count of a test module decreases. The callback function checks whether the reference count has dropped to zero and simultaneously checks whether the test module is marked as "pending release." If both conditions are met, memory release and cache cleanup operations are performed immediately. This delayed release mechanism ensures that test modules currently in use are not forcibly interrupted, and only the memory of modules that are no longer needed is released. Optionally, the calculation cycle for the overall heat score can be adaptively adjusted, shortening the cycle to improve response speed when memory pressure is high and lengthening the cycle to reduce computational overhead when memory is sufficient.

[0093] In one embodiment of the invention, a lightweight data acquisition agent is deployed in a virtual execution environment. This agent runs as a kernel module or privileged process. It obtains hardware performance counter values ​​by reading model-specific registers of the central processing unit (CPU), acquires memory bandwidth and latency data by accessing the memory controller's status registers via memory-mapped input / output, and reads input / output device load and network status information exposed by the operating system kernel through system call interfaces. The acquired raw performance data is preprocessed, including unit normalization, outlier filtering, and timestamp alignment. Unit normalization identifies the original unit of the raw performance data, which may be megahertz, bytes, milliseconds, or processor cycles. All values ​​are converted to standard International System of Units (SI) according to a predefined conversion rule table, for example, converting megahertz to hertz and milliseconds to seconds. Timestamp alignment uses a single high-precision monotonically increasing clock source to timestamp all data acquisition points. This high-precision monotonically increasing clock source can be a timestamp counter or a high-precision event timer. For data points from different hardware sources with different acquisition frequencies, an interpolation algorithm is applied to compensate for time deviations. The interpolation algorithm generates time series data at equal intervals based on the timestamps and values ​​of adjacent data points.

[0094] In practice, outlier filtering employs an anomaly detection algorithm based on statistical distribution. It calculates the moving average and standard deviation of the performance data values ​​within a sliding data window, the size of which is configurable. Data points exceeding a reasonable range are filtered out. The upper and lower bounds of the reasonable range are determined by the sum of multiples of the moving average and standard deviation, using the following formula:

[0095]

[0096] Where: characters Represents the value of the i-th raw performance data point to be judged; character This represents the moving average of all data points within a sliding window centered on that data point; character Represents the standard deviation of all data points within the same window; character This is a preset sensitivity coefficient used to control the strictness of filtering. When the inequality holds, the data points... Outliers were identified and filtered out from the data stream. See Table 1 for a partial result of the preprocessed raw CPU utilization data.

[0097] Table 1: Example snippet comparing performance data before and after preprocessing

[0098]

[0099] In some embodiments, preprocessed performance data is associated with corresponding test phase metadata, which includes a unique identifier for the test case, the test step name, and the expected performance metric type. This association is achieved by maintaining a state machine in the data acquisition agent. The state machine changes the current test phase as the test case is executed, thus semantically tagging each set of collected performance data. These semantic tags include descriptive labels such as "memory bandwidth test - read" and "CPU stress test - floating-point operation." The encapsulation process converts the semantically tagged structured performance records into binary or text-formatted log entries according to a preset log format. Each log entry contains a fixed-length header and a variable-length payload. The header fields include a test case identifier, a high-precision timestamp, and a performance metric type enumeration value, while the payload field contains the converted standard unit value.

[0100] In some embodiments, encapsulated log entries are streamed to the server via a persistent transport protocol. This persistent transport protocol ensures that unsent data is cached during temporary network interruptions and transmission resumes once the connection is restored. The server-side log collector is a network service that listens on a specific transport control protocol port, receiving connection and log streams from the machine under test (DUT). In a specific implementation, the server parses and reassembles the received log stream. Parsing involves disassembling log entries according to a known log format and extracting individual fields. Reassembly involves reorganizing discrete log entries from the same DUT and the same test case into a temporally continuous and logically complete test execution trajectory in memory, based on timestamps and test phase information. Optionally, the server can generate a performance profile after reassembly. This performance profile is a time-series graph showing the changes in specific hardware metrics during the test. Optionally, the sampling frequency of the lightweight data acquisition agent can be dynamically adjusted according to the test configuration; different sampling frequencies can be set for different test phases. It is understood that the conversion rule table used for unit standardization is stored in the lightweight data acquisition agent's configuration file, allowing customization for counter differences across different hardware platforms. It is understandable that the choice of a high-precision monotonically increasing clock source depends on the hardware platform; on platforms that support constant-rate timestamp counters, constant-rate timestamp counters are preferred.

[0101] See Figure 4This paper presents the statistical distribution of CPU utilization in five core stages (kernel loading stage, module dynamic injection stage, memory cache management stage, performance data acquisition stage, and memory cleanup stage) of a memory-based, master-disk-free system testing process. Kernel loading stage (blue box): CPU utilization in this stage is generally high and stable, concentrated between 85% and 95%. This indicates that the decompression, verification, and startup processes of the minimized kernel continuously consume significant CPU resources with minimal fluctuations, and only a few discrete data points due to differences in hardware initialization. Module dynamic injection stage (orange box): CPU utilization in this stage is concentrated between 65% and 80%, with a significant upper bound anomaly (utilization approximately 145%). This anomaly is typically triggered by instantaneous resource contention or high-frequency interrupts during parallel module loading. The overall downward shift of the box reflects that CPU resource requirements are lower during module transmission and relocation compared to the kernel loading stage. Memory Cache Management Phase (Green Box): During this phase, CPU utilization ranges from 45% to 65%, with noticeable anomalies (upper bound approximately 98%, lower bound approximately 25%). This is directly related to background operations such as the periodic scanning of the cache eviction algorithm and module heat calculation. When cache space is insufficient, high-frequency module eviction operations will briefly increase CPU load, while utilization will drop during idle periods. Performance Data Acquisition Phase (Red Box): During this phase, CPU utilization is concentrated between 75% and 90%, with a stable overall distribution. This reflects the stable CPU resource usage during the process of the lightweight acquisition agent continuously reading hardware counters and preprocessing performance data. The compactness of the box also reflects the low volatility of the data acquisition process. Memory Cleanup Phase (Purple Box): During this phase, CPU utilization is generally at its lowest level, concentrated between 0% and 15%, with only a few instantaneous peaks (approximately 25%) triggered by memory fill operations. This is because memory cleanup mainly relies on the hardware-level memory controller to perform fill operations, with the CPU only undertaking lightweight tasks such as instruction triggering and status monitoring. Box plots not only quantify the CPU resource consumption characteristics of each test phase, but also provide key basis for performance optimization of the test process. For example, for abnormal peaks in the module dynamic injection phase, CPU load can be smoothed by optimizing the module parallel loading strategy; for fluctuations in the memory cache management phase, resource consumption can be reduced by adjusting the scan cycle of the cache replacement strategy.

[0102] In one embodiment of the present invention, after all test tasks are completed, a cleanup and restart process for the device under test (DUT) is triggered. The test scheduler monitors the execution status of all test tasks and maintains a task status mapping table. The task status mapping table records the unique identifier of each test task and its current status, including "waiting," "in execution," "completed," or "failed." The test scheduler receives task status updates from the DUT via a heartbeat mechanism or explicit completion notification. When the status of the last test task changes to "completed," the test scheduler triggers the cleanup process. The test scheduler sends a safe shutdown request to the DUT. The safe shutdown request is a structured message transmitted over the network. The message format includes a message type field, a request identifier field, and an option flag field. The option flag field sets an immediate restart identifier and a memory clear identifier.

[0103] In practice, the machine under test (DUT) runs a receive service daemon that listens for network connections from the test scheduler. Upon receiving a safe shutdown request, the daemon parses the message and confirms that both the immediate restart identifier and the clear memory identifier are set. The DUT suspends all currently executing user-mode test processes. This suspension operation involves sending a signal to each test process and waiting for the process to enter a state where it can be safely stopped. The DUT synchronizes any unuploaded log data in memory to the server. This synchronization operation involves checking the log sending buffer and immediately sending all remaining log entries in the buffer to the server-side log collector via a persistent transmission protocol, then waiting for the collector to return confirmation. The system's low-level interface is invoked to send a clear command to the memory controller. This system low-level interface is a set of privileged functions provided by the operating system kernel, allowing direct interaction with hardware registers. The physical memory address space mapping information is obtained by parsing the system address mapping table to determine the range of user-available memory that needs to be cleared, excluding address segments reserved for the basic input / output system or other hardware reserved areas.

[0104] In implementation, a special memory fill command is constructed. This command is a data structure containing a command opcode, the target start physical address, the target end physical address, and a fill mode field. The fill mode is specified as zero fill or random number fill. Zero fill means writing zero values ​​to all target memory cells, while random number fill means using a random sequence generated by a hardware random number generator. The fill command is written to the memory controller's command register via memory-mapped input / output (MRP). MRP maps specific addresses within a physical memory address range to virtual addresses in user space. A write operation is performed on this virtual address to complete the write to the memory controller's command register. The memory controller's status register is monitored. This status register is read from another MRP address, and the system waits for the operation completion confirmation bit in the status register to be set by hardware, indicating that the fill operation has been completed by the memory controller.

[0105] In some embodiments, the verification of the padding effect is achieved by randomly sampling a portion of memory addresses and checking their contents. The sampling process uses a pseudo-random algorithm to generate a series of physical addresses falling within the target address range. The contents of each sampled address are read and compared with the expected value of the padding pattern specified in the padding command to check whether all sampled points have changed to the expected padding pattern. The success of the verification operation can be determined by statistical confidence, and its mathematical model for sampling verification can be expressed as determining whether the following conditions are met:

[0106]

[0107] Where: characters Indicates in The number of times the content matches the expected filling pattern in a random sample; character This represents the total number of random samples, which is a pre-set, sufficiently large number; character This represents a very small, acceptable error tolerance. The clearing operation is considered successful when the condition is met. A hardware reset signal is triggered, which is achieved by writing to a platform-specific reset control register. The address and write value of the reset control register are defined by the hardware specification. The write operation causes the CPU and chipset of the device under test to perform a hard reboot. During the early hardware initialization phase of the reboot process, the existing data in memory is automatically cleared by the memory controller or basic input / output system, and the device under test returns to an initial state where it can be re-booted for network communication.

[0108] In some embodiments, the safe shutdown request may include a delay parameter, allowing a brief delay window between sending the request and the actual cleanup and restart to complete the final resource reclamation. Optionally, the padding mode can be configured to perform multiple padding operations, such as first padding with random numbers and then padding with zeros, to enhance the security of data cleanup. Optionally, before calling the underlying system interfaces, the operating system kernel will unload all unnecessary kernel modules and shut down all external devices to ensure that the memory cleanup process is not disturbed. It is understood that the time taken for the memory padding operation is proportional to the total physical memory capacity of the machine under test, and the test scheduler will set a reasonable timeout after issuing the safe shutdown request to wait for the machine under test to come back online. It is understood that the entire cleanup and restart process ensures the one-time nature and isolation of the test environment, and the machine under test will not retain any data or state related to the previous test after restarting.

[0109] See Figure 5 In the memory filling effect verification stage, the relationship between sampling size, success rate, and confidence level was quantified using trend curves on a logarithmic scale. The experiment aimed to verify the filling effect, with the core objective being to evaluate the effectiveness of the memory filling and clearing operation. In the experimental design, the horizontal axis used a logarithmic scale for the sampling size, and the vertical axis represented the ratio, encompassing the sampling success rate (blue curve) and confidence level (orange curve), with a gray dashed line indicating a 99% acceptable error tolerance. As the sampling size increased from the order of 10 to the order of 10³, the sampling success rate showed a significant monotonic upward trend, gradually approaching 1.00 from 0.95, indicating that a larger sample size more accurately reflects the overall memory filling effect. The confidence level curve steadily increased from 0.98 to nearly 0.99, reflecting that the statistical reliability of the sampling verification results improved with increasing sample size. When the sampling size reached the order of 10³, both curves exceeded the 99% acceptable error tolerance, indicating that the sampling results at this point had sufficient statistical confidence, and the memory filling and clearing operation could be considered successfully executed.

[0110] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A memory-based whole-system testing method without master disks, characterized in that, include: On the server side, a resource pool containing multiple operating system images and test packages is built, with each test package consisting of several independent functional modules; The system provides boot information to the machine under test (DUT) through the network boot protocol service, enabling the DUT to obtain a minimal operating system kernel from the server and load it into memory for execution during the boot phase. A virtual execution environment is created in memory, which provides a temporary file system independent of local storage to hold temporary data generated during the test. A module loading sequence is generated according to the test task requirements, and the corresponding test program module is dynamically obtained from the server's resource pool according to the module loading sequence and injected into the virtual execution environment. Establish a memory cache management mechanism to monitor the usage frequency of test program modules in memory and maintain a set of active modules according to a preset replacement strategy; During the execution of the test task, the hardware performance data of the machine under test is captured in real time, and the hardware performance data is encapsulated into a standardized log stream and uploaded to the server; After completing all test tasks, a clear command is sent to the device under test to restart the device and clear all temporary data in memory.

2. The memory-based whole-system testing method for removing the master disk as described in claim 1, characterized in that, The provision of boot information to the machine under test (DUT) via the network boot protocol service, enabling the DUT to obtain a minimal operating system kernel from the server and load it into memory for execution during the boot phase, includes: Configure the Dynamic Host Configuration Protocol service on the server side to assign a network address to the machine under test and specify the location information of the Network Startup Protocol server. The initial bootloader is transmitted to the machine under test via Simple File Transfer Protocol Service. The initial bootloader contains the acquisition path and verification information of the operating system kernel. After the machine under test executes the initial bootloader, it downloads a compressed, minimal operating system kernel image file from the server. The minimal operating system kernel contains only the basic drivers and system services required to run the test. Extract the downloaded minimal operating system kernel image file to a specified area of ​​memory, and jump to the kernel entry point to start the operating system; During the operating system startup process, the test program package directory stored on the server is mounted via the network file system protocol, forming a virtual read-only file system view.

3. The memory-based whole-system testing method for removing the master disk as described in claim 2, characterized in that, The step of decompressing the downloaded minimal operating system kernel image file to a specified area of ​​memory includes: A contiguous storage space in memory is allocated as the kernel running area, the size of which is preset according to the uncompressed size of the minimized operating system kernel image file; A streaming decompression algorithm is used to download and decompress simultaneously, and the decompressed kernel data is written into the kernel execution area in order of address offset. During the decompression process, the checksum of the kernel data is calculated and compared with the checksum pre-calculated on the server side to verify data integrity. After decompression and verification are completed, memory protection attributes are set to mark the kernel runtime area as an executable-only area to prevent the test program from accidentally modifying the kernel code.

4. The memory-based whole-system testing method for removing the master disk as described in claim 1, characterized in that, The step of generating a module loading sequence according to the test task requirements, and dynamically obtaining the corresponding test program module from the server's resource pool according to the module loading sequence and injecting it into the virtual execution environment includes: Parse the test task configuration file, extract the test items to be executed and their dependencies, and generate a module dependency graph in the form of a directed acyclic graph; The module dependency graph is topologically sorted to obtain a linear module loading sequence, ensuring that dependent modules are loaded before the modules they depend on. According to the order of the module loading sequence, a Hypertext Transfer Protocol request is sent to the server to obtain the binary data of the corresponding test program module; In the memory space of the virtual execution environment, an independent load address range is allocated for each module, and address references within the module are relocated; Register the loaded module information into the module management table, recording the module's entry point, size, and status information.

5. A memory-based whole-system testing method for removing the master disk as described in claim 4, characterized in that, The step of initiating a Hypertext Transfer Protocol request to the server in the order of the module loading sequence to obtain the binary data of the corresponding test program module includes: Generate a unique identifier for each test program module based on its content hash value, and use the unique identifier as the cache lookup key; Before initiating a Hypertext Transfer Protocol request, first check if a copy of the module with the same unique identifier exists in the local memory cache; If a valid copy exists in the memory cache, the module data in the cache is used directly, skipping the network transmission step; If a valid copy is not found in the memory cache, a request is sent to the server, and the hash value of the received module data is verified to be consistent with the expected unique identifier. The newly acquired module data is stored in the memory cache, and the access timestamp and reference count in the cache metadata are updated.

6. The memory-based whole-system testing method for removing the master disk according to claim 1, characterized in that, The aforementioned memory cache management mechanism monitors the usage frequency of test program modules in memory and maintains a set of active modules according to a preset replacement strategy, including: Maintain access history for each cached test program module, recording the most recent access time and the number of historical accesses; Periodically scan the access history of all cache modules and calculate the overall popularity score of each module, which is based on a weighted combination of access time and access frequency; Determine the amount of cache space that needs to be released based on the current available memory space and the preset memory usage threshold; Candidate modules are selected and eliminated in order of their overall popularity score from low to high until the released cache space meets the demand. For modules selected for elimination, if they are not currently referenced by any test task, the memory they occupy will be released immediately; otherwise, they will be marked as pending release and released after the reference is removed.

7. A memory-based whole-system testing method for removing the master disk as described in claim 1, characterized in that, During the execution of the test task, the hardware performance data of the machine under test is captured in real time, and the hardware performance data is encapsulated into a standardized log stream and uploaded to the server, including: A lightweight data acquisition agent is deployed in a virtual execution environment, which obtains raw performance data by directly reading hardware performance counters and system status registers; The collected raw performance data is preprocessed, including unit standardization, outlier filtering, and timestamp alignment. The preprocessed performance data is encapsulated according to a preset log format. Each log entry contains a test case identifier, timestamp, performance metric type, and value. The encapsulated log entries are streamed to the log collector on the server side using a persistent transmission protocol. The server parses and reassembles the received log stream to restore the complete test execution trajectory and performance profile.

8. A memory-based whole-system testing method for removing the master disk as described in claim 7, characterized in that, The preprocessing of the collected raw performance data includes unit standardization, outlier filtering, and timestamp alignment, including: Identify the original units of the raw performance data and convert them to standard International System of Units (SI) representation according to predefined conversion rules; An anomaly detection algorithm based on statistical distribution is applied to calculate the moving average and standard deviation of performance data values, and to filter out data points that exceed the reasonable range. A high-precision clock source is used to stamp each data point with a monotonically increasing timestamp to ensure time consistency of data from different hardware counters. Interpolation of timestamps is performed to compensate for time deviations caused by differences in collection frequency, generating time series data with equal intervals. The processed performance data is associated with the corresponding test phase metadata to form a structured performance record with semantic tags.

9. A memory-based whole-system testing method for removing the master disk as described in claim 1, characterized in that, After completing all test tasks, a clear command is sent to the device under test (DUT) to restart the DUT and clear all temporary data in its memory, including: The test scheduler monitors the execution status of all test tasks and triggers a cleanup process when it detects that the last test task has been completed. Send a safe shutdown request to the machine under test, the request carrying identifiers for immediate restart and memory clearing; After receiving a safe shutdown request, the machine under test suspends all running processes and synchronizes the log data in memory to the server. The system calls the underlying interface to send a clear command to the memory controller, causing the memory controller to perform a memory fill operation and write all memory units into a specific pattern. The hardware reset signal is triggered, causing the device under test to restart. During the startup process, the original data in the memory is automatically cleared by the hardware.

10. A memory-based whole-system testing method for removing the master disk as described in claim 9, characterized in that, The process of calling the system's underlying interface to send a clear command to the memory controller, causing the memory controller to perform a memory filling operation, includes: Obtain the address space mapping information of physical memory, determine the memory range that needs to be cleared, and exclude the basic input / output system area reserved for hardware; Construct a special memory fill command, specifying the fill mode and target address range, wherein the fill mode is zero fill or random number fill; The fill command is written to the command register of the memory controller using memory-mapped input / output. Monitor the status register of the memory controller and wait for a confirmation signal that the fill operation is complete; To verify the filling effect, randomly sample a portion of memory addresses and check if their contents have changed to the specified filling pattern to confirm that the clearing operation was successfully executed.

Images