Parallel test task scheduling method oriented to high-computing-power GPU (Graphics Processing Unit) chip

Abstract

The invention relates to the field of GPU chip testing, in particular to a parallel test task scheduling method oriented to a high-computing-power GPU chip. The specific implementation process comprises the following steps: constructing a test task-resource demand matrix, calculating transient thermal stress generated by a test task, and verifying a test safety margin; aggregating the plurality of test tasks into a parallel test execution cluster according to the test safety margin, and planning a GPU test scheduling graph containing space mapping and an execution time window for the parallel test execution cluster; and setting a fault causal traceability chain, performing analysis when the test fails, outputting a fault source context, and troubleshooting a fault cause. According to the method, the test task-resource demand matrix is constructed, a fine scheduling mode of first planning and second execution is adopted, and active security verification is introduced, so that the security of the GPU chip is guaranteed, efficient parallel test of the high-computing-power GPU chip is realized, and the execution efficiency of the test task and the accuracy of fault positioning are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of GPU chip testing, in particular to a parallel test task scheduling method for high-computing-power GPU chips. BACKGROUND

[0002] In the field of high-computing-power GPU chip testing, in order to overcome the bottleneck of long time consumption of traditional serial testing methods, the industry has generally adopted parallel testing as the mainstream technical solution. That is, by using the thousands of parallel processing cores possessed by the GPU chip itself, the testing cases that need to be executed one by one are developed into computing tasks that can be executed concurrently through CUDA or OpenCL parallel computing programming framework. These concurrent testing tasks are submitted to the GPU in batches and rely on the task scheduler for real-time resource allocation and execution management, and the overall testing period of the chip is greatly shortened through large-scale parallel processing.

[0003] However, the existing technology has inherent defects in the mismatch between the general GPU scheduler and the special needs of chip testing and verification in practice. The general scheduler is "blind" to the content and intention of the testing task, and cannot optimize the allocation of hardware resources according to the testing target. The parallel efficiency is far from the theoretical value and it is easy to cause uneven test coverage. The scheduling process has uncertainty, making it difficult to reproduce and accurately locate the fault when the test fails, which seriously affects the yield analysis and debugging efficiency of the chip.

[0004] Therefore, a parallel test task scheduling method for high-computing-power GPU chips is proposed. SUMMARY

[0005] The purpose of the present application is to provide a parallel test task scheduling method for high-computing-power GPU chips, which realizes parallel testing of GPU chips while ensuring the integrity of GPU chips.

[0006] To achieve the above purpose, the present application provides the following technical solutions: A parallel test task scheduling method for high-computing-power GPU chips, comprising: parsing a set of testing tasks and constructing a testing task-resource demand matrix, the testing task-resource demand matrix containing the resource demand of each testing task for the physical execution unit inside the GPU and the matching relationship with the testing target; According to the real-time state of the GPU, the testing target and the testing task-resource demand matrix, the transient thermal stress generated by the testing task on the GPU is calculated and the testing safety margin is checked, and a plurality of testing tasks are aggregated into a parallel test execution cluster according to the testing safety margin; a GPU test scheduling diagram is planned for the spatial mapping of each testing task to the physical execution unit of the GPU and the execution time window of the parallel test execution cluster; A fault cause chain is set for the parallel test task, the GPU test scheduling diagram is associated and bound with a real-time execution log of a corresponding physical execution unit of the GPU, and when a test task fails, the fault cause chain is analyzed to output a fault source context causing the failure and to troubleshoot the fault cause.

[0007] Preferably, the specific implementation process of the analyzing and constructing the test task-resource demand matrix includes: The execution logic and resource demand description of all test tasks are extracted through syntax analysis, the calculation core type and storage resource type required by the test task are identified and quantified as a function parameter for the execution logic, the video memory capacity and memory bandwidth required for task execution are obtained and quantified as a constraint condition for the resource demand description, the function parameter and the constraint condition are integrated to generate a resource demand vector for the test task, a data matrix is created with all test tasks as the first dimension and the physical execution units and resource types that can be scheduled in the GPU as the second dimension, and the resource demand vector corresponding to the test task is filled into the data matrix to construct a test task-resource demand matrix.

[0008] Preferably, the specific implementation process of calculating the transient thermal stress generated by the test task on the GPU and checking the test safety margin according to the real-time state of the GPU, the test target, and the test task-resource demand matrix includes: The real-time state of the GPU including the current temperature and power consumption baseline of the physical execution unit is obtained, the resource demand vector corresponding to the test task in the test task-resource demand matrix is combined to calculate the power consumption superposition value generated when the test task is executed on each physical execution unit of the GPU, the power consumption superposition value is input as a heat source for heat conduction operation to output the transient thermal stress, the working temperature safety threshold of the GPU is extracted from the test target, the peak thermal stress in the transient thermal stress is difference-operated with the working temperature safety threshold to obtain the test safety margin.

[0009] Preferably, the specific implementation process of aggregating multiple test tasks into a parallel test execution cluster according to the test safety margin includes: The test safety margin is taken as a core constraint, test tasks are selected from the test task set according to a preset scheduling strategy and added to a parallel test execution cluster candidate set, the resource demand vectors of all test tasks in the parallel test execution cluster candidate set are updated based on the test task-resource demand matrix, the transient thermal stress generated by the test task on the GPU is re-evaluated and the remaining test safety margin is checked, when the test safety margin still satisfies a preset condition, test tasks are continuously selected from the test task set and added to the parallel test execution cluster candidate set, and when the test safety margin does not satisfy the preset condition, the expansion of the current parallel test execution cluster candidate set is terminated and the parallel test execution cluster is aggregated.

[0010] The specific implementation process of the preferred GPU test scheduling graph for planning each test task to the spatial mapping of the GPU physical execution unit and the execution time window of the parallel test execution cluster includes: According to the test task-resource requirement matrix, a resource view containing the GPU topology structure and the real-time available state of each physical execution unit is obtained, and the resource requirement of each test task contained in the parallel test execution cluster to the physical execution unit is extracted; the resource requirement is input into the resource view for fitting operation to obtain the spatial mapping; the spatial mapping contains each test task in the parallel test execution cluster that should be loaded to the GPU physical execution unit; the execution time interval of each test task in the spatial mapping is determined according to the data dependency relationship between the test tasks to form the execution time window; the spatial mapping and the execution time window are integrated to generate the GPU test scheduling graph containing complete scheduling instructions.

[0011] The specific implementation process of the preferred fault cause trace chain set for the parallel test task includes: According to the GPU test scheduling graph, the data structure of the fault cause trace chain is initialized; when the test task is started to be executed, the execution log data stream generated from each physical execution unit of the GPU is captured in real time; the execution log data stream is parsed and the cause node is matched for each execution instance of the test task, and the cause node records the task identification, the predetermined physical execution unit mapping relationship and the execution time window information; the execution instance and the cause node that are successfully matched are associated and bound to generate the trace information unit containing the scheduling intention and the execution live; all the trace information units are sequentially linked according to the execution time window order of the test task in the GPU test scheduling graph to construct the fault cause trace chain.

[0012] The specific implementation process of the preferred fault cause trace chain set for the parallel test task includes: After receiving the fault signal of the test failure, the end trace information unit directly associated with the fault event is searched and locked in the fault cause trace chain according to the time stamp carried in the fault signal; the end trace information unit is taken as the starting point of the analysis, and the reverse tracing traversal is performed along the fault cause trace chain; in the reverse tracing traversal process, the GPU real-time execution log segments and the corresponding GPU test scheduling graph information encapsulated by the previous trace information units are batch extracted; the extracted multiple real-time execution log segments and the scheduling graph information are time-aligned and state-associated, the continuous GPU execution state sequence before and after the fault occurrence point is reconstructed, and the fault source context is output; the fault source context is subjected to abnormality detection to troubleshoot the fault cause.

[0013] Compared with the prior art, the present application has the following beneficial effects: 1. The present application accurately grasps the resource demand of each task on the physical execution unit inside the GPU by analyzing the test task set and constructing a "test task-resource demand matrix". Meanwhile, considering the real-time state of the GPU and the test target, the present application efficiently aggregates multiple test tasks to be tested into a "parallel test execution cluster". This "planning first, then execution" fine scheduling method can effectively improve the parallel processing efficiency and shorten the overall test cycle of the chip.

[0014] 2. The present application introduces the calculation and verification of "transient thermal stress" and "test safety margin" before task scheduling. By evaluating the thermal effect of the parallel execution task cluster on the GPU, it ensures that it will not exceed the working temperature safety threshold of the GPU. This effectively avoids the physical damage or instability caused by excessive test load, ensuring the safety of the test process and the integrity of the chip.

[0015] 3. The present application associates and binds the GPU test scheduling diagram with the real-time execution log of the physical execution unit, matches a "cause and effect node" for each test task execution instance and sets up a "fault cause and effect trace chain". When the test fails, it can quickly lock the end information unit directly associated with the fault event in the trace chain according to the timestamp of the fault signal. At the same time, it traverses the trace chain in reverse, extracts the execution log and scheduling information related to the previous sequence in batches and reconstructs the continuous GPU execution state sequence before and after the fault occurrence point. This effectively improves the accuracy and efficiency of fault location and provides strong support for chip yield analysis and debugging. BRIEF DESCRIPTION OF DRAWINGS

[0016] Fig. 1 A parallel test task scheduling method flowchart for high-computing-power GPU chips is proposed for the present application. Fig. 2 A parallel test execution cluster aggregation process schematic diagram is proposed for the present application. Fig. 3 A fault cause and effect trace chain troubleshooting principle diagram is proposed for the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present application clearer, the following will refer to the drawings and combine specific examples to make a detailed description of the present application. It must be understood that the specific examples described here are only used to explain the present application, and are not intended to constitute any form of limitation on the scope of protection of the present application. Therefore, all equivalent changes or modifications conceived by those of ordinary skill in the art based on the content disclosed in the present application without creative labor shall fall within the scope of protection requested by the present application.

[0018] With reference to Figs. 1 to 3 The present application provides a parallel test task scheduling method for a high-computing-power GPU chip, and the technical solutions are as follows: Embodiment one: With reference to Fig. 1 The parallel test task scheduling method for a high-computing-power GPU chip in the present embodiment comprises: parsing a test task set and constructing a test task-resource demand matrix, wherein the test task-resource demand matrix contains resource demands of each test task for GPU internal physical execution units and matching relationships with test targets; calculating transient thermal stress generated by test tasks on a GPU according to real-time states of the GPU, test targets and the test task-resource demand matrix, and checking a test safety margin, and according to the test safety margin, aggregating multiple test tasks into a parallel test execution cluster; planning spatial mapping of each test task to a GPU physical execution unit and a GPU test scheduling diagram of an execution time window for the parallel test execution cluster; setting a fault cause and effect traceability chain for the parallel test task, associating and binding the GPU test scheduling diagram with real-time execution logs of corresponding physical execution units of the GPU; when a test task fails, analyzing based on the fault cause and effect traceability chain, outputting a fault source context causing the failure and troubleshooting the fault cause.

[0019] Further, the specific implementation process of parsing the test task set and constructing the test task-resource demand matrix comprises: extracting execution logic and resource demand descriptions of all test tasks through syntax analysis; for the execution logic, identifying computing core types and storage resource types required to be called by the test task and quantifying them as function parameters; for the resource demand description, obtaining video memory capacity and memory bandwidth required for task execution and quantifying them as constraint conditions; integrating the function parameters and constraint conditions to generate a resource demand vector for the test task; creating a data matrix with all test tasks as the first dimension and GPU internal schedulable physical execution units and resource types as the second dimension; filling the resource demand vector corresponding to the test task into the data matrix to construct a test task-resource demand matrix.

[0020] The object to be tested is a new high-performance computing GPU, which has 32 stream multiprocessors (SMs) integrated inside, each SM containing 64 CUDA cores, totaling 2048 CUDA cores. The GPU is equipped with 16GB of video memory, with a theoretical memory bandwidth of 512GB / s. The test task set contains three independent test cases: a matrix multiplication test to verify the performance of the matrix multiplication unit, an FFT test to test the Fast Fourier Transform (FFT) function, and a high-load memory bandwidth stress test memory bandwidth stress test.

[0021] The test task written in the CUDA parallel computing language is received. For the matrix multiplication test, it is logically analyzed by the syntax analysis function, and it is identified that its core computing load mainly calls the CUDA core of the GPU for floating point operation. By extracting the thread block and grid of the kernel function in the code, the configuration parameters are set to dimGrid(512, 1) and dimBlock(256, 1), so that the task needs to schedule at least 131072 threads when running at full load, which will occupy a large number of CUDA core resources. At the same time, by analyzing its resource requirement description, it is identified that the task needs to allocate two 512MB floating point matrices as input and generate a 512MB output matrix, so the demand for computing core type in its function parameters is quantified as "CUDA core intensive", and the demand for memory capacity in its constraint condition is quantified as 1.5GB.

[0022] For the FFT test, the syntax analysis function identifies that its execution logic contains calls to specific signal processing instructions that can be mapped to the TensorCore inside the GPU. Therefore, its function parameters are quantified as having specific requirements for "TensorCore". At the same time, the data processing volume requires 512MB of memory, and this value is also accurately extracted and used as its constraint condition.

[0023] For the memory bandwidth stress test, its execution logic is identified as large-scale, continuous data transfer operations, with low demand for computing cores, but frequent calls to storage resources. Its resource requirement description indicates the need to continuously maintain a data transfer rate close to the theoretical limit of the chip, so its constraint condition is quantified as requiring at least 450GB / s of memory bandwidth and 8GB of memory capacity.

[0024] After the functional parameters and constraints of all test tasks are quantified, these discrete information is integrated into a standardized resource requirement vector for each test task. For example: the resource requirement vector of the matrix multiplication test can be represented as (CUDA core requirement: high, TensorCore requirement: none, memory capacity: 1.5 GB, memory bandwidth: 80 GB / s). Then a data matrix, i.e. a test task-resource requirement matrix, is created. The first dimension of the matrix is composed of all test tasks, i.e. contains three entries of matrix multiplication test, FFT test and memory bandwidth stress. The second dimension is composed of all physical execution units and key resource types available for scheduling in the GPU, such as "CUDA core", "TensorCore", "memory capacity (GB)" and "memory bandwidth (GB / s)". Finally, the resource requirement vector previously generated for each test task is accurately filled into the corresponding row of the data matrix to construct a structured and quantifiable test task-resource requirement matrix. This matrix intuitively reflects the specific demand profile of each task in the entire test set for hardware resources. Thus, it provides an accurate and reliable data basis for subsequent scheduling decisions, such as transient thermal stress calculation, parallel execution cluster construction and fault cause chain generation.

[0025] By constructing the test task-resource requirement matrix, the scheduling function can predict the specific impact of each task on hardware resources, thereby providing a foundation for subsequent efficient task aggregation and parallel scheduling, avoiding low test efficiency and uneven coverage caused by resource conflicts or overload, and effectively improving the overall efficiency and reliability of the test.

[0026] Further, according to the real-time state of the GPU, the test target and the test task-resource requirement matrix, the specific implementation process of calculating the transient thermal stress generated by the test task on the GPU and checking the test safety margin includes: Obtain the real-time state of the GPU containing the current temperature and power consumption baseline of the physical execution unit, combine the resource requirement vector corresponding to the test task in the test task-resource requirement matrix, calculate the power consumption superposition value generated when the test task is executed on each physical execution unit of the GPU; input the power consumption superposition value as a heat source for heat conduction operation, output the transient thermal stress; extract the working temperature safety threshold of the GPU from the test target, perform difference operation between the peak thermal stress in the transient thermal stress and the working temperature safety threshold, and obtain the test safety margin.

[0027] After constructing the accurate test task-resource requirement matrix, the matrix multiplication test and the FFT test two tasks are aggregated into one parallel test execution cluster. Before executing the decision, first through the on-board management controller interface to obtain the real-time state of each physical execution unit of the GPU chip. For example: by reading the built-in digital temperature sensor, the initial temperature of all 32 stream multiprocessors (SM) is obtained. The temperature of the memory controller is 48°C, the chip is in standby state, the power consumption baseline of each SM is 5 watts, and the power consumption baseline of the memory system is 10 watts.

[0028] In combination with the resource requirement vectors corresponding to the matrix multiplication test and the FFT test in the constructed test task-resource requirement matrix, and through a pre-established power model calibrated by the power consumption characteristics of the chip, the power consumption superposition value generated by the parallel execution of the two tasks on each physical execution unit of the GPU is calculated. The power model can accurately map resource requirements such as CUDA core occupancy, memory bandwidth usage, etc. to the corresponding power consumption. According to the model calculation, the resource requirement vector shows that the matrix multiplication test task is "CUDA core intensive", if allocated to execute on SM0 to SM15, it will make the power consumption of the 16 SM units increase by 30 watts on the 5-watt baseline; and the FFT test task, due to its call to TensorCore, if allocated to execute on SM16 to SM23, it will make the power consumption of the 8 SM units increase by 22 watts. At this time, the non-uniform power distribution diagram on the surface of the chip when executing in parallel is obtained: the total power consumption of SM0 to SM15 is 35 watts, the total power consumption of SM16 to SM23 is 27 watts, and the remaining SM and other units maintain the baseline power consumption. This distribution diagram is the power consumption superposition value.

[0029] The power consumption superposition value is input as a heat source to call a built-in chip-level heat conduction operation model for transient analysis. The model is built based on a finite element analysis method, and internally contains the physical layout of the chip, the thermal conductivity of the material, and the parameters of the heat dissipation system. By simulating the generation, conduction and dissipation of heat on the silicon wafer, a set of transient thermal stresses representing the temperature change of the chip over time when executing the task cluster is output within a simulation time of hundreds of milliseconds. The simulation of transient thermal stress can effectively avoid the transient performance instability or even permanent physical damage caused by local overheating of the chip due to improper test load superposition. The operation result shows that the temperature of SM8 reaches a peak of 89°C after about 500ms, while the temperatures of other regions are relatively low. The 89°C is the peak thermal stress of this evaluation. At the same time, the working temperature safety threshold set to ensure the long-term reliability of the chip is extracted from the test target configuration file of this test, and the threshold is 95°C. Finally, the difference between the simulated peak thermal stress of 89°C and the working temperature safety threshold of 95°C is calculated, and the result is 6°C. This positive result is the test safety margin, which indicates that the current scheduled parallel task combination is safe in terms of heat, and the highest temperature generated is still within the acceptable safety range, so the scheduling scheme can pass the verification.

[0030] The embodiment improves passive overheating protection mechanism to active safety verification by introducing heat conduction simulation based on accurate power consumption model before task scheduling. Not only the integrity of high-value test chips is guaranteed, but also the overall test period is effectively compressed, and the efficiency and quality of test verification are improved.

[0031] Further, the specific implementation process of aggregating multiple test tasks into a parallel test execution cluster according to the test safety margin includes: With the test safety margin as the core constraint, test tasks are selected from the test task set and added to the parallel test execution cluster candidate set according to the preset scheduling strategy. The resource demand vector of all test tasks in the parallel test execution cluster candidate set is updated based on the test task-resource demand matrix, the transient thermal stress generated on the GPU is re-evaluated, and the remaining test safety margin is verified. When the test safety margin still meets the preset condition, test tasks are continuously selected from the test task set and added to the parallel test execution cluster candidate set. When the test safety margin does not meet the preset condition, the expansion of the current parallel test execution cluster candidate set is terminated, and the parallel test execution cluster is aggregated.

[0032] Reference Fig. 2 After obtaining the specific test safety margin value, the parallel test execution cluster is constructed by iterative verification. There are multiple tasks to be scheduled in the test task set, such as matrix multiplication test, FFT test, and ray tracing test.

[0033] The scheduling strategy takes the importance of the test task as the benchmark to give each test task a clear priority. For example: set the task of verifying the basic functions of the chip core operation unit (such as CUDA core) as the highest priority, and set the high-load stress test or auxiliary function verification at a lower priority. When building the execution cluster, use the iterative check method to select tasks from the set of tasks to be tested in order of priority from high to low and add them one by one to the candidate set. Ensure that all tasks that pass the safety check are combined and aggregated into a parallel test execution cluster that can not only prioritize key function verification, but also be stable and efficient in terms of thermal performance.

[0034] Taking the test safety margin as the core constraint, first select the highest priority matrix multiplication test from the test task set and add it to an initially empty parallel test execution cluster candidate set. Based on the test task-resource demand matrix, update the cumulative resource demand vector of all tasks in the candidate set (currently only containing the matrix multiplication test). Through thermal stress evaluation, it is calculated that executing this task alone will produce a peak thermal stress of 70℃. The set working temperature safety threshold is 95℃, and the remaining test safety margin value is 95℃-70℃=25℃. Since this safety margin still meets the preset condition of being greater than 0℃, it is determined that tasks can continue to be added to the candidate set.

[0035] Continue to select the second highest priority FFT test from the remaining test tasks and add it to the candidate set. At this time, the candidate set contains (matrix multiplication test, FFT test). Again, based on the test task-resource demand matrix, update the combined resource demand vector of the two tasks in the candidate set on different physical execution units, and perform transient thermal stress evaluation again based on this. The calculation shows that the peak thermal stress of this task combination is 89℃. According to this, the remaining test safety margin is checked again, resulting in 95℃-89℃=6℃. This result still meets the preset condition of being greater than 0℃, indicating that the thermal load of the current candidate set is still within the safe range, so the expansion process continues.

[0036] Next, the task of ray-tracing test with the highest power consumption is attempted to be added to the candidate set containing (matrix multiplication test, FFT test). Meanwhile, the combined resource requirement vector is updated again, and a new round of thermal stress assessment is performed. Since the ray-tracing test itself is extremely heavy, the superposition of its power consumption with the power consumption of the previous two tasks causes the peak thermal stress output by the heat conduction operation to reach 98°C. When checking the test safety margin at this moment, the calculation result is 95°C-98°C=-3°C. At this time, the test safety margin does not meet the preset condition of being greater than 0°C. This result clearly indicates that adding the ray-tracing test to the current candidate set will cause the chip temperature to exceed the safety threshold. Therefore, the expansion of the candidate set of the current parallel test execution cluster is terminated, and the ray-tracing test task that was last attempted to be added is removed from the candidate set. Finally, the candidate set containing (matrix multiplication test, FFT test) is confirmed as a stable and safe combination, and is formally aggregated into a parallel test execution cluster, waiting for subsequent resource mapping and scheduling execution.

[0037] In this embodiment, the intelligent construction of parallel test task combination is realized by using the iterative aggregation method with safety margin as closed-loop feedback. The maximum parallel task capacity of each scheduling batch can be adaptively explored and determined under the premise of ensuring chip safety, thereby improving the throughput and device utilization of GPU parallel testing.

[0038] Further, the specific implementation process of planning a GPU test scheduling diagram for the spatial mapping of each test task to a GPU physical execution unit and the execution time window of the parallel test execution cluster includes: According to the test task-resource requirement matrix, a resource view containing the GPU topology structure and the real-time available state of each physical execution unit is obtained, and the resource requirements of each test task contained in the parallel test execution cluster for the physical execution unit are extracted; the resource requirements are input into the resource view for fitting operation to obtain a spatial mapping; the spatial mapping contains each test task in the parallel test execution cluster that should be loaded to a GPU physical execution unit; according to the data dependency relationship between the test tasks, the execution time interval of each test task in the spatial mapping is determined to form an execution time window; the spatial mapping and the execution time window are integrated to generate a GPU test scheduling diagram containing complete scheduling instructions.

[0039] The parallel test execution cluster to be planned is (matrix multiplication test, FFT test). First, a GPU resource view is generated in real time by querying a GPU driver program interface. The resource view not only contains the static topology structure of the GPU having 32 streaming multiprocessors (SMs), but also contains the real-time available state of each physical execution unit. For example, at this moment, all 32 SMs are in an idle state, more than 14 GB of available space is available for the 16 GB of video memory, and the PCIe bus and memory controller are in standby state.

[0040] From the constructed test task-resource requirement matrix, we extracted the quantitative resource requirements of the physical execution units for the matrix multiplication and FFT tests within the execution cluster. The matrix multiplication test's requirement vector indicated that it required the parallel computing power of 16 SMs and 1.5GB of video memory; the FFT test required 8 SMs with TensorCore functionality and 0.5GB of video memory.

[0041] The resource requirement vectors of each task within the parallel test execution cluster are matched to a real-time resource view that includes the GPU's static physical topology and dynamic availability. A fitting operation is performed. A constraint satisfaction search is performed to generate preliminary allocation plans that meet basic requirements, such as compute unit type, number, and memory capacity. Among all feasible plans, a multi-objective optimization strategy is used to prioritize those that maximize communication locality (i.e., mapping the compute units required by a single task to physically contiguous or adjacent areas) and minimize resource fragmentation. This translates the abstract task requirements into an efficient execution layout that is tightly coupled to the hardware. The resulting calculations produce a clear spatial mapping: matrix multiplication test tasks are assigned to SM0 through SM15 for execution, while FFT test tasks are assigned to SM16 through SM23 for execution. This spatial mapping defines the physical "location" of each test task on the chip.

[0042] At the same time, the execution window is determined based on the logical relationship between tasks. The FFT test is designed to verify the spectral characteristics of the matrix multiplication test results, so there is a clear data dependency between the two. That is, the FFT test must begin only after the matrix multiplication test completes and produces the result matrix. The matrix multiplication test takes approximately 120ms to execute on 16 SMs. Based on this data dependency, an execution time interval is determined for each test task in the spatial mapping: the execution window for the matrix multiplication test is set from T0, the start of scheduling, to T0+120ms; the execution window for the FFT test is set from T0+120ms to T0+200ms immediately thereafter (its own execution takes 80ms). This series of non-overlapping time intervals together constitutes the execution window.

[0043] The aforementioned space mapping is integrated with the determined execution time window to generate a GPU test scheduling chart containing complete scheduling instructions. The scheduling chart is a structured data object, and its content can be directly issued to the underlying task scheduler of the GPU. For example, one instruction explicitly describes: "at T0, load the calculation kernel of the matrix multiplication test to SM0-SM15, transfer the input data from the host memory to the allocated 1.5 GB video memory area, and start execution"; another instruction is: "focus on the test situation of SM0-SM15, once the completion signal is received, load the calculation kernel of the FFT test to SM16-SM23, point its input to the video memory address of the result matrix of the matrix multiplication test, and start execution".

[0044] The embodiment generates a GPU test scheduling chart containing accurate space mapping and execution time window for the aggregated parallel test execution cluster, and converts the originally abstract and loose parallel task set into a specific hardware execution instruction set. It ensures that each task can obtain the necessary calculation unit and video memory, effectively reduces the on-chip communication delay, and improves the overall efficiency of parallel execution.

[0045] Further, the fault causal trace chain is set for the parallel test task, and the specific implementation process of associating and binding the GPU test scheduling chart with the real-time execution log of the corresponding physical execution unit of the GPU includes: According to the GPU test scheduling chart, the data structure of the fault causal trace chain is initialized; when the test task is started, the execution log data stream generated from each physical execution unit of the GPU is captured in real time; the execution log data stream is parsed and each execution instance of the test task is matched with a causal node, which records the task identification, the predetermined physical execution unit mapping relationship and the execution time window information; the matched execution instance and the causal node are associated and bound to generate a trace information unit containing scheduling intention and execution live; all trace information units are sequentially linked according to the execution time window sequence of the test tasks in the GPU test scheduling chart to construct the fault causal trace chain.

[0046] When the GPU test scheduling chart generated by the execution cluster is ready to be issued (matrix multiplication test, FFT test), the data structure of the fault causal trace chain is initialized in the memory according to the content of the scheduling chart. The data structure contains two sequentially linked causal nodes in the initial state, corresponding to the matrix multiplication test and the FFT test. The first causal node preloads the task identification of the matrix multiplication test, its predetermined physical execution unit mapping relationship (i.e. SM0 to SM15) and its execution time window information (i.e. 120 ms from T0).

[0047] When the GPU starts to execute the test task at T0 moment, the execution log data stream generated by each physical execution unit is captured from the hardware performance monitor and debug register of the GPU in real time, which contains the underlying events with nanosecond-level timestamp. For example: "SM5 executes floating-point multiply-add instruction at T0+15ms" and "memory controller reports address misaligned access at T0+45ms". When the log from SM5 with timestamp T0+15ms is parsed, the current causal node is immediately queried and determined as the node corresponding to the matrix multiplication test. Since the source (SM5) and timestamp (T0+15ms) of the log fall within the predetermined physical mapping relationship (SM0-SM15) and execution time window (T0 to T0+120ms) recorded by the causal node, it is determined that the matching is successful.

[0048] After the matching is successful, the specific execution log data stream is associated and bound to the causal node of the matrix multiplication test as "execution live", and the node itself records the "scheduling intention". This binding operation generates a complete trace information unit, which integrally contains all information of what is planned to do (scheduling intention) and what actually happens (execution live). As the test proceeds, all logs within the execution time window of the matrix multiplication test and from its mapping unit will be continuously bound to the trace information unit. The trace information units are permanently and orderly linked together in the order of the preset execution time window in the GPU test scheduling diagram, so that the underlying event stream has clearer logical relationship and business meaning. When the matrix multiplication test is completed, the next node in the trace chain is automatically moved to, and the execution instance of the FFT test starts to match and bind the logs from SM16 to SM23. Finally, when the entire execution cluster completes the test, a complete, coherent and one-to-one corresponding fault causal trace chain with the scheduling plan is constructed.

[0049] The embodiment constructs a fault causal trace chain for each parallel test task execution, so that the test execution process is no longer a "black box". This not only helps to quickly diagnose failures, but also can be used to analyze performance bottlenecks and detect potential hardware unstable behaviors, thereby providing strong and structured data support for chip yield analysis, driver optimization and firmware debugging, and shortening the chip verification and yield improvement cycle.

[0050] Further, the specific implementation process of analyzing based on the fault causal trace chain when the test task fails, outputting the fault source context causing the failure and troubleshooting the failure cause includes: Upon receiving the fault signal of test failure, the end trace information unit directly associated with the fault event is retrieved and locked in the fault causal trace chain according to the timestamp carried in the fault signal; the end trace information unit is taken as a parsing starting point, and reverse tracing is performed along the fault causal trace chain; during the reverse tracing, the GPU real-time execution log segments and the corresponding GPU test scheduling graph information encapsulated by the respective preceding trace information units are batch extracted; the extracted multiple real-time execution log segments and scheduling graph information are time-aligned and state-associated, a continuous GPU execution state sequence before and after the fault occurrence point is reconstructed, and a fault source context is output; the fault source context is subjected to abnormality detection, and the fault cause is investigated.

[0051] With reference to Fig. 3 When the execution cluster containing (matrix multiplication test, FFT test) sends a fault signal of test failure during the test process. The fault signal contains a timestamp T0+125.7ms, an error code 0x1F04 (representing illegal memory access), and a physical hardware unit ID reporting the exception.

[0052] According to the timestamp T0+125.7ms carried in the fault signal, a quick search is performed in the fault causal trace chain constructed. Since the time point falls within the execution time window (T0+120ms to T0+200ms) of the FFT test task, the trace information unit associated with the FFT test execution instance is immediately locked and marked as the end trace information unit directly associated with the fault event.

[0053] The FFT test trace information unit that is locked is taken as a parsing starting point, and reverse tracing is performed along the link pointer previously established in the fault causal trace chain. The tracing operation will backtrack from the FFT test node to the trace information unit of the preceding matrix multiplication test.

[0054] During the reverse tracing, a batch data extraction operation is performed. From the FFT test trace information unit that is the parsing starting point, all GPU real-time execution log segments from T0+120ms to the fault occurrence time T0+125.7ms encapsulated by the FFT test trace information unit and the corresponding scheduling graph information, i.e., the planning of the execution of the FFT test on SM16 to SM23, are extracted. From the backtracked preceding matrix multiplication test trace information unit, the execution log segments from T0 to T0+120ms and the scheduling planning of the execution on SM0 to SM15 are extracted.

[0055] The two extracted real-time execution log segments, which are continuous in time but logically separated, are time-aligned and state-associated with their respective scheduling graph information. Thus, the discrete underlying logs are spliced and restored into a continuous GPU execution state sequence from T0 to the failure point T0+125.7ms, with high-level scheduling intent annotations. For example: the reconstructed sequence clearly shows that at T0+119.8ms, the matrix multiplication test writes a pointer value to its scheduled output memory address at the end of its execution, but the value is 0x00000000 (null pointer); immediately after, at T0+125.6ms, the FFT test attempts to read input data from the above address according to the data dependency relationship in the scheduling graph, triggering an illegal memory access exception of the physical hardware unit. This complete reconstruction of the execution sequence containing the cause-and-effect relationship is the failure source context.

[0056] The built-in exception detection function performs in-depth analysis on this output failure source context. Through preset diagnostic rules, such as: "detecting whether there is a pattern of writing a null pointer followed by a read operation". The error output of the matrix multiplication test is quickly identified as the root cause of the failure of the FFT test. Finally, a clear diagnostic report is generated, pointing out that the direct cause of the failure is the illegal memory access of the FFT test, and the root cause lies in the logical defect of the matrix multiplication test task itself, thus completing the accurate investigation of the failure cause.

[0057] This embodiment analyzes based on the fault cause and effect traceability chain when the test fails, and directly locks the end trace information unit by using the timestamp of the fault signal. Blind search in the data log is avoided, and the history execution segment directly related to the fault can be automatically and selectively focused on, the interference of irrelevant information is weakened, and the chip development and verification period is shortened.

[0058] This embodiment improves passive overheating protection to active risk avoidance by introducing predictive safety checks based on accurate power and thermal models. It effectively prevents tasks that may cause the chip temperature to exceed the limit from being incorrectly aggregated, and avoids physical damage to the chip caused by improper test load stacking.

[0059] Through intelligent task aggregation, the number of tasks executed in parallel can be maximized while ensuring chip safety. Combining multiple tasks into parallel execution clusters shortens the test period and improves the throughput of the chip test platform.

[0060] By constructing the "fault cause traceability chain" that strongly correlates the scheduling intention and the execution live, and using the precise matching mechanism of hardware ID marking, the debugging efficiency is effectively improved. Thus, in a complex concurrent execution environment, the complete event sequence related to the fault can be automatically and quickly stripped from the underlying log, and the root cause of the fault can be accurately traced, thereby shortening the development verification and yield improvement cycle of the chip.

[0061] Embodiment two: This embodiment will deploy the above-mentioned parallel test task scheduling method for high-computing-power GPU chips in the X-chip test center, and realize parallel testing of GPU chips.

[0062] In the initial stage of the test process, the scheduling server receives a test task set containing multiple independent test cases. Specifically, it includes matrix multiplication tests for verifying the peak performance of CUDA cores, fast Fourier transform tests for verifying the dedicated TensorCore unit, and high-load ray tracing tests. The syntax analysis function uses a static code analyzer based on LLVM to quantify the execution logic and resource requirements of the task. For the matrix multiplication test, the proportion of floating-point operation instructions and global memory access instructions in the CUDA kernel is analyzed (about 8.2 according to statistics). When this proportion is higher than the preset threshold of 5.0, the computing core requirement is accurately quantified as "CUDA core: high". By scanning the calling parameters of all cudaMalloc functions in the code and summing them up, the memory capacity requirement is quantified as 1.5GB; combined with the thread configuration and memory access mode, the average memory bandwidth requirement is predicted to be about 80GB / s through the built-in empirical estimation model. The fast Fourier transform test is quantified as "TensorCore: medium" because it calls the TensorCore dedicated API and determines that it needs 0.5GB of memory. The quantified information is integrated into the resource requirement vector and filled into the test task-resource requirement matrix.

[0063] The real-time state of the GPU chip is obtained through the management interface, and it is known that the initial temperature of all stream multiprocessors (SMs) is 42°C, the power consumption baseline of each SM is 5 watts, and the design safety temperature threshold of the chip is 95°C. According to the test task-resource requirement matrix constructed in the previous step, the power consumption superposition distribution generated by the combination of tasks is calculated using a multiple linear regression power model. The power model describes the power consumption of each SM unit as a linear combination of the baseline power consumption and multiple variables such as CUDA core occupancy and TensorCore unit activity level. The coefficients are fitted by conducting a large number of benchmark tests on the actual chip. After calculation, if the matrix multiplication test is scheduled on SM0 to SM15, the power consumption of these units will increase to 35 watts, and the power consumption of SM16 to SM23 executing the fast Fourier transform test will increase to 27 watts. This non-uniform power distribution map is then input as a heat source for heat conduction calculation. A heat conduction model based on two-dimensional finite element analysis is used to abstract the physical layout of the chip into a grid and assign thermal conductivity parameters to key materials such as silicon. At the same time, the heat exchange with the heat dissipation system is simulated through a boundary condition of equivalent thermal resistance of 0.2 K / W. After a short transient thermal simulation, the results show that the temperature in the SM8 region will reach a peak of 88°C at about 480 ms. Finally, the test safety margin is calculated to be 7°C (95°C-88°C), so it is determined that this combination of tasks is safe in terms of heat.

[0064] According to the calculated test safety margin, a parallel test execution cluster is constructed. To ensure the determinism and reproducibility of the scheduling decision, a multi-level ordering scheduling strategy is adopted: all tasks to be tested are first sorted in descending order of predefined priority, and if the priority is the same, they are sorted in ascending order of required memory capacity. First, the matrix multiplication test with the highest priority is selected and added to the candidate set. The peak temperature of its separate execution is 70°C after thermal verification, leaving a remaining safety margin of 25°C. Then, the fast Fourier transform test with the next priority is added to the candidate set, and the joint thermal verification peak temperature is 88°C, leaving a remaining safety margin of 7°C, which still meets the condition of being greater than the preset buffer value of 5°C. When trying to add the power-hungry ray tracing test, the joint thermal verification predicts a peak temperature of 99°C, which exceeds the preset safety threshold. Therefore, this addition is rejected and the expansion of the current execution cluster is terminated. Finally, a parallel test execution cluster containing the matrix multiplication test and the fast Fourier transform test is officially created.

[0065] After the cluster construction is performed, a GPU test schedule graph is planned for it, including space mapping and execution time window. The calculation kernel of the matrix multiplication test is assigned to SM0 to SM15, and the fast Fourier transform test is mapped to SM16 to SM23. Considering that the fast Fourier transform test needs to use the calculation result of the matrix multiplication test, and the execution time of 120 ms is estimated according to the performance model, the execution time window of T0 to T0+120 ms is set for the matrix multiplication test, and the execution time window of T0+120 ms to T0+200 ms is set for the fast Fourier transform test. These accurate space and time instructions are integrated into a structured schedule graph object for delivery to the GPU driver for execution.

[0066] In the test execution phase, to ensure the accurate association of logs and scheduling intentions, a hardware-assisted matching mechanism is used when the schedule graph is delivered: unique runtime IDs (such as 0x1A01 and 0x1A02) are assigned to the matrix multiplication test and the fast Fourier transform test, respectively, and these IDs are written into specific context registers of the SM units where the tasks will be executed. The hardware log system of the chip is configured to automatically attach the ID of the current context when generating any event record. In an actual test, the matrix multiplication test wrote a null pointer to its output address at T0+119.8 ms due to a logical defect, and the write operation log was automatically marked with ID: 0x1A01. Then, the fast Fourier transform test started at T0+125.6 ms and tried to read data from the address, triggering an illegal memory access interrupt of the physical hardware unit, and the related log was marked with ID: 0x1A02. When receiving the interrupt signal, according to the timestamp and the ID in the log, the fast Fourier transform test node where the fault occurred is immediately locked in the pre-constructed trace chain, and the preceding matrix multiplication test node is traced back along the chain. By batch extracting all log fragments marked as 0x1A01 and 0x1A02 and reconstructing the time sequence, the complete fault sequence from "wrong writing" to "illegal reading" is automatically clearly restored. The final diagnostic report clearly indicates that the direct cause of the fault is the illegal memory access of the fast Fourier transform test, and the root cause is the wrong output of the matrix multiplication test.

[0067] It should be noted that the embodiments described above are only exemplary, and the purpose is to help understand the application, not to limit the application. Those skilled in the art can make various changes and improvements after understanding the core idea of the application. Therefore, the protection scope of the application is defined by the appended claims and their equivalents.

Claims

1. A parallel test task scheduling method for a large-computing-power GPU chip, characterized in that, The application comprises the following steps: parsing a test task set and constructing a test task-resource requirement matrix, which contains resource requirements of each test task on GPU internal physical execution units and matching relationship with test targets; calculating transient thermal stress generated by test tasks on GPU according to real-time state of GPU, test targets and the test task-resource requirement matrix, and checking test safety margin, aggregating multiple test tasks into parallel test execution clusters according to the test safety margin; planning spatial mapping of each test task to GPU physical execution units and GPU test scheduling diagram of execution time window for the parallel test execution clusters; setting fault cause-effect traceability chain for the parallel test tasks, associating and binding the GPU test scheduling diagram with real-time execution log of corresponding physical execution units of GPU; when test task fails, analyzing based on the fault cause-effect traceability chain, outputting fault source context causing failure and troubleshooting fault cause.

2. The parallel test task scheduling method for a large-computing-power GPU chip according to claim 1, characterized in that, The specific implementation process of parsing the test task set and constructing the test task-resource requirement matrix comprises: extracting execution logic and resource requirement description of all test tasks through syntax analysis; identifying calculation core type and storage resource type required to be called by the test task according to the execution logic and quantifying as function parameters; obtaining video memory capacity and memory bandwidth required for task execution according to the resource requirement description and quantifying as constraint conditions; integrating the function parameters and constraint conditions to generate a resource requirement vector for the test task; creating a data matrix with all test tasks as the first dimension and GPU internal schedulable physical execution units and resource types as the second dimension; filling the resource requirement vector corresponding to the test task into the data matrix to construct the test task-resource requirement matrix.

3. The parallel test task scheduling method for a large-computing-power GPU chip according to claim 1, characterized in that, The specific implementation process of calculating transient thermal stress generated by test tasks on GPU according to real-time state of GPU, test targets and the test task-resource requirement matrix, and checking test safety margin comprises: obtaining real-time state of GPU containing current temperature and power consumption baseline of physical execution units, combining resource requirement vector corresponding to test tasks in the test task-resource requirement matrix to calculate power consumption superposition value generated when executing test tasks on each physical execution unit of GPU; inputting the power consumption superposition value as a heat source for heat conduction operation to output transient thermal stress; extracting working temperature safety threshold of GPU from test targets, performing difference operation between peak thermal stress in the transient thermal stress and the working temperature safety threshold to obtain test safety margin.

4. The parallel test task scheduling method for a large-computing-power GPU chip according to claim 1, characterized in that, The specific implementation process of aggregating multiple test tasks into parallel test execution clusters according to the test safety margin comprises: According to a preset scheduling strategy, a test task is selected from a test task set and added to a parallel test execution cluster candidate set, taking a test safety margin as a core constraint; a resource demand vector of all test tasks in the parallel test execution cluster candidate set is updated based on the test task-resource demand matrix, transient thermal stress generated on a GPU is re-evaluated, and a remaining test safety margin is checked; when the test safety margin still meets a preset condition, a test task is continuously selected from the test task set and added to the parallel test execution cluster candidate set; when the test safety margin does not meet the preset condition, expansion of the current parallel test execution cluster candidate set is terminated, and the parallel test execution cluster is aggregated.

5. The parallel test task scheduling method for a large-computing-power GPU chip according to claim 1, wherein, A specific implementation process of planning a spatial mapping of each test task to a physical execution unit of a GPU and a GPU test scheduling graph of an execution time window includes: According to the test task-resource demand matrix, a resource view containing a GPU topology structure and a real-time available state of each physical execution unit is obtained, and resource demands of each test task contained in the parallel test execution cluster for the physical execution unit are extracted; the resource demands are input into the resource view for fitting operation, and a spatial mapping is obtained; the spatial mapping contains each test task in the parallel test execution cluster and should be loaded to the physical execution unit of the GPU; an execution time interval is determined for each test task in the spatial mapping according to a data dependency relationship between the test tasks, and an execution time window is formed; the spatial mapping and the execution time window are integrated, and a GPU test scheduling graph containing complete scheduling instructions is generated.

6. The parallel test task scheduling method for a large-computing-power GPU chip according to claim 1, wherein, A specific implementation process of setting a fault causal trace chain for the parallel test task and associating and binding the GPU test scheduling graph with a real-time execution log of a corresponding physical execution unit of the GPU includes: According to the GPU test scheduling graph, a data structure of the fault causal trace chain is initialized; when a test task is started to be executed, an execution log data stream generated from each physical execution unit of the GPU is captured in real time; the execution log data stream is parsed, and a causal node is matched for each execution instance of the test task, the causal node recording a task identifier, a predetermined physical execution unit mapping relationship and execution time window information; the execution instance that is successfully matched is associated and bound with the causal node, and a trace information unit containing a scheduling intention and an execution live is generated; all trace information units are sequentially linked according to an execution time window order of the test task in the GPU test scheduling graph, and the fault causal trace chain is constructed.

7. The parallel test task scheduling method for a large-computing-power GPU chip according to claim 1, wherein, A specific implementation process of analyzing, when a test task fails, a fault source context causing the failure based on the fault causal trace chain and outputting the fault source context and troubleshooting a fault cause includes: After receiving the fault signal of test failure, according to the time stamp carried in the fault signal, the end trace information unit directly associated with the fault event is retrieved and locked in the fault causal trace chain; the end trace information unit is taken as the analysis starting point, and reverse tracing traversal is performed along the fault causal trace chain; in the reverse tracing traversal process, the GPU real-time execution log segments and the corresponding GPU test scheduling graph information encapsulated by the respective previous trace information units are batch extracted; the extracted multiple real-time execution log segments and the scheduling graph information are time-aligned and state-associated, the continuous GPU execution state sequence before and after the fault occurrence point is reconstructed, and the fault source context is output; the fault source context is subjected to abnormality detection to investigate the fault cause.

Images