A GPU thread state information sampling monitoring system, method and GPU

Abstract

The application discloses a GPU thread state information sampling monitoring system, method and GPU, comprising: a stream multiprocessor group composed of a plurality of serially connected stream multiprocessors; a global sampling unit arranged in the stream multiprocessor group and used for detecting whether there is an active thread in each stream multiprocessor before executing each round of sampling operation, determining a target stream multiprocessor of the present round of sampling according to a detection result, and issuing a thread state sampling command to the target stream multiprocessor according to a first sampling period; and a plurality of local sampling units, each of which is arranged in a stream multiprocessor and used for sampling the state of each thread in the corresponding target stream multiprocessor according to a second sampling period after receiving the thread state sampling command issued by the global sampling unit, and uploading the collected thread state information to the global sampling unit. The application can realize fine-grained monitoring of thread states, and greatly reduce the data volume and bandwidth occupation.

Description

Technical Field

[0001] This application relates to the field of computing processor technology, and in particular to a GPU thread state information sampling and monitoring system, method, and GPU. Background Technology

[0002] GPUs (also known as parallel computing processors or graphics processing units) are the core processing devices for high-performance parallel computing. They contain hundreds or thousands of concurrently executing threads. Accurate monitoring of the running status of these threads is of great significance for GPU performance optimization, debugging analysis, and reliability assurance.

[0003] However, existing GPU thread state sampling and monitoring technologies can only provide global statistics using performance counters, failing to capture the fine-grained state of individual threads. While thread tracing can achieve fine-grained monitoring, it generates massive amounts of data, severely consuming memory bandwidth and impacting GPU execution efficiency. In summary, current technologies cannot achieve fine-grained, low-overhead monitoring of GPU internal thread states without excessively consuming system resources.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] The purpose of this application is to provide a GPU thread state information sampling and monitoring system, method, and GPU to solve the problem that the existing technology cannot achieve fine-grained, low-overhead monitoring of the internal thread state of the GPU without excessively consuming system resources.

[0006] The first aspect of this application provides a GPU thread state information sampling and monitoring system, including: A streaming multiprocessor group consists of multiple serially connected streaming multiprocessors within a GPU. A global sampling unit is deployed in the streaming multiprocessor group and communicates with each streaming multiprocessor. Before each round of sampling operation, it detects whether there are active threads in each streaming multiprocessor, determines the target streaming multiprocessor for this round of sampling based on the detection results, and issues a thread status sampling command to the target streaming multiprocessor according to the first sampling period. Multiple local sampling units are deployed in a stream multiprocessor and are communicatively connected to each thread in the corresponding stream multiprocessor and the global sampling unit. After receiving a thread state sampling command issued by the global sampling unit, the local sampling unit samples the state of each thread in the corresponding target stream multiprocessor according to the second sampling period and uploads the collected thread state information to the global sampling unit.

[0007] In some embodiments of this application, the serial connection order of each stream multiprocessor in the stream multiprocessor group is consistent with the order in which the global sampling unit issues thread state sampling commands to the target stream multiprocessor and the order in which the global sampling unit receives thread state information.

[0008] In some embodiments of this application, the rules by which the global sampling unit determines the target stream multiprocessor for this round of sampling include: If an active thread is detected in one or more of the streaming multiprocessors in the streaming multiprocessor group, the first streaming multiprocessor in which an active thread is detected is identified as the target streaming multiprocessor. If no active thread is detected in one or more stream multiprocessors in the stream multiprocessor group, the second stream multiprocessor in which no active thread is detected will be excluded from the target stream multiprocessor range of this round of sampling, and no thread state sampling command will be issued to the second stream multiprocessor, and the last active thread in the second stream multiprocessor has ended running.

[0009] In some embodiments of this application, the system further includes: If it is not detected in time that there is only one active thread about to terminate in one or more stream multiprocessors in the stream multiprocessor group, and the third stream multiprocessor with only one active thread about to terminate is still identified as the target stream multiprocessor, then the local sampling unit deployed in the third stream multiprocessor will upload bubbling data to the global sampling unit after receiving the thread state sampling command issued by the global sampling unit. The bubbling data is used to indicate that there is no valid thread state information in this round of sampling.

[0010] In some embodiments of this application, the first sampling period is the time interval between two thread state sampling commands issued by the global sampling unit to the same target stream multiprocessor. The first sampling period can be configured as a dynamic period or a fixed period. When a dynamic period is used, the first sampling period changes dynamically according to the number of stream multiprocessors with active threads in the target stream multiprocessor group.

[0011] In some embodiments of this application, the second sampling period is the local single-round sampling time and the theoretical sampling interval of a single thread; wherein, the local single-round sampling time is the total time taken for the local sampling unit to complete a full round of state sampling for all active threads in the corresponding target stream multiprocessor, which can be configured as a dynamic period or a fixed period, and the theoretical sampling interval of a single thread is the product of the first sampling period and the local single-round sampling time of the corresponding target stream multiprocessor, which is used to characterize the time interval between two consecutive samplings of a single thread; When a dynamic period is used, the local single-round sampling time is the product of the sampling time of a single thread and the number of active threads in the corresponding target stream multiprocessor, and is dynamically adjusted according to the number of active threads.

[0012] In some embodiments of this application, after each round of sampling operation, the global sampling unit updates the active thread detection results of each stream multiprocessor and adjusts the target stream multiprocessor range for the next round of sampling based on the updated detection results.

[0013] In some embodiments of this application, adjusting the target stream multiprocessor range for the next round of sampling based on the updated detection results includes: If no active threads are detected within the original target stream multiprocessor, they are excluded from the next round of target stream multiprocessor range, and the issuance of sampling commands is stopped. If an active thread is detected to reappear within a non-target stream multiprocessor, it will be included in the next round of target stream multiprocessor scope, and a sampling command will be issued according to the first sampling period.

[0014] In some embodiments of this application, the local sampling unit performs sampling and uploading in the following manner: After receiving the thread state sampling command issued by the global sampling unit, the state of each thread in the corresponding stream multiprocessor is sampled one by one according to the second sampling period, and the thread state information obtained from each sampling is temporarily stored in the local buffer. After sampling all target threads within the current sampling period, the multiple thread state information temporarily stored in the local buffer is packaged into a sampling data block, and the sampling data block is serially uploaded to the global sampling unit.

[0015] In some embodiments of this application, the thread state information collected by the local sampling unit includes at least one of thread emission blocking reasons and thread high-frequency change signals; the global sampling unit summarizes the thread state information uploaded by all target stream multiprocessors and outputs it to the GPU external storage unit.

[0016] The second aspect of this application provides a method for sampling and monitoring GPU thread state information, including: Multiple serially connected streaming multiprocessors in the GPU are grouped into a streaming multiprocessor group. A global sampling unit is deployed in the streaming multiprocessor group and communicates with each streaming multiprocessor. A local sampling unit is deployed in each streaming multiprocessor in the streaming multiprocessor group and communicates with each thread in the corresponding streaming multiprocessor and the global sampling unit. Before performing each round of sampling operations, the global sampling unit detects whether there are active threads in each stream multiprocessor, determines the target stream multiprocessor for this round of sampling based on the detection results, and issues a thread status sampling command to the target stream multiprocessor according to the first sampling period. After receiving the thread state sampling command issued by the global sampling unit, the local sampling unit performs state sampling on each thread in the corresponding target stream multiprocessor according to the second sampling period, and uploads the collected thread state information to the global sampling unit. The global sampling unit aggregates the thread state information uploaded by all target stream multiprocessors and outputs it to the GPU external storage unit.

[0017] A third aspect of this application provides a GPU including any of the systems described in the first aspect above, through which the sampling and monitoring of the running state of internal threads of the GPU is implemented.

[0018] A fourth aspect of this application provides a chip including a processor and a memory storing processor-executable instructions, wherein the processor executes the instructions to implement the steps of the method described in the second aspect above.

[0019] The fifth aspect of this application provides a computer storage medium storing computer program instructions that, when executed by a processor, implement the steps of the method described in the second aspect above.

[0020] The sixth aspect of this application provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the method described in the second aspect above.

[0021] This application discloses a GPU thread state information sampling and monitoring system, method, and GPU. The system may include: a streaming multiprocessor group, composed of multiple serially connected streaming multiprocessors within the GPU; a global sampling unit, deployed in the streaming multiprocessor group and communicatively connected to each streaming multiprocessor, used to detect the presence of active threads within each streaming multiprocessor before each round of sampling operations, and determine the target streaming multiprocessor for the current round of sampling based on the detection results, and issue a thread state sampling command to the target streaming multiprocessor according to a first sampling period; and multiple local sampling units, each deployed in a streaming multiprocessor and communicatively connected to each thread within the corresponding streaming multiprocessor and the global sampling unit, used to sample the state of each thread within the corresponding target streaming multiprocessor according to a second sampling period after receiving the thread state sampling command issued by the global sampling unit, and upload the collected thread state information to the global sampling unit. By using multiple serially connected streaming multiprocessors within the GPU to form a streaming multiprocessor group, a physical basis for serial data transmission can be provided, simplifying wiring complexity. By deploying global sampling units within the streaming multiprocessor group, active threads are detected and the target streaming multiprocessor for each sampling round is determined before each round of sampling. Sampling operations are only performed on streaming multiprocessors with active threads (target streaming multiprocessors), avoiding invalid sampling of streaming multiprocessors without active threads. This reduces data generation at the source and controls memory access bandwidth consumption. By deploying a local sampling unit in each streaming multiprocessor, fine-grained sampling of threads within each streaming multiprocessor can be achieved, breaking through the granularity limitations of traditional global performance counter statistics and enabling fine-grained monitoring of thread states. Furthermore, by standardizing the settings of the first and second sampling cycles, the problem of thread tracing technology recording all data irregularly can be avoided, further controlling bandwidth consumption and achieving a balance between monitoring granularity and bandwidth consumption. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is an architecture diagram of a GPU thread state information sampling and monitoring system provided in the embodiments of this specification; Figure 2 This is a flowchart illustrating a GPU thread state information sampling and monitoring method provided in the embodiments of this specification; Figure 3 This is a schematic diagram of the global sampling unit provided in the embodiments of this specification; Figure 4 This is a schematic diagram of the local sampling unit provided in the embodiments of this specification; Figure 5 This is a schematic diagram of a global sampling unit with no active threads, provided in the embodiments of this specification. Figure 6 This is a schematic diagram of data uploading and cavitation provided in the embodiments of this specification; Figure 7 This is a schematic diagram illustrating the structural composition of a GPU thread state information sampling and monitoring device provided in the embodiments of this specification; Figure 8 This is a schematic diagram of the structure of a chip provided in an embodiment of this specification. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0026] This specification provides the operational steps of the methods described in the embodiments or flowcharts, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or device products, the methods shown in the embodiments or drawings can be executed sequentially or in parallel.

[0027] It should be noted that in the embodiments of this specification, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of the present invention. However, they do not mean that the applicant has used or necessarily used such solutions.

[0028] As a high-performance parallel computing processor, a GPU contains hundreds or thousands of concurrently executing threads. When all these threads are running, it is not easy to observe their internal running state.

[0029] Currently, methods for obtaining the internal state of a thread include performance counters and thread tracing tools. Performance counters use hardware counters to count specific events (such as cache hit rate and instruction issuance), providing global statistics after program execution. However, this technique only provides overall statistics and cannot obtain fine-grained states of individual threads, making it difficult to pinpoint the cause of a thread's pause or abnormal behavior. Thread tracing techniques record the timestamps of every instruction and event executed by a thread, constructing a detailed runtime timeline. However, this technique generates massive amounts of log data, severely consuming memory bandwidth resources, impacting the normal memory access efficiency of the GPU, and posing significant challenges to subsequent data analysis and processing.

[0030] Therefore, existing technologies struggle to balance monitoring granularity, bandwidth usage, and hardware overhead. How to achieve fine-grained, low-overhead monitoring of the internal thread state of a GPU without excessively consuming system resources has become a pressing technical problem in this field.

[0031] To enrich the methods for monitoring GPU thread running status, this specification provides a GPU thread state information sampling and monitoring system. By using multiple serially connected streaming multiprocessors within the GPU to form a streaming multiprocessor group, a physical basis for serial data transmission is provided, simplifying wiring complexity. By deploying global sampling units within the streaming multiprocessor group, active threads are detected and the target streaming multiprocessor for each sampling round is determined. Sampling operations are performed only on streaming multiprocessors with active threads (target streaming multiprocessors), avoiding invalid sampling of inactive streaming multiprocessors. This reduces data generation at the source and controls memory access bandwidth consumption. By deploying a local sampling unit in each streaming multiprocessor, fine-grained sampling of threads within each streaming multiprocessor can be achieved, breaking through the granularity limitations of traditional global performance counter statistics and enabling fine-grained monitoring of thread states. Furthermore, by standardizing the settings of the first and second sampling cycles, the problem of irregular full data recording in thread tracing technology can be avoided, further controlling bandwidth consumption and achieving a balance between monitoring granularity and bandwidth consumption.

[0032] In summary, this invention constructs a core architecture that balances fine-grained monitoring of GPU thread states with low memory access bandwidth consumption. It can achieve coordinated global unified scheduling and local fine-grained sampling, and can dynamically determine sampling targets based on the active thread states within a streaming multiprocessor, reducing invalid sampling and data generation from the source. While acquiring fine-grained thread state information within a single streaming multiprocessor, it effectively controls memory access bandwidth usage, filling the technical gap in the prior art between monitoring granularity and bandwidth consumption.

[0033] The GPU thread state information sampling and monitoring system in the embodiments of this specification will be described in detail below with reference to the accompanying drawings.

[0034] Figure 1 This is an architecture diagram of a GPU thread state information sampling and monitoring system provided in the embodiments of this specification. The system may include: A streaming multiprocessor group consists of multiple serially connected streaming multiprocessors within a GPU. A global sampling unit is deployed in the streaming multiprocessor group and communicates with each streaming multiprocessor. Before each round of sampling operation, it detects whether there are active threads in each streaming multiprocessor, determines the target streaming multiprocessor for this round of sampling based on the detection results, and issues a thread status sampling command to the target streaming multiprocessor according to the first sampling period. Multiple local sampling units are deployed in a stream multiprocessor and are communicatively connected to each thread in the corresponding stream multiprocessor and the global sampling unit. After receiving a thread state sampling command issued by the global sampling unit, the local sampling unit samples the state of each thread in the corresponding target stream multiprocessor according to the second sampling period and uploads the collected thread state information to the global sampling unit.

[0035] Specifically, the GPU thread state information sampling and monitoring system may include a streaming multiprocessor group (SM group), a global sampling unit, and a local sampling unit.

[0036] The SM group, which can be formed by serially connecting multiple streaming multiprocessors (SMs) within the GPU, serves as the physical carrier for sampling, laying the foundation for serial transmission and unified scheduling. The global sampling unit (BSU) is deployed within the SM group, communicating bidirectionally with each SM. It executes three core actions: "pre-sampling detection → target SM determination → command issuance according to the first sampling cycle." The detection action can be performed before each round of sampling to ensure the real-time nature of the sampling target, and the command issuance can be performed according to the first sampling cycle to ensure the regularity of sampling. Multiple local sampling units, deployed one-to-one within each SM, communicate bidirectionally with the threads within the corresponding SM (i.e., the thread warps in the corresponding SM thread group warps) and the global sampling unit. They execute three core actions: "receiving commands → sampling according to the second sampling cycle → uploading data." The sampling action can be linked with global commands, and the data upload can interface with the global sampling unit, enabling coordinated sampling and scheduling.

[0037] In this multiprocessor stream (SM), the local sampling unit is responsible for sampling and collecting the state information of warps within its SM, while the global sampling unit is responsible for collecting the warp state information transmitted from the local sampling unit of each SM. That is, the global sampling unit samples the SM, while the local sampling unit samples the warps within the SM. Since it doesn't require generating a large amount of sampling information, and since the SMs in the group are transmitted serially or in series, the global sampling unit samples each SM periodically (it can perform periodic sampling of each SM according to a first sampling period). Similarly, the local sampling unit also performs periodic sampling of active warps within the SM (it can perform periodic sampling of warps within each SM according to a second sampling period).

[0038] In this embodiment, before performing each round of sampling operations, the global sampling unit can detect whether there are active threads in each stream multiprocessor. Based on the detection results, it determines the target stream multiprocessor for this round of sampling, and then sequentially issues or sends thread state sampling commands to each target stream multiprocessor in a round-robin manner (i.e., issues thread state sampling commands to the target stream multiprocessors according to the first sampling period to perform periodic sampling on the target stream multiprocessors, such as SM0→SM1→SM2→……→SMN). The thread state sampling commands can be issued through a dedicated control path without occupying the data path for normal instruction execution.

[0039] Upon receiving a thread state sampling command, the local sampling unit of the target stream multiprocessor immediately initiates internal round-robin sampling, sequentially accessing the currently active warp (i.e., periodically sampling the state of each thread within the corresponding target stream multiprocessor according to the second sampling cycle, such as warp0→warp1→warp2→……→warpN). The local sampling unit of the target stream multiprocessor then serially uploads the collected thread state information to the global sampling unit (e.g., SM0→SM1→SM2→……→SMN→Global Sampling Unit). The global sampling unit can receive the thread state information uploaded by each target stream multiprocessor and write it to the GPU's external storage unit, completing this round of sampling and monitoring.

[0040] By constructing a core architecture that balances fine-grained monitoring of GPU thread states with low memory access bandwidth consumption, fine-grained monitoring of GPU thread states can be achieved. At the same time, by dynamically determining the target stream multiprocessor for this round of sampling, invalid sampling can be avoided and memory access bandwidth consumption can be effectively controlled.

[0041] In some embodiments, the serial connection order of each stream multiprocessor in the above-mentioned stream multiprocessor group is consistent with the order in which the global sampling unit issues thread state sampling commands to the target stream multiprocessor and the order in which the global sampling unit receives thread state information.

[0042] Specifically, based on the serial connection of the streaming multiprocessor group, it can be ensured that the serial connection order is completely consistent with the command issuance order and data reception order of the global sampling unit. For example, if the streaming multiprocessor group is serially connected in the order of SM1→SM2→SM3→SM4, the global sampling unit issues sampling commands to the target streaming multiprocessor in this order and simultaneously receives the thread status information uploaded by each streaming multiprocessor in this order. In this way, the serial matching of sampling command issuance and data transmission can be achieved, avoiding command errors and data reception conflicts caused by inconsistent order. This can improve the scheduling efficiency of the global sampling unit and the orderliness of data transmission, while simplifying the internal communication logic of the GPU and reducing the complexity of hardware implementation.

[0043] In some embodiments, the rules by which the global sampling unit determines the target stream multiprocessor for this round of sampling may include: If an active thread is detected in one or more of the streaming multiprocessors in the streaming multiprocessor group, the first streaming multiprocessor in which an active thread is detected is identified as the target streaming multiprocessor. If no active thread is detected in one or more stream multiprocessors in the stream multiprocessor group, the second stream multiprocessor in which no active thread is detected will be excluded from the target stream multiprocessor range of this round of sampling, and no thread state sampling command will be issued to the second stream multiprocessor, and the last active thread in the second stream multiprocessor has ended running.

[0044] Specifically, since all warps do not start and end at the same time, the number of active warps within a single SM varies at different times. Therefore, the sampling time interval for each warp is not constant; once the last warp of an SM in the previous sampling round finishes running, the global sampling unit in the next round will cancel the sampling command issued to that SM.

[0045] In this embodiment, if the global sampling unit detects that there are active threads in one or more stream processors in the stream multiprocessor group, it determines the first stream multiprocessor with active threads as the target stream multiprocessor. If it detects that there are no active threads in one or more stream multiprocessors in the stream multiprocessor group, it excludes the second stream multiprocessor with no active threads from the target stream multiprocessor range of this round of sampling, and does not issue a thread state sampling command to the second stream multiprocessor. The last active thread in the second stream multiprocessor has already ended running before this round of sampling.

[0046] If the global sampling unit detects that there are active threads in SM1, SM2, and SM3, and detects that the last thread of SM4 has finished running, then SM1, SM2, and SM3 are each regarded as a target stream multiprocessor, and SM4 is excluded from the target stream multiprocessor range. In this round of sampling, no thread status sampling command is issued to SM4. If an active thread is detected in SM4 again in the future, it can be added back to the target stream multiprocessor range.

[0047] By refining the rules for determining target stream multiprocessors in the global sampling unit, clarifying the exclusion criteria and operation methods for invalid sampling targets, and further avoiding invalid scheduling and sampling of stream multiprocessors without active threads, the generation of such invalid data can be completely eliminated from the source, further reducing the invalid occupation of memory access bandwidth. At the same time, making the target determination logic clearer can improve the operability of system execution.

[0048] In some embodiments, the system may further include: If it is not detected in time that there is only one active thread about to terminate in one or more stream multiprocessors in the stream multiprocessor group, and the third stream multiprocessor with only one active thread about to terminate is still identified as the target stream multiprocessor, then the local sampling unit deployed in the third stream multiprocessor will upload bubbling data to the global sampling unit after receiving the thread state sampling command issued by the global sampling unit. The bubbling data is used to indicate that there is no valid thread state information in this round of sampling.

[0049] Specifically, due to the lack of uniform synchronization between the start and end times of threads within the GPU, some streaming multiprocessors may exhibit a state where only one active thread remains, and that thread is about to finish running. However, during the active thread detection phase before each round of sampling, the global sampling unit may fail to capture or detect this dynamic change in a timely manner. Unable to perceive in real-time that all active threads within the streaming multiprocessor are about to finish, it will still classify such streaming multiprocessors as third-level streaming multiprocessors with active threads based on conventional detection results, and include them in the target streaming multiprocessor range for this round of sampling, while simultaneously issuing a thread state sampling command to them. When this sampling command is transmitted to the third-level streaming multiprocessor, its only remaining active thread has already finished running, and there are no active threads with collectable state information. At this point, the sampling command becomes invalid, and the corresponding local sampling unit has no valid thread state information to collect and upload. For this type of scenario, after receiving the sampling command issued by the global sampling unit, the local sampling unit in the third-stream multiprocessor will upload cavitation data to the global sampling unit. This cavitation data is signal data with a clear identification function, which can directly feed back the result of "no valid thread status information in this round of sampling" to the global sampling unit, allowing the global sampling unit to quickly identify this type of invalid sampling scenario.

[0050] If the global sampling unit detects one active thread in SM2 before the current sampling round but fails to detect that the thread is about to terminate, it identifies SM2 as a third-stream multiprocessor and includes it in the target stream multiprocessor scope, issuing a sampling command. When the sampling command is transmitted to SM2, the single active thread has terminated, and there are no active threads in SM2. At this time, the local sampling unit deployed in SM2, upon receiving the sampling command, will upload bubbly data to the global sampling unit. The global sampling unit can directly determine from this bubbly data that the current sampling of SM2 is invalid, and there is no valid data to receive and process. If the global sampling unit subsequently detects that a new active thread has restarted in SM2, it can reinstate it into the target stream multiprocessor scope and resume issuing sampling commands.

[0051] By establishing targeted data upload rules for abnormal scenarios involving global sampling unit detection latency, and clarifying the upload method for bubble data without valid thread state information and the core identification function of bubble data, this effectively covers special abnormal scenarios in the system sampling process. This allows the global sampling unit to quickly and accurately identify invalid sampling results, avoiding redundant processing such as subsequent data reception and parsing of invalid samples, thus improving the data processing efficiency of the global sampling unit. Simultaneously, it standardizes the operation of local sampling units under abnormal scenarios, ensuring that the system maintains closed-loop sampling and data transmission logic even in special cases such as detection latency and dynamic thread termination. This avoids problems such as data transmission interruption and result judgment confusion, improving the overall robustness and operational stability of the system. Furthermore, the formulation of these rules further improves the supporting logic for sampling target selection and data upload, further reducing the transmission of invalid data from the perspective of abnormal scenario handling, reducing the unnecessary occupation of memory access bandwidth, and making the system's bandwidth optimization design more comprehensive.

[0052] In some embodiments, the first sampling period is the time interval between two thread state sampling commands issued by the global sampling unit to the same target stream multiprocessor. The first sampling period can be configured as a dynamic period or a fixed period. When a dynamic period is used, the first sampling period changes dynamically according to the number of stream multiprocessors with active threads in the target stream multiprocessor group.

[0053] Specifically, the first sampling period of the global sampling unit is the command issuance time interval for multiple processors of the same target stream. To adapt to different GPU operating loads and sampling requirements, it can be designed as a dynamic or fixed configurable mode, and the two modes can be flexibly switched according to the actual application scenario.

[0054] The fixed period can be a preset constant time value. Regardless of the number of target stream multiprocessors with active threads in the current stream multiprocessor group, the time interval between two sampling commands issued by the global sampling unit to the same target stream multiprocessor remains constant. The dynamic period has no fixed preset value. Its time interval is negatively correlated with the total number of target stream multiprocessors with active threads in the current stream multiprocessor group. The global sampling unit first determines the number of all target stream multiprocessors in this round, and then evenly distributes the preset total sampling time of a single round to each target stream multiprocessor. This results in the dynamic period for issuing commands to a single target stream multiprocessor. The more target stream multiprocessors there are, the shorter the first sampling period for a single target stream multiprocessor, and vice versa.

[0055] For example: the global sampling unit is set to a preset total sampling duration of 12ms, and the streaming multiprocessor group includes four streaming multiprocessors: SM1, SM2, SM3, and SM4. When it is detected that there are active threads in all 4 stream multiprocessors and they are all target stream multiprocessors, the dynamic first sampling period is 12ms ÷ 4 = 3ms. That is, the global sampling unit sends commands to SM1, SM2, SM3 and SM4 in sequence. The time interval between sending commands to the same stream multiprocessor (such as SM1) is 3ms. When it is detected that only SM1, SM2, and SM3 have active threads, totaling 3 target stream multiprocessors, the dynamic first sampling period = 12ms ÷ 3 = 4ms, and the command issuance interval of the global sampling unit to a single target stream multiprocessor is adjusted to 4ms; When using the fixed period mode, with a preset fixed period of 12ms, the time interval between two commands issued by the global sampling unit to the same target stream multiprocessor remains unchanged at 12ms, regardless of whether the current number of target stream multiprocessors is 4, 3, or 2.

[0056] By implementing a dual-mode configurable design for the first sampling period of the global sampling unit and clearly defining the adjustment rules for the dynamic period, the issuance rhythm of sampling commands can be adapted to the actual operating load and sampling requirements of the GPU. The fixed-period mode ensures the regularity and stability of sampling command issuance, facilitating time-dimensional traceability and analysis of sampled data, and is suitable for GPU performance debugging scenarios with strict requirements for sampling timing. The dynamic period mode enables dynamic matching of the sampling frequency with the number of target stream multiprocessors. When the GPU is under high load and the number of target stream multiprocessors is large, the sampling interval of a single target stream multiprocessor is automatically shortened, increasing the sampling frequency and ensuring real-time thread state monitoring. When the GPU is under low load and the number of target stream multiprocessors is small, the sampling interval is automatically extended, reducing unnecessary command issuance and sampling operations, further reducing system resource consumption. Simultaneously, the dynamic period adjustment method, based on the average distribution of the total sampling time in a single round, simplifies the scheduling logic of the global sampling unit, achieving dynamic period adaptation without complex calculations, thus improving system execution efficiency and further optimizing the flexibility and economy of GPU thread state sampling from the sampling scheduling level.

[0057] In some embodiments, the second sampling period is the local single-round sampling time and the theoretical sampling interval of a single thread; wherein, the local single-round sampling time is the total time taken for the local sampling unit to complete a full round of state sampling for all active threads in the corresponding target stream multiprocessor, which can be configured as a dynamic period or a fixed period, and the theoretical sampling interval of a single thread is the product of the first sampling period and the local single-round sampling time of the corresponding target stream multiprocessor, which is used to characterize the time interval between two consecutive samplings of a single thread; When a dynamic period is used, the local single-round sampling time is the product of the sampling time of a single thread and the number of active threads in the corresponding target stream multiprocessor, and is dynamically adjusted according to the number of active threads.

[0058] Specifically, the second sampling period of the local sampling unit is "local single-round sampling time" and "single-thread theoretical sampling interval (single warp theoretical sampling interval)", which takes into account the local sampling execution rhythm and the global sampling timing relationship, adapts to the dynamic changes in the number of active threads in the streaming multiprocessor, and also sets two configurable modes, dynamic and fixed, to fit the actual operation logic of GPU thread sampling.

[0059] The dynamic period is designed for local single-round sampling time and is real-time adaptive. The local sampling unit first obtains the current number of active threads in the streaming multiprocessor and combines it with the fixed sampling time of a single thread preset at the GPU hardware level. The product of the two is used to obtain the local single-round sampling time. When the number of active threads changes, this time is adjusted synchronously to ensure that the single-round sampling duration exactly covers the sampling of all active threads one by one, without redundant waiting or sampling omissions. The fixed period is also designed for local single-round sampling time and is preset constant. The time value is set in advance according to the maximum number of active threads supported by the streaming multiprocessor. Regardless of the actual number of active threads in the streaming multiprocessor, the local sampling unit completes a round of sampling according to this fixed duration. If the number of active threads is less than the maximum number, there is no sampling action for the remaining period after sampling is completed; if the number of active threads exceeds the maximum number, the threads that have not been sampled are postponed to the next sampling cycle.

[0060] The theoretical sampling interval for a single warp is a combination of the global scheduling timing and the local execution timing. It directly reflects the actual sampling frequency of a single warp. This value changes synchronously with the changes in the first sampling period and the local single-round sampling time. It is the core indicator that reflects the overall sampling rhythm of the system and is also completely consistent with the sampling interval calculation in the subsequent specific embodiments.

[0061] For example: If the fixed sampling time for a single thread at the GPU hardware level is set to 1ms, and the maximum number of active threads supported by a certain stream multiprocessor is 8, the corresponding preset fixed second sampling period is 8ms, and the fixed first sampling period for this SM by the global sampling unit is 12ms. When using the dynamic cycle mode, if there are currently 8 active threads in the multiprocessor stream, the local single-round sampling time is 1ms × 8 = 8ms, corresponding to a theoretical single-warp sampling interval of 12ms × 8ms = 96ms. The local sampling unit needs 8ms to complete the polling and sampling of each of the 8 threads. If 2 threads stop running during the process, the number of active threads becomes 6, and the local single-round sampling time is adjusted to 1ms × 6 = 6ms, corresponding to a theoretical single-warp sampling interval of 12ms × 6ms = 72ms. The local sampling unit can complete a complete round of sampling in 6ms. If there is only 1 active thread, the local single-round sampling time is 1ms, corresponding to a theoretical single-warp sampling interval of 12ms × 1ms = 12ms.

[0062] When using the fixed period mode, the preset fixed local single-round sampling time is 8ms, the global first sampling period is still 12ms, and the theoretical sampling interval of a single warp is constant at 96ms. If there are currently 6 active threads in the streaming multiprocessor, the local sampling unit completes the sampling of 6 threads in 6ms, and there is no sampling action for the remaining 2ms until the fixed period of 8ms ends. If the number of active threads in the streaming multiprocessor increases to 9 due to GPU task scheduling, the local sampling unit completes the sampling of 8 threads in 8ms, and the remaining 1 thread is deferred to the next fixed period of 8ms for sampling.

[0063] By setting the second sampling period to the local single-round sampling time and supporting both dynamic and fixed period configurations, the local sampling rhythm can be adapted to the actual number of active threads within the streaming multiprocessor. With a dynamic period, the second sampling period adjusts in real-time according to the number of active threads, accurately matching the current thread's sampling needs. This ensures complete acquisition of the states of all active threads while reducing invalid waiting time and improving the execution efficiency of the local sampling unit. With a fixed period, the local sampling timing remains stable, facilitating unified scheduling with the first sampling period of the global sampling unit and improving the overall system timing synchronization and operational reliability. Furthermore, by defining the theoretical sampling interval of a single warp as the product of the first sampling period and the local single-round sampling time, the actual sampling frequency of a single thread can be directly reflected, creating a complete timing loop between global scheduling and local execution. This ensures both the fine-grained and real-time sampling of GPU thread states and helps control the amount of data collected and transmission overhead, improving overall system stability and resource utilization.

[0064] In some embodiments, after each round of sampling is completed, the global sampling unit updates the active thread detection results of each stream multiprocessor and adjusts the target stream multiprocessor range for the next round of sampling based on the updated detection results.

[0065] Specifically, the global sampling unit operates on a round-by-round basis. The criteria for completing each round are: issuing sampling commands to all target stream multiprocessors in the current round and receiving all thread status information (or bubble data) uploaded by the target stream multiprocessors. After completing the current round, the global sampling unit re-executes the active thread detection operation on all stream multiprocessors within the stream multiprocessor group, covering both target and non-target stream multiprocessors, to obtain the latest status of active threads within each stream multiprocessor, thereby updating the global active thread detection result database. After updating the results, the global sampling unit will re-select and determine the target stream multiprocessor range for the next round of sampling based on the latest detection results, achieving dynamic updates of the sampling target rounds and ensuring that the sampling target in each round matches the actual active status of the stream multiprocessors.

[0066] For example, a streaming multiprocessor group includes SM1, SM2, SM3, and SM4. Before the first round of sampling, SM1, SM2, and SM3 are detected as target streaming multiprocessors, while SM4 has no active threads and is considered a non-target streaming multiprocessor. After the sampling commands for SM1, SM2, and SM3 are issued and data is received, the first round of sampling ends. At this time, the global sampling unit re-detects SM1-SM4 and finds that SM3 has no active threads, while SM4 has restarted active threads. The detection results are then updated: SM1 and SM2 have active threads, SM3 has no active threads, and SM4 has active threads. Based on this updated result, the target streaming multiprocessor range for the next round of sampling is adjusted to SM1, SM2, and SM4, discarding the original target streaming multiprocessor SM3 and including the original non-target streaming multiprocessor SM4.

[0067] The above method enables dynamic updating of sampling targets in rounds, ensuring real-time synchronization between sampling targets and the actual state of active threads in the streaming multiprocessor. This avoids invalid sampling and missed thread state sampling, forming a closed-loop scheduling logic of "sampling-detection-update-adjustment," which can improve the intelligence and adaptability of the system's sampling strategy.

[0068] In some embodiments, adjusting the target stream multiprocessor range for the next round of sampling based on the updated detection results includes: If no active threads are detected within the original target stream multiprocessor, they are excluded from the next round of target stream multiprocessor range, and the issuance of sampling commands is stopped. If an active thread is detected to reappear within a non-target stream multiprocessor, it will be included in the next round of target stream multiprocessor scope, and a sampling command will be issued according to the first sampling period.

[0069] Specifically, this rule outlines the operational standards for adjusting the target stream multiprocessor range for the next round of global sampling. It establishes differentiated adjustment strategies only for the original target stream multiprocessors and non-target stream multiprocessors, and all adjustments are executed based on the latest active thread detection results updated after each round of sampling. For the original target stream multiprocessors from the previous round, if the updated detection results show no active threads, they are directly removed from the target stream multiprocessor range for the next round, and no further sampling commands are issued to them. For the non-target stream multiprocessors from the previous round, if the updated detection results show that active threads have reappeared, they are immediately included in the target stream multiprocessor range for the next round, and sampling commands are issued to them normally according to the preset first sampling period, allowing for unified scheduling with other target stream multiprocessors.

[0070] The above methods can prevent newly added active threads from being missed in streaming multiprocessors, further improving sampling efficiency and reducing the invalid use of memory access bandwidth.

[0071] In some embodiments, the local sampling unit performs sampling and uploading in the following manner: After receiving the thread state sampling command issued by the global sampling unit, the state of each thread in the corresponding stream multiprocessor is sampled one by one according to the second sampling period, and the thread state information obtained from each sampling is temporarily stored in the local buffer. After sampling all target threads within the current sampling period, the multiple thread state information temporarily stored in the local buffer is packaged into a sampling data block, and the sampling data block is serially uploaded to the global sampling unit.

[0072] Specifically, the sampling and uploading actions of the local sampling unit follow the core process of "sampling sequentially - local temporary storage - unified packaging - serial uploading." All operations are triggered upon receiving the sampling command from the global sampling unit and strictly match the timing requirements of the second sampling period. Sampling each thread sequentially is a serial sampling method, collecting state information from only a single thread at a time, thus avoiding information corruption caused by simultaneous sampling from multiple threads. The local buffer is a temporary storage module configured within the streaming multiprocessor, used to temporarily store the thread state information of a single sample in real time, decoupling the sampling and uploading actions, allowing the next thread to start sampling without waiting for the upload to complete. The target thread can be all threads that the local sampling unit plans to complete sampling in the current second sampling period, dynamically changing with the configuration of the second sampling period (dynamic / fixed) and the active thread states within the streaming multiprocessor. Packaging into sampling data blocks integrates the multi-threaded state information in the buffer into a continuous data packet according to the sampling order, ensuring data timing consistency. Serial upload strictly follows the serial connection order of the streaming multiprocessor group, transmitting data blocks to the global sampling unit within the corresponding time period to avoid bus conflicts caused by simultaneous uploads from multiple streams and multiple processors.

[0073] For example: suppose a local sampling unit of a streaming multiprocessor SM1 receives a global sampling command, the second sampling period is 8ms (corresponding to 8 target threads), the sampling time for a single thread is 1ms, and SM1 is the first serial node in the streaming multiprocessor group: 0ms: Start sampling of warp1, complete sampling in 1ms, temporarily store warp1 state information in local buffer, and start sampling of warp2 at the same time; 1ms-7ms: Sampling of warp2-warp8 is completed sequentially according to the above logic. After each warp is completed, the state information is temporarily stored in the buffer. Sampling of all 8 target threads in the current sampling period is completed at 8ms. 8ms: The local sampling unit packages the state information temporarily stored in the buffer in the order of warp1-warp8 into an SM1 sampling data block; After 8ms: Following the serial upload order of SM1→SM2→SM3→SM4, the sampled data block is uploaded to the global sampling unit. After the upload is completed, the buffer is cleared and the unit waits for the next round of sampling commands.

[0074] If it is a dynamic second sampling period, when the number of active threads in SM1 becomes 6, after sampling of all target threads is completed in 6ms, the data is packaged and uploaded immediately. The process is the same as above, except that the data block contains the status information of the 6 threads.

[0075] The above method allows local sampling to continue without waiting for the upload to complete, fully utilizing the timing resources of the second sampling cycle. Simultaneously, packaging the sampled information into data blocks for serial upload reduces the number of transmissions, lowers GPU memory bus occupancy, and effectively controls bandwidth consumption. Furthermore, the combination of local buffer storage and sequential packaging ensures the timing consistency of thread state information, providing accurate data support for subsequent time-dimensional analysis of thread execution states. In addition, serial uploading matches the serial connection rules of the streaming multiprocessor group, avoiding bus conflicts and improving the overall data transmission stability of the system.

[0076] In some embodiments, the thread state information collected by the local sampling unit includes at least one of the thread emission blocking reasons and thread high-frequency change signals; the global sampling unit summarizes the thread state information uploaded by all target stream multiprocessors and outputs it to the GPU external storage unit.

[0077] Specifically, the local sampling unit samples core state information during thread execution, prioritizing information that has practical guiding significance for GPU performance analysis and troubleshooting. Among these, thread launch blocking causes refer to various factors that cause threads to be blocked during the launch execution phase, such as memory access conflicts, resource contention, and instruction dependencies; these are core evidence for analyzing GPU thread execution efficiency. High-frequency thread change signals are key semaphores that change dynamically in real time during thread execution, such as thread execution status signals, register read / write signals, and on-chip memory access signals; these can directly reflect the real-time running status of the thread. The local sampling unit can collect one type of information alone or both simultaneously, depending on actual monitoring needs. All collected information is encoded according to a standardized format and then temporarily stored, packaged, and uploaded along with the sampling process.

[0078] After receiving the sampled data blocks (or bubbly data) uploaded by all target stream multiprocessors, the global sampling unit performs a data aggregation operation: first, it parses each data block to filter out valid thread state information; then, it integrates all valid information into a global GPU thread state sampling dataset according to the stream multiprocessor number and thread sampling order; finally, it outputs this dataset to the GPU's external storage unit (such as video memory or host memory) through the GPU's on-chip communication interface. The sampled data in the GPU's external storage unit can be read by GPU drivers and performance analysis tools for subsequent GPU performance optimization, fault location, task scheduling adjustment, etc., such as providing users with GPU performance analysis and optimization. Simultaneously, after each round of sampling, it can update the active thread detection results of each stream multiprocessor, providing a basis for adjusting the target stream multiprocessor group in the next round of sampling.

[0079] For example: Sampling information acquisition: When the local sampling unit of SM1 samples warp1, it acquires that the reason for the thread's transmit blocking is "memory access conflict", and at the same time, it acquires the thread's high-frequency changing signals as "register write signal set to 1, on-chip memory read signal set to 0"; when sampling warp2, it acquires that the blocking reason is "instruction dependency", and the high-frequency changing signals are "execution status signal is 'waiting', memory access signal set to 1". After encoding the two types of information, they are temporarily stored in the buffer. Data aggregation and output: After receiving the sampling data blocks uploaded by SM1, SM2, and SM3, the global sampling unit parses out the thread blocking reasons and high-frequency change signals of SM1 (warp1-warp8), SM2 (warp1-warp6), and SM3 (warp1-warp5), and integrates them into a global sampling dataset in the order of "SM1-warp1, SM1-warp2...SM2-warp1...SM3-warp5", and then outputs the dataset to the GPU memory; Data utilization: GPU performance analysis tools read the dataset from video memory and analyze that the main reason for the current blocking of GPU threads is memory access conflict. Based on this, the GPU's memory access scheduling strategy is optimized to improve the overall parallel computing efficiency.

[0080] Figure 2This is a flowchart illustrating a GPU thread state information sampling and monitoring method provided in the embodiments of this specification. Although this specification provides method operation steps or device structures as shown in the following embodiments or figures, more or fewer operation steps or module units may be included in the method or device based on conventional or non-inventive effort. In steps or structures where there is no logically necessary causal relationship, the execution order of these steps or the module structure of the device is not limited to the execution order or module structure described in the embodiments and figures of this specification. When the method or module structure is applied in actual devices or terminal products, it can be executed sequentially or in parallel according to the method or module structure shown in the embodiments or figures (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed processing environment). Figure 2 As shown, the method may include: S201: Multiple serially connected streaming multiprocessors in the GPU are grouped into a streaming multiprocessor group. A global sampling unit is deployed in the streaming multiprocessor group and communicates with each streaming multiprocessor. A local sampling unit is deployed in each streaming multiprocessor in the streaming multiprocessor group and communicates with each thread in the corresponding streaming multiprocessor and the global sampling unit. S202: Before performing each round of sampling operations, the global sampling unit detects whether there are active threads in each stream multiprocessor, determines the target stream multiprocessor for this round of sampling based on the detection results, and issues a thread status sampling command to the target stream multiprocessor according to the first sampling period. S203: After receiving the thread state sampling command issued by the global sampling unit, the local sampling unit performs state sampling on each thread in the corresponding target stream multiprocessor according to the second sampling period, and uploads the collected thread state information to the global sampling unit. S204: The global sampling unit summarizes the thread status information uploaded by all target stream multiprocessors and outputs it to the GPU external storage unit.

[0081] By adding local sampling units and global sampling units in the GPU, warps within the target SM and target SMs within the SM group are periodically sampled respectively. Only key warp running status information is obtained, which achieves finer monitoring granularity than performance counters and avoids the massive data and high bandwidth consumption of thread tracing.

[0082] For specific implementation details of S201~S204 above, please refer to the description in the system section. This manual will not elaborate on them.

[0083] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. For details, please refer to the foregoing descriptions of the relevant processing embodiments; they will not be repeated here.

[0084] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0085] In a specific implementation scenario, see Figure 3 As shown, the most common way for the global sampling unit to periodically sample different SMs (or sm) is round robin (polling or cyclic sampling): In the current round, the global sampling unit can send sampling commands to each SM in the group in the order of sm0→sm1→sm2→sm3→…→smn. After completing the sampling of all SMs, the next round of sampling starts again from SM0. That is, the corresponding sampling command sampling omd can be sent to each SM (to SMs) in the order of sm0→sm1→sm2→sm3→…→smn.

[0086] See Figure 4 As shown, when a local sampling unit of a SM receives a sampling command from the global sampling unit, it will periodically sample the state of the active warps within its SM. The most common periodic sampling method is the round-roubin method: the local sampling unit can sequentially sample the state of the active warps within its SM in the order warp0→warp1→warp2→warp3→…→warpN-3→warpN-2→warpN-1→warpN. After sampling all active warps, it starts the next round from warp0. Typically, the time interval for sampling the state information of a warp is the sampling time interval issued by the global sampling unit to its SM multiplied by the time interval for each warp to complete one round within the SM.

[0087] See Figure 5As shown, in the current round, the global sampling unit can issue the sampling command `sampling omd` to each SM in the group in the order of `sm0→sm1→sm2→sm3→…→smn`. After sampling all SMs, the next round of sampling starts again from SM0. If there is no active warp in SM1 in the previous round, the global sampling unit will skip SM1 in the next round and will not issue a sampling command to it. It will only sample the other SMs with active warps, and can issue the sampling command `sampling omd` to each SM in the group in the order of `sm0→sm2→sm3→…→smn`.

[0088] See Figure 6 As shown, the global sampling unit sends sampling commands (cmd) to each SM, and the sending order can be SM0→SM1→SM2→SM3. In SM1, there is only one active warp remaining, and this warp is about to end its operation. The global detection unit does not detect this state and still sends sampling commands to SM1 according to the polling cycle of SM0→SM1→SM2→SM3. When the sampling command is transmitted to SM1 via the serial link, all warps in SM1 have finished running and there are no active warps available for sampling. The local sampling unit within SM1 has no valid warp status information to collect and serially uploads empty packet data (no warp packet) to the global sampling unit.

[0089] Finally, the uploaded data packets of each stream multiprocessor are serially transmitted back to the global sampling unit in the order of SM0 warp packet → no warp packet → SM2 warp packet → SM3 warp packet. The transmission period corresponding to the empty packet data is the bubble generated in the sampling link.

[0090] The following example uses a specific SM group configuration to demonstrate the entire process from deployment to sampling and data upload, while also showing how to handle abnormal situations.

[0091] The GPU is divided into 1 SM group, and 1 SM group contains 4 SMs: SM1, SM2, SM3, and SM4. Each SM initially has 8 active warps (warp1-warp8); The global sampling unit polls the SM in a round of sampling of 4 SMs every 12ms (i.e., the first sampling period for a single SM is 12ms). Polling method of warp by local sampling unit: 1 warp is sampled every 1ms (that is, the local single round sampling time in a single SM = 8 × 1 = 8ms, adopting a dynamic periodic mode, which is adjusted according to the number of active warps). Sampling unit deployment: The SM group is equipped with one global sampling unit, and each of the four SMs is equipped with one local sampling unit, all of which use serial transmission.

[0092] Step 1: Architecture Deployment and Initialization

[0093] according to Figure 1 The architecture connects SM1-SM4 serially to form an SM group. A global sampling unit is deployed within the SM group, and a corresponding local sampling unit is deployed within each SM. All sampling units are initialized.

[0094] The local sampling unit detects the active status (status sampling) of the warp within its corresponding SM in real time and waits for sampling commands. The global sampling unit is responsible for detecting whether there are active threads in each SM before the start of each round of sampling, determining the target stream multiprocessor for this round of sampling, and then sending sampling commands to the target SM and collecting the sampling data sent by the local sampling unit.

[0095] Step 2: Issuance of global sampling command and initiation of local sampling ( Figure 3 + Figure 4 )

[0096] At 0ms, the global sampling unit prepares to start the first round of polling. It first checks the active thread status of SM1-SM4. If active warps exist in SM1, SM2, SM3, and SM4, all four SMs are identified as the target stream multiprocessors for the first round. Sampling commands are then sent sequentially to SM1→SM2→SM3→SM4. These commands are sent through a dedicated control path, without occupying the data path for normal instruction execution. Commands are sent to one SM every 3ms (the polling of all four SMs is completed in 12ms, matching the dynamic allocation rules of the first sampling period). At 0m, the local sampling unit of SM receives the sampling command and immediately starts internal roundrobin sampling, sequentially accessing the currently active warp1-warp8, sampling one warp every 1ms; At 3ms, the local sampling unit of SM2 receives the sampling command and starts polling sampling; at 6ms and 9ms, the local sampling units of SM3 and SM4 start sequentially.

[0097] Step 3: Warp sampling under ideal conditions

[0098] Taking warp3 within SM1 as an example, calculate its theoretical sampling interval and demonstrate the sampling process: Theoretical sampling interval (single warp) = first sampling period 12ms × local single round sampling time 8ms = 96ms; 0ms: SM1 receives the global sampling command, starts the local sampling unit, and begins sampling of warp1; 1ms: Warp1 sampling is complete and stored in the local buffer, while sampling of warp2 is started at the same time; 2ms: Warp2 sampling is complete and stored in the local buffer, while sampling of warp3 is started simultaneously; 3ms: Warp3 sampling is complete and stored in the local buffer, while sampling of warp4 begins. ... 8ms: warp8 sampling is completed and stored in the local buffer, completing one round of sampling and uploading of 8 warps. Then, the state data of each warp is packaged into an SM1 warp packet and serially uploaded to the global sampling unit (the SM1 warp packet contains the state data of warp1...warp8), waiting for the next round of global sampling command.

[0099] At 12ms, the global sampling unit detects that SM1 still has active threads, includes it in the second round of target SMs, issues the second round of sampling commands, and the local sampling unit of SM1 restarts polling: 12ms: SM1 receives the global sampling command, starts the local sampling unit, and begins sampling of warp1; 13ms: Sampling of warp1 is complete and stored in the local buffer. At the same time, sampling of warp2 is started. 14ms: Warp2 sampling is complete and stored in the local buffer, while sampling of warp3 begins; 15ms: Sampling of warp3 is complete and stored in the local buffer, while sampling of warp4 begins. ... 20ms: warp8 sampling is complete, stored in the local buffer, completing one round of sampling and uploading of 8 warps, waiting for the next round of global sampling command.

[0100] In this way, ideally, the state of warp3 will be successfully sampled and uploaded once every 96ms. The global sampling unit will summarize the warp states of all SMs and output them to external memory.

[0101] Step 4: Dynamically adjust sampling (reduce the number of warps within the SM, Figure 5 )

[0102] At 60ms, warp7 and warp8 in SM4 finish running, and the number of active warps in SM4 becomes 6. The local sampling unit uploads this status to the global sampling unit, and the local single-round sampling time of SM4 is dynamically adjusted to 6×1=6ms. The corresponding theoretical sampling interval for a single warp becomes 12ms×6ms=72ms. At 72ms, after the global sampling unit completes the third round of sampling, it detects that the last active warp (warp6) of SM4 is about to end its operation, and determines that SM4 does not need to be included in the next round of target SM; At 84ms, the global sampling unit starts the fourth round of sampling, only detecting active threads in SM1, SM2, and SM3, and identifying these three as the target SMs for the fourth round. The sampling command for SM4 is canceled. The total sampling time for a single round is still calculated as 12ms, and the first sampling period of a single SM is adjusted to 12ms / 3=4ms.

[0103] Step 5: Sampling cavitation generation ( Figure 6 )

[0104] At 90ms, only warp1 remained active in SM2 and was about to terminate its operation; the global sampling unit failed to detect this state in time. At 96ms, the global sampling unit starts the fifth round of sampling, still including SM2 in the target SM, and sends sampling commands to SM2 according to the polling order; At 99ms, the sampling command is transmitted to SM2. At this time, warp1 in SM2 has completely finished running and there is no active warp available for sampling. The local sampling unit of SM2 has no valid data uploaded, and generates and uploads an empty data packet (no warp packet). The presence of the bubble does not affect the data transmission of subsequent SMs, but only leaves a gap in the final data stream. During the serial transmission to the global sampling unit, a bubble appears in the corresponding time period of SM2, and the global sampling unit only receives idle signals during this time period.

[0105] Step 6: Data Output and Monitoring

[0106] The global sampling unit aggregates all valid warp status information (excluding invalid data during bubbling periods) and periodically outputs it to the GPU's external memory. Users can read the data from the external memory to observe the specific running status of each warp, achieving fine-grained monitoring. Moreover, the amount of data is much smaller than that of thread tracing, and it does not consume a large amount of memory access bandwidth.

[0107] By dividing the work between global and local sampling units, periodic sampling of the SM (Streaming Service) and warps is achieved, bridging the monitoring granularity gap between performance counters (global) and thread tracing (instruction-level). The default round-robin method ensures the orderliness of sampling, and the sampling frequency is adjustable, avoiding the generation of massive amounts of data. Dynamically canceling sampling commands based on the status of active warps within the SM reduces invalid sampling and improves bandwidth utilization. A fixed / dynamic dual sampling period scheme is provided to adapt to different user needs and expand the applicable scenarios of the solution.

[0108] Based on the same inventive concept, this specification also provides a GPU thread state information sampling and monitoring device in its embodiments, as described in the following embodiments. Since the principle by which the GPU thread state information sampling and monitoring device solves the problem is similar to that of the GPU thread state information sampling and monitoring method, the implementation of the GPU thread state information sampling and monitoring device can refer to the embodiments of the GPU thread state information sampling and monitoring method; repeated details will not be elaborated further. References Figure 7 As shown, the device may include: Deployment module 701 can be used to form a stream multiprocessor group from multiple serially connected stream multiprocessors in the GPU, deploy a global sampling unit in the stream multiprocessor group and communicate with each stream multiprocessor, and deploy a local sampling unit in each stream multiprocessor in the stream multiprocessor group and communicate with each thread in the corresponding stream multiprocessor and the global sampling unit. The global sampling unit execution module 702 can be used to detect whether there are active threads in each stream multiprocessor before the global sampling unit performs each round of sampling operation, and determine the target stream multiprocessor for this round of sampling based on the detection results, and issue a thread status sampling command to the target stream multiprocessor according to the first sampling period; The local sampling unit execution module 703 can be used to sample the status of each thread in the corresponding target stream multiprocessor according to the second sampling period after the local sampling unit receives the thread status sampling command issued by the global sampling unit, and upload the collected thread status information to the global sampling unit. The global sampling unit upload module 704 can be used to aggregate the thread status information uploaded by all target stream multiprocessors by the global sampling unit and output it to the GPU external storage unit.

[0109] This specification also provides a GPU, including the system described in any of the above embodiments, through which the sampling and monitoring of the running state of internal threads of the GPU is implemented.

[0110] This specification also provides an embodiment of a chip, such as... Figure 8As shown, the chip may include a processor 801 and a memory 802, wherein the processor 801 and the memory 802 can be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0111] Processor 801 can be a parallel computing processor or a graphics processing unit (CPU), specifically integrating the global sampling unit and local sampling unit described in the foregoing embodiments, and including a stream multiprocessor group composed of several stream multiprocessors connected serially. That is, processor 801 integrates the aforementioned GPU thread state information sampling and monitoring system. Processor 801 drives the global sampling unit and local sampling unit in the system to work together by running non-transitory software programs, instructions, and modules stored in memory 802, relying on the serial architecture of the stream multiprocessor group to achieve fine-grained, low-overhead sampling and monitoring of the internal thread state of the GPU.

[0112] The memory 802, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the GPU thread state information sampling and monitoring method in this embodiment of the invention.

[0113] The memory 802 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function (including programs that drive the global sampling unit and the local sampling unit). The data storage area may store data created by the processor 801, including sampled thread state information, cavitation data, active thread detection results, sampling period configuration parameters, etc. Furthermore, the memory 802 may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 802 may optionally include memory remotely located relative to the processor 801, and these remote memories can be connected to the processor 801 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0114] The one or more modules are stored in the memory 802, and when executed by the processor 801, they perform the following: Figure 2 The GPU thread state information sampling and monitoring method shown can simultaneously implement all the functions of any of the above systems, and complete the entire sampling and monitoring process from sampling target determination, command issuance, local sampling, data upload to anomaly handling.

[0115] By integrating the GPU thread state information sampling and monitoring system of the present invention into the GPU chip through the above-mentioned chip, a complete hardware solution is formed, realizing the integrated design of the sampling and monitoring system and the GPU chip, simplifying the GPU hardware deployment process, while ensuring the real-time performance and reliability of sampling and monitoring, and reducing the overall hardware complexity and memory access bandwidth consumption of the GPU.

[0116] For details regarding the chip, please refer to the relevant descriptions and effects in the above method embodiments; they will not be repeated here.

[0117] This specification also provides a computer storage medium storing computer program instructions, which, when executed, implement the steps of the GPU thread state information sampling and monitoring method described above.

[0118] This specification also provides a computer program product, which includes a computer program that, when executed, implements the steps of the GPU thread state information sampling and monitoring method described above.

[0119] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0120] The various embodiments in this specification are described in a progressive manner. For the same or similar parts between the various embodiments, please refer to each other. The focus of each embodiment is to describe the differences from other embodiments.

[0121] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions.

[0122] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0123] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute certain parts of the methods of various embodiments of this application.

[0124] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer chips, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0125] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0126] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to the embodiments described herein by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

1. A GPU thread state information sampling and monitoring system, characterized in that, include: A streaming multiprocessor group consists of multiple serially connected streaming multiprocessors within a GPU. A global sampling unit is deployed in the streaming multiprocessor group and communicates with each streaming multiprocessor. Before each round of sampling operation, it detects whether there are active threads in each streaming multiprocessor, determines the target streaming multiprocessor for this round of sampling based on the detection results, and issues a thread status sampling command to the target streaming multiprocessor according to the first sampling period. Multiple local sampling units are deployed in a stream multiprocessor and are communicatively connected to each thread in the corresponding stream multiprocessor and the global sampling unit. After receiving a thread state sampling command issued by the global sampling unit, the local sampling unit samples the state of each thread in the corresponding target stream multiprocessor according to the second sampling period and uploads the collected thread state information to the global sampling unit.

2. The system according to claim 1, characterized in that, The serial connection order of each stream multiprocessor in the stream multiprocessor group is consistent with the order in which the global sampling unit issues thread state sampling commands to the target stream multiprocessor and the order in which the global sampling unit receives thread state information.

3. The system according to claim 1, characterized in that, The rules by which the global sampling unit determines the target stream multiprocessor for this round of sampling include: If an active thread is detected in one or more of the streaming multiprocessors in the streaming multiprocessor group, the first streaming multiprocessor in which an active thread is detected is identified as the target streaming multiprocessor. If no active thread is detected in one or more stream multiprocessors in the stream multiprocessor group, the second stream multiprocessor in which no active thread is detected will be excluded from the target stream multiprocessor range of this round of sampling, and no thread state sampling command will be issued to the second stream multiprocessor, and the last active thread in the second stream multiprocessor has ended running.

4. The system according to claim 1, characterized in that, The system also includes: If it is not detected in time that there is only one active thread about to terminate in one or more stream multiprocessors in the stream multiprocessor group, and the third stream multiprocessor with only one active thread about to terminate is still identified as the target stream multiprocessor, then the local sampling unit deployed in the third stream multiprocessor will upload bubbling data to the global sampling unit after receiving the thread state sampling command issued by the global sampling unit. The bubbling data is used to indicate that there is no valid thread state information in this round of sampling.

5. The system according to claim 1, characterized in that, The first sampling period is the time interval between two thread state sampling commands issued by the global sampling unit to the same target stream multiprocessor. The first sampling period can be configured as a dynamic period or a fixed period. When a dynamic period is used, the first sampling period changes dynamically according to the number of stream multiprocessors with active threads in the target stream multiprocessor group.

6. The system according to claim 1, characterized in that, The second sampling period is the local single-round sampling time and the theoretical sampling interval of a single thread; wherein, the local single-round sampling time is the total time taken for the local sampling unit to complete a full round of state sampling for all active threads in the corresponding target stream multiprocessor, which can be configured as a dynamic period or a fixed period, and the theoretical sampling interval of a single thread is the product of the first sampling period and the local single-round sampling time of the corresponding target stream multiprocessor, which is used to characterize the time interval between two consecutive samplings of a single thread; When a dynamic period is used, the local single-round sampling time is the product of the sampling time of a single thread and the number of active threads in the corresponding target stream multiprocessor, and is dynamically adjusted according to the number of active threads.

7. The system according to claim 1, characterized in that, After each round of sampling is completed, the global sampling unit updates the active thread detection results of each stream multiprocessor and adjusts the target stream multiprocessor range for the next round of sampling based on the updated detection results.

8. The system according to claim 7, characterized in that, The adjustment of the target stream multiprocessor range for the next round of sampling based on the updated detection results includes: If no active threads are detected within the original target stream multiprocessor, they are excluded from the next round of target stream multiprocessor range, and the issuance of sampling commands is stopped. If an active thread is detected to reappear within a non-target stream multiprocessor, it will be included in the next round of target stream multiprocessor scope, and a sampling command will be issued according to the first sampling period.

9. The system according to claim 1, characterized in that, The local sampling unit performs sampling and uploading in the following manner: After receiving the thread state sampling command issued by the global sampling unit, the state of each thread in the corresponding stream multiprocessor is sampled one by one according to the second sampling period, and the thread state information obtained from each sampling is temporarily stored in the local buffer. After sampling all target threads within the current sampling period, the multiple thread state information temporarily stored in the local buffer is packaged into a sampling data block, and the sampling data block is serially uploaded to the global sampling unit.

10. The system according to claim 1, characterized in that, The thread state information collected by the local sampling unit includes at least one of the following: thread emission blocking reasons and thread high-frequency change signals; the global sampling unit summarizes the thread state information uploaded by all target stream multiprocessors and outputs it to the GPU external storage unit.

11. A method for sampling and monitoring GPU thread state information, characterized in that, include: Multiple serially connected streaming multiprocessors in the GPU are grouped into a streaming multiprocessor group. A global sampling unit is deployed in the streaming multiprocessor group and communicates with each streaming multiprocessor. A local sampling unit is deployed in each streaming multiprocessor in the streaming multiprocessor group and communicates with each thread in the corresponding streaming multiprocessor and the global sampling unit. Before performing each round of sampling operations, the global sampling unit detects whether there are active threads in each stream multiprocessor, determines the target stream multiprocessor for this round of sampling based on the detection results, and issues a thread status sampling command to the target stream multiprocessor according to the first sampling period. After receiving the thread state sampling command issued by the global sampling unit, the local sampling unit performs state sampling on each thread in the corresponding target stream multiprocessor according to the second sampling period, and uploads the collected thread state information to the global sampling unit. The global sampling unit aggregates the thread state information uploaded by all target stream multiprocessors and outputs it to the GPU external storage unit.

12. A GPU, characterized in that, The system includes any one of claims 1-10, which enables sampling and monitoring of the running state of threads within a GPU.

13. A chip, characterized in that, It includes a processor and a memory storing processor-executable instructions, wherein the processor, when executing the instructions, implements the steps of the method of claim 11.

14. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, implement the steps of the method of claim 11.

15. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the steps of the method of claim 11.

Images