Thread block configuration method, device, readable storage medium and program product

Abstract

The present application discloses a thread block configuration method, device, readable storage medium and program product, which relate to the field of computer technology. The method comprises collecting operating status indicators of each system and obtaining various preset bottleneck features, calculating feature scores for various bottleneck features according to the operating status indicators of each system, and performing weight calculation according to the calculated feature scores, and then configuring thread blocks in combination with the operating status indicators of each system and the feature weight information. The method fully considers the influence of various bottleneck features on the thread block configuration, adopts a specific thread block configuration strategy according to the feature weight information of various bottleneck features, solves the technical problems of being unable to adapt to dynamic load changes, high maintenance cost, high data acquisition cost, and high complexity of model deployment and migration, and achieves the technical effect of being able to adapt to dynamic load changes, saving maintenance cost and data acquisition cost, and being able to dynamically generate the optimal thread block size.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a thread block configuration method, device, readable storage medium, and program product. Background Art

[0002] In parallel computing architectures, the thread block size configuration (block_size) directly impacts computing resource utilization. Currently, there are two common thread block configuration methods: one is to statically select the optimal size configuration at runtime by querying a pre-generated thread block size recommendation table; the other is to use a trained neural network model to predict the optimal thread block size based on the current input data shape.

[0003] However, both of these thread block configuration methods have their own drawbacks. First, statically selecting the optimal size configuration by querying a pre-generated thread block size recommendation table cannot adapt to dynamic load changes and relies on manual parameter tuning, increasing maintenance costs. Second, predicting the optimal thread block size by calling a trained neural network model relies on large amounts of training data, resulting in high data acquisition costs, high prediction latency, and high hardware dependence, which increases the complexity of model deployment and migration. Summary of the Invention

[0004] The present application provides a thread block configuration method, device, readable storage medium and program product to at least solve the problems in related technologies such as the inability to adapt to dynamic load changes, high maintenance costs, high data acquisition costs, and high complexity of model deployment and migration.

[0005] This application provides a thread block configuration method, including:

[0006] Collect the operating status indicators of each system and obtain the preset bottleneck characteristics;

[0007] Calculate the feature scores of each bottleneck feature according to the operating status indicators of each system to obtain the feature scores;

[0008] Calculate the feature weight of each feature score to obtain the feature weight information corresponding to each bottleneck feature;

[0009] Thread blocks are configured based on the operating status indicators of each system and the weight information of each feature.

[0010] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned thread block configuration methods when executing the computer program.

[0011] The present application also provides a non-volatile computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned thread block configuration methods are implemented.

[0012] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned thread block configuration methods when executed by a processor.

[0013] Through this application, by collecting the operating status indicators of each system and obtaining various preset bottleneck features, the feature scores of various bottleneck features are calculated according to the operating status indicators of each system, and the weights are calculated according to the calculated feature scores. Then, the thread blocks are configured in combination with the operating status indicators of each system and the feature weight information. The influence of various bottleneck features on the thread block configuration is fully considered, and a specific thread block configuration strategy is adopted according to the feature weight information of various bottleneck features. Therefore, it can solve technical problems such as inability to adapt to dynamic load changes, high maintenance costs, high data acquisition costs, and high complexity of model deployment and migration. It achieves the technical effect of being able to adapt to dynamic load changes, saving maintenance costs and data acquisition costs, and being able to dynamically generate the optimal thread block size. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0015] Figure 1 A flowchart of an implementation method of a thread block configuration method provided in an embodiment of the present application;

[0016] Figure 2 A flowchart of another thread block configuration method provided in an embodiment of the present application;

[0017] Figure 3 A structural block diagram of a thread block configuration provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0019] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0020] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0021] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the thread block configuration method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0022] An embodiment of the present application provides a thread block configuration method, and the method is described in detail in conjunction with the execution flow of the thread block configuration method.

[0023] See also Figure 1 , Figure 1 This is a flowchart of an implementation of a thread block configuration method provided in an embodiment of the present application. The method may include the following steps.

[0024] S101: Collect operating status indicators of each system and obtain various preset bottleneck characteristics.

[0025] During system operation, various system operation status indicators are collected, such as the Streaming Multiprocessor (SM) occupancy rate, memory bandwidth utilization, and Level 1 Cache (L1) hit rate.

[0026] You can also get various preset bottleneck features, which may include computationally intensive features, memory intensive features, and control flow intensive features.

[0027] S102: Calculating feature scores for each type of bottleneck feature based on the operating status indicators of each system to obtain feature scores.

[0028] After collecting the operating status indicators of each system and obtaining various preset bottleneck features, feature scores of various bottleneck features are calculated according to the operating status indicators of each system to obtain each feature score.

[0029] A sliding window weighting mechanism can be used to process time series indicator data. The default window size is set to 10 sampling periods to balance real-time performance and stability. For the indicator set Mi within the i-th window period, the feature score can be calculated using the following formula.

[0030] ;

[0031] ;

[0032] ;

[0033] in, Represents calculation The mean of Indicates the stream processor occupancy rate, Indicates memory bandwidth utilization, Indicates the instruction transmission efficiency, is the calculated score of the i-th sliding window, is the memory score of the i-th sliding window, is the control flow score of the i-th sliding window.

[0034] S103: Calculate feature weights for each feature score to obtain feature weight information corresponding to each bottleneck feature.

[0035] After calculating each feature score, feature weights are calculated for each feature score to obtain feature weight information corresponding to each bottleneck feature. For example, each feature score can be normalized first, and then feature weight information is calculated based on the normalized feature scores.

[0036] Following the example in step S102, the weighted ratio normalization is finally performed to form a standardized feature vector:

[0037] ;

[0038] in, To calculate the intensity, is the memory intensity, To control the flow intensity.

[0039] A circular buffer can be maintained to store the system operation status indicators of the latest N sampling periods. During each update, the median of each system operation status indicator in the window is calculated first, and then a three-dimensional feature vector is generated based on a predefined feature discrimination rule.

[0040] S104: Thread block configuration is performed according to the operating status indicators of each system and the weight information of each feature.

[0041] After calculating the feature weights corresponding to each bottleneck feature, thread block configuration is performed based on the system's operating status indicators and the feature weights. This information can be used to determine the current bottleneck scenario. This information can then be combined with resource constraints to dynamically generate the optimal thread block size.

[0042] Through this application, by collecting the operating status indicators of each system and obtaining various preset bottleneck features, the feature scores of various bottleneck features are calculated according to the operating status indicators of each system, and the weights are calculated according to the calculated feature scores. Then, the thread blocks are configured in combination with the operating status indicators of each system and the feature weight information. The influence of various bottleneck features on the thread block configuration is fully considered, and a specific thread block configuration strategy is adopted according to the feature weight information of various bottleneck features. Therefore, it can solve technical problems such as inability to adapt to dynamic load changes, high maintenance costs, high data acquisition costs, and high complexity of model deployment and migration. It achieves the technical effect of being able to adapt to dynamic load changes, saving maintenance costs and data acquisition costs, and being able to dynamically generate the optimal thread block size.

[0043] See also Figure 2 , Figure 2 This is a flowchart of another thread block configuration method provided in an embodiment of the present application. The method may include the following steps.

[0044] S201: Obtaining the operating status indicators of each system by performing asynchronous sampling of indicators, and obtaining various preset bottleneck characteristics.

[0045] When collecting system performance indicators, asynchronous sampling is performed to obtain these indicators. For example, this is done using the Compute Unified Device Architecture Profiling Tools Interface (CUPTI). This asynchronous sampling of system performance indicators ensures that the hardware performance data collection process does not interfere with the main computing process.

[0046] When collecting system operation status indicators, performance indicators that are highly sensitive to thread block configuration are selected to ensure that the monitoring results can accurately reflect the impact of thread block configuration on hardware resource utilization.

[0047] By deeply analyzing the characteristics of the currently executing Compute Unified Device Architecture (CUDA) computing task, these characteristics are crucial for subsequent adjustments to thread block configuration. Specifically, when stream processor occupancy remains high while memory bandwidth utilization is low, the current task can be determined to be compute-intensive. Conversely, if memory bandwidth utilization is close to peak while stream processor occupancy is insufficient, it indicates that the task is limited by memory access. For control flow-intensive tasks, this manifests as low instruction issuance efficiency. By analyzing these characteristics, accurate task characteristics can be provided, enabling more reasonable thread block configuration decisions based on the actual needs of the task, thereby achieving optimal allocation and utilization of hardware resources and improving the execution efficiency and scalability of CUDA programs.

[0048] For example, compute-intensive features: When the stream processor occupancy rate is continuously higher than the dynamically adjusted threshold (such as 70%) and the memory bandwidth utilization rate is lower than the lower limit set in the current environment (such as 30%), the task is judged to have high compute density. Memory-intensive features: If the memory bandwidth utilization rate exceeds 60% and the stream processor occupancy rate is less than 40%, accompanied by a decrease in the first-level / texture (L1 / TEX) cache hit rate, it is identified as a memory bottleneck-dominated task. Control-flow-intensive features: Through the quantification of instruction emission efficiency, when the normalized value exceeds 0.5, it indicates that the task has control flow divergence.

[0049] S202: Calculating feature scores for each bottleneck feature based on the operating status indicators of each system to obtain feature scores.

[0050] S203: Calculate feature weights for each feature score to obtain feature weight information corresponding to each bottleneck feature.

[0051] S204: Obtain preset thread block size candidates.

[0052] Preset candidate thread block sizes, such as 128, 160, 192, 224, ..., 1024. Obtain the preset candidate thread block sizes.

[0053] In a specific implementation of the present application, step S204 may include the following steps:

[0054] Step 1: Obtain system hardware resource constraints;

[0055] Step 2: Determine the thread block size candidates based on the system hardware resource constraints.

[0056] For the convenience of description, the above two steps can be combined for explanation.

[0057] Thread blocks are subject to system hardware resource constraints. By obtaining these constraints and determining candidate thread block sizes based on them, we avoid illegal thread block size settings and ensure the legitimacy of these settings.

[0058] S205: Select a current thread block size candidate from each thread block size candidate according to each system operation status indicator and each feature weight information.

[0059] After obtaining the preset thread block size candidates, the current thread block size candidate is selected from the thread block size candidates according to the system operation status indicators and the feature weight information.

[0060] S206: Obtaining a system performance feedback value during actual system operation based on the current thread block size candidate.

[0061] After the current thread block size candidate is selected, a system performance feedback value in actual system operation based on the current thread block size candidate is obtained, such as the system execution time.

[0062] S207: Performing adaptive trade-off reinforcement learning between exploration and utilization according to the system performance feedback value to obtain a target thread block size, and configuring the system's thread blocks to the target thread block size.

[0063] After obtaining system performance feedback from actual system operation based on the current thread block size candidate, the system uses this feedback to perform reinforcement learning on the adaptive trade-off between exploration and utilization to determine the target thread block size. The system's thread blocks are then configured to that target thread block size, which is then safely and efficiently applied to the runtime configuration of CUDA kernel functions. This enables dynamic tuning of thread block size. The closed-loop feedback mechanism adapts to different hardware architectures and computing tasks, providing high scalability.

[0064] The system deeply analyzes the characteristics of currently executing CUDA computing tasks by combining computing task characteristics (such as data access patterns and compute density distribution). Based on this, the system integrates hardware status and task characteristics, using a multi-strategy fusion approach combining a rules engine and reinforcement learning to dynamically generate the optimal thread block size. This achieves optimal allocation and utilization of hardware resources, significantly improving resource utilization and program execution efficiency.

[0065] In a specific embodiment of the present application, after collecting the operating status indicators of each system, the method may further include the following steps:

[0066] Step 1: Obtain the preset indicator weights corresponding to each preset indicator dimension;

[0067] Step 2: normalize the operating status indicators of each system to obtain normalized indicator values;

[0068] Step 3: Calculate the dimension score corresponding to each preset indicator dimension based on each normalized indicator value;

[0069] Step 4: Perform weighted sum calculation on the scores of each dimension according to the preset indicator weights to obtain a comprehensive score;

[0070] Step 5: Adjust the indicator sampling frequency based on the comprehensive score.

[0071] For the convenience of description, the above five steps can be combined for explanation.

[0072] Obtain the preset indicator weights corresponding to each preset indicator dimension, normalize each system operation status indicator to obtain each normalized indicator value, calculate the dimension score corresponding to each preset indicator dimension based on each normalized indicator value, and perform a weighted summation of each dimension score based on the preset indicator weights to obtain a comprehensive score. The indicator sampling frequency is adjusted based on the comprehensive score. During the process of collecting monitoring indicators, adaptive sampling frequency control based on the comprehensive score of each system operation status indicator and asynchronous sampling are used to ensure accurate data collection and low overhead.

[0073] A multi-level monitoring system can be established from macro (stream processor level) to micro (thread bundle level) to comprehensively cover all dimensions of hardware performance.

[0074] At the macro level of the stream processor, we focus on the stream processor occupancy rate (active blocks per SM / max blocks per SM) and the active warp ratio (active warps per SM / max warps per SM). These two metrics provide a direct reflection of stream processor resource utilization and provide important evidence for determining whether thread block configuration is appropriate. For example, a low stream processor occupancy rate may indicate that the thread block size is too small, resulting in underutilization of stream processor resources. A high active warp ratio, on the other hand, may indicate intense contention among warps within a thread block, impacting execution efficiency.

[0075] At the memory level, monitoring focuses on memory bandwidth utilization and L1 / TEX cache hit rates. These metrics directly correlate to the efficiency of a CUDA program's access to memory resources and are key indicators of whether a function is limited by memory bandwidth during execution. For example, low memory bandwidth utilization may indicate that data transfer is a performance bottleneck, while L1 / TEX cache hit rates reflect the effectiveness of data locality utilization and are crucial for optimizing memory access patterns.

[0076] At the warp level, monitoring metrics include instruction issuance efficiency. This metric provides insight into warp execution details within a thread block, providing guidance for optimizing parallel execution within the thread block. For example, low instruction issuance efficiency may indicate resource contention or complex instruction dependencies within the thread block, impacting parallel execution efficiency.

[0077] In a specific embodiment of the present application, adjusting the indicator sampling frequency according to the comprehensive score may include the following steps:

[0078] Step 1: Obtain the preset upper and lower limits of the comprehensive score;

[0079] Step 2: When the comprehensive score exceeds the upper limit of the comprehensive score, the indicator sampling frequency is adjusted to the first preset frequency;

[0080] Step 3: When the comprehensive score is lower than the lower limit of the comprehensive score, the indicator sampling frequency is adjusted to the second preset frequency;

[0081] The first preset frequency is lower than the second preset frequency.

[0082] For the convenience of description, the above three steps can be combined for explanation.

[0083] Obtain the preset upper and lower limits of the comprehensive score. When the comprehensive score exceeds the upper limit, adjust the indicator sampling frequency to the first preset frequency. When the comprehensive score is lower than the lower limit, adjust the indicator sampling frequency to the second preset frequency. The first preset frequency is lower than the second preset frequency. For example, the first preset frequency can be set to 30Hz and the second preset frequency can be set to 200Hz. By lowering the indicator sampling frequency to the first preset frequency when the comprehensive score exceeds the upper limit, oversampling is avoided. By raising the indicator sampling frequency to the second preset frequency when the comprehensive score is lower than the lower limit, potential changes can be captured.

[0084] In a specific embodiment of the present application, after collecting the operating status indicators of each system, the method may further include the following steps:

[0085] Smoothing of the operating status indicators of each system.

[0086] Since hardware performance indicators may have short-term fluctuations, after collecting the operating status indicators of each system, they need to be smoothed, such as by using a moving average or an exponentially weighted average, to reduce noise interference.

[0087] In a specific embodiment of the present application, after collecting the operating status indicators of each system, the method may further include the following steps:

[0088] Step 1: Obtain the indicator threshold corresponding to each system operation status indicator;

[0089] Step 2: When the system is determined to be in a bottleneck scenario based on the system operation status indicators and the thresholds of the indicators, the thread block size is adjusted.

[0090] For the convenience of description, the above two steps can be combined for explanation.

[0091] Obtain the indicator threshold corresponding to each system operation status indicator. When the system is determined to be in a bottleneck scenario based on each system operation status indicator and each indicator threshold, adjust the thread block size to reduce memory pressure or increase stream processor occupancy.

[0092] Bottleneck scenarios can include memory-limited scenarios, compute-limited scenarios, compute-limited and high-computing-density scenarios, etc.

[0093] In a specific embodiment of the present application, obtaining the indicator thresholds corresponding to the respective system operation status indicators may include the following steps:

[0094] Step 1: Obtain the historical indicator sets corresponding to each system operation status indicator;

[0095] Step 2: Calculate the mean of each historical indicator set to obtain the mean of the historical indicators corresponding to each historical indicator set;

[0096] Step 3: Calculate the variance of each historical indicator set to obtain the historical indicator variance corresponding to each historical indicator set;

[0097] Step 4: Calculate the indicator threshold corresponding to each system operation status indicator based on each system operation status indicator, each historical indicator mean, and each historical indicator variance.

[0098] For the convenience of description, the above four steps can be combined for explanation.

[0099] Obtain the historical indicator sets corresponding to each system's operating status indicator, calculate the mean of each historical indicator set, and calculate the variance of each historical indicator set to obtain the variance of each historical indicator set. Based on each system's operating status indicator, the mean of each historical indicator, and the variance of each historical indicator, calculate the indicator threshold corresponding to each system's operating status indicator. The system dynamically adjusts the indicator threshold boundaries based on the mean and variance of stream processor utilization and memory bandwidth over a period of time, avoiding inaccurate feature classification caused by improper static threshold settings.

[0100] In a specific embodiment of the present application, when it is determined that the system is in a bottleneck scenario based on various system operating status indicators and thresholds of various indicators, thread block size adjustment may include the following steps:

[0101] Step 1: When the memory bandwidth utilization in each system operation status indicator exceeds the current memory bandwidth utilization threshold and the first-level cache hit rate is lower than the current first-level cache hit rate threshold, the current bottleneck scenario is determined to be a memory-limited scenario;

[0102] Step 2: Reduce the thread block size based on memory-constrained scenarios.

[0103] For the convenience of description, the above two steps can be combined for explanation.

[0104] When the memory bandwidth utilization in each system operation status indicator exceeds the current memory bandwidth utilization threshold and the first-level cache hit rate is lower than the current first-level cache hit rate threshold, the current bottleneck scenario is determined to be a memory-limited scenario. The thread block size is reduced based on the memory-limited scenario to reduce memory pressure.

[0105] In a specific embodiment of the present application, when it is determined that the system is in a bottleneck scenario based on various system operating status indicators and thresholds of various indicators, thread block size adjustment may include the following steps:

[0106] Step 1: When the stream processor occupancy rate in each system operation status indicator is lower than the current stream processor occupancy rate threshold, and the thread bundle efficiency exceeds the current memory bandwidth utilization threshold, the current bottleneck scenario is determined to be a compute-limited scenario;

[0107] Step 2: Increase the thread block size based on the computationally constrained scenario.

[0108] For the convenience of description, the above two steps can be combined for explanation.

[0109] When the stream processor occupancy rate in each system operation status indicator is lower than the current stream processor occupancy rate threshold and the thread bundle efficiency exceeds the current memory bandwidth utilization threshold, the current bottleneck scenario is determined to be a compute-limited scenario. The thread block size is increased based on the compute-limited scenario to improve the stream processor occupancy rate.

[0110] In a specific embodiment of the present application, when it is determined that the system is in a bottleneck scenario based on various system operating status indicators and thresholds of various indicators, thread block size adjustment may include the following steps:

[0111] When the current bottleneck scenario is determined to be a computationally limited and high-computing-density scenario based on various system operating status indicators and thresholds of various indicators, the thread block size is increased.

[0112] When the current bottleneck scenario is determined to be computationally limited and high-computing-density based on the system operating status indicators and the thresholds of each indicator, the thread block size is increased to improve the stream processor occupancy rate.

[0113] To prevent frequent sampling and excessive metrics from impacting operational efficiency, during actual deployment, we continuously sample several key metrics, including SM occupancy, active thread bundle ratio, memory bandwidth utilization, L1 / TEX cache hit rate, and instruction issuance efficiency. The following pseudocode illustrates the basic process of data collection and processing and the hardware interface layer implementation.

[0114] .

[0115] In the process of collecting and monitoring the operating status indicators of the monitoring system, adaptive sampling frequency control is adopted. The adjustment of the sampling frequency is based on the current monitored indicator data, which is achieved through a weighted comprehensive scoring method to dynamically balance the sampling overhead and data accuracy.

[0116] For example, the weight of the stream processor utilization metric is 0.4, the weight of the memory utilization metric is 0.4, and the weight of the warp-level metric is 0.2. The normalized values ​​of each metric are multiplied by the weights and summed to obtain a comprehensive score. The sampling frequency is then dynamically adjusted based on the comprehensive score. The weights of metrics can also be adjusted for different types of tasks. The following example illustrates this idea:

[0117] When the comprehensive score exceeds a threshold (e.g., 0.8), the sampling frequency is reduced;

[0118] When the comprehensive score is lower than a threshold (such as 0.2), the sampling frequency is increased.

[0119] .

[0120] The pseudo code for smoothing the collected system operation status indicators is as follows.

[0121] .

[0122] This module maintains a circular buffer storing hardware metrics from the last N sampling cycles. During each update, it first calculates the median of each metric within the window and then generates a three-dimensional feature vector based on predefined feature discrimination rules. To mitigate noise interference within a single sampling cycle, the module employs a dual validation mechanism: a short-term window (5-10 cycles) for rapid response to feature changes, and long-term trend analysis (50-100 cycles) to correct for systematic deviations. This layered processing ensures both real-time performance and robust feature discrimination, providing a reliable input feature space for the subsequent configuration decision engine. The following is pseudocode for the core functionality of this module.

[0123]

[0124] .

[0125] To improve the sensitivity of bottleneck analysis, a dynamic weight mechanism is introduced to evaluate computing and memory bottlenecks. The pseudo code is as follows.

[0126]

[0127]

[0128] .

[0129] Reinforcement learning policy input: Dynamic weights can be used as components of the state to improve the convergence and generalization capabilities of reinforcement learning.

[0130] The reinforcement learning module is primarily responsible for further fine-tuning the thread block size recommendation strategy when empirical rules and performance models fail to cover or are ineffective. The thread block configuration search considers factors such as bottleneck type, resource utilization, and instruction issuance efficiency, aiming to maximize overall efficiency while meeting hardware resource constraints. To enhance decision-making flexibility and scalability, a caching mechanism, empirical rules, and reinforcement learning fine-tuning mechanisms are introduced to achieve multi-strategy integration.

[0131] The reinforcement learning submodule is defined as:

[0132] State: The hardware state vector under the current configuration (occupancy, mem_util, warp_eff, and the dynamic weight calculated above);

[0133] Action: A choice among the candidate block_size options (e.g., {128, 160, 192, 224,...,1024});

[0134] Reward: The performance benefit of the current block configuration in actual operation (reduced execution time);

[0135] Policy: Adopt Strategy,adaptively balances exploration and exploitation.

[0136] The reinforcement learning module uses cached data for training and continuously fine-tunes the configuration recommendation strategy.

[0137]

[0138] .

[0139] After obtaining the thread block configuration parameters, they are securely applied to the CUDA kernel launch process, while also integrating an execution performance monitoring mechanism. Its core objectives include preventing illegal configurations (such as exceeding shared memory or thread limits), optimizing kernel launch overhead using the configuration cache, and providing real-time performance data during kernel runtime. The pseudocode is as follows.

[0140]

[0141]

[0142] .

[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0144] An embodiment of the present application also provides a thread block configuration device.

[0145] See also Figure 3 , Figure 3 This is a block diagram of a thread block configuration provided in an embodiment of the present application. The thread block configuration device may include:

[0146] The indicator and feature acquisition module 31 is used to collect the operating status indicators of each system and obtain various preset bottleneck features;

[0147] The feature score obtaining module 32 is used to calculate the feature scores of various bottleneck features according to the operating status indicators of each system to obtain the feature scores;

[0148] The feature weight information obtaining module 33 is used to calculate the feature weight of each feature score to obtain the feature weight information corresponding to each bottleneck feature;

[0149] The thread block configuration module 34 is used to configure thread blocks according to the system operation status indicators and the feature weight information.

[0150] Through this application, by collecting the operating status indicators of each system and obtaining various preset bottleneck features, the feature scores of various bottleneck features are calculated according to the operating status indicators of each system, and the weights are calculated according to the calculated feature scores. Then, the thread blocks are configured in combination with the operating status indicators of each system and the feature weight information. The influence of various bottleneck features on the thread block configuration is fully considered, and a specific thread block configuration strategy is adopted according to the feature weight information of various bottleneck features. Therefore, it can solve technical problems such as inability to adapt to dynamic load changes, high maintenance costs, high data acquisition costs, and high complexity of model deployment and migration. It achieves the technical effect of being able to adapt to dynamic load changes, saving maintenance costs and data acquisition costs, and being able to dynamically generate the optimal thread block size.

[0151] In a specific implementation of the present application, the indicator and feature acquisition module 31 is specifically a module that obtains the operating status indicators of each system by performing asynchronous sampling of indicators.

[0152] In a specific embodiment of the present application, the device may further include:

[0153] The indicator weight acquisition module is used to obtain the preset indicator weights corresponding to each preset indicator dimension;

[0154] The indicator normalization module is used to normalize the operating status indicators of each system to obtain the normalized indicator values;

[0155] The dimension score calculation module is used to calculate the dimension score corresponding to each preset indicator dimension according to each normalized indicator value;

[0156] The comprehensive score acquisition module is used to perform weighted sum calculation on the scores of each dimension according to the preset indicator weights to obtain a comprehensive score;

[0157] The sampling frequency adjustment module is used to adjust the indicator sampling frequency according to the comprehensive score.

[0158] In a specific embodiment of the present application, the sampling frequency adjustment module may include:

[0159] The submodule for obtaining the upper and lower limits of the comprehensive score is used to obtain the preset upper and lower limits of the comprehensive score;

[0160] A first preset frequency adjustment submodule, configured to adjust the indicator sampling frequency to a first preset frequency when the comprehensive score exceeds an upper limit of the comprehensive score;

[0161] A second preset frequency adjustment submodule, configured to adjust the indicator sampling frequency to a second preset frequency when the comprehensive score is lower than the comprehensive score lower limit;

[0162] The first preset frequency is lower than the second preset frequency.

[0163] In a specific embodiment of the present application, the device may further include:

[0164] The smoothing processing module is used to perform smoothing processing on the operating status indicators of each system after collecting the operating status indicators of each system.

[0165] In a specific embodiment of the present application, the device may further include:

[0166] The indicator threshold acquisition module is used to obtain the indicator threshold corresponding to each system operation status indicator after collecting the system operation status indicators;

[0167] The thread block size adjustment module is used to adjust the thread block size when it is determined that the system is in a bottleneck scenario based on various system operation status indicators and various indicator thresholds.

[0168] In a specific embodiment of the present application, the indicator threshold acquisition module may include:

[0169] The historical indicator set acquisition submodule is used to obtain the historical indicator sets corresponding to each system operation status indicator;

[0170] The mean calculation submodule is used to perform mean calculation on each historical indicator set to obtain the mean of the historical indicators corresponding to each historical indicator set;

[0171] The variance calculation submodule is used to perform variance calculation on each historical indicator set to obtain the historical indicator variance corresponding to each historical indicator set;

[0172] The indicator threshold calculation submodule is used to calculate the indicator threshold corresponding to each system operation status indicator based on each system operation status indicator, the mean value of each historical indicator and the variance of each historical indicator.

[0173] In a specific implementation of the present application, the thread block size adjustment module may include:

[0174] A memory-limited scenario determination submodule is configured to determine that the current bottleneck scenario is a memory-limited scenario when the memory bandwidth utilization in each system operation status indicator exceeds the current memory bandwidth utilization threshold and the first-level cache hit rate is lower than the current first-level cache hit rate threshold;

[0175] The thread block size reduction submodule is used to reduce the thread block size according to memory-constrained scenarios.

[0176] In a specific implementation of the present application, the thread block size adjustment module may include:

[0177] A computation-limited scenario determination submodule is configured to determine that the current bottleneck scenario is a computation-limited scenario when the stream processor occupancy rate in each system operation status indicator is lower than the current stream processor occupancy rate threshold and the thread bundle efficiency exceeds the current memory bandwidth utilization threshold;

[0178] The thread block size up-scaling submodule is used to increase the thread block size according to the computationally constrained scenario.

[0179] In a specific embodiment of the present application, the thread block size adjustment module is specifically a module that increases the thread block size when the current bottleneck scenario is determined to be a computationally limited and high computational density scenario based on various system operation status indicators and various indicator thresholds.

[0180] In a specific implementation of the present application, the thread block configuration module 34 may include:

[0181] A candidate acquisition submodule is used to obtain preset candidate items for each thread block size;

[0182] A candidate selection submodule is used to select a current thread block size candidate from each thread block size candidate according to each system operation status indicator and each feature weight information;

[0183] The performance feedback value acquisition submodule is used to obtain the system performance feedback value in the actual operation of the system based on the current thread block size candidate;

[0184] The thread block configuration submodule is used to perform adaptive trade-off reinforcement learning between exploration and utilization based on the system performance feedback value, obtain the target thread block size, and configure the system's thread block to the target thread block size.

[0185] In a specific embodiment of the present application, the candidate item acquisition submodule may include:

[0186] A hardware resource constraint acquisition unit, used to acquire system hardware resource constraints;

[0187] The candidate item determination unit is used to determine each thread block size candidate item according to the system hardware resource constraints.

[0188] For the description of the features in the embodiment corresponding to the thread block configuration device, please refer to the relevant description of the embodiment corresponding to the thread block configuration method, which will not be repeated here.

[0189] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned thread block configuration method embodiments.

[0190] An embodiment of the present application further provides a non-volatile computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above-mentioned thread block configuration method embodiments when running.

[0191] In an exemplary embodiment, the non-volatile computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0192] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned thread block configuration method embodiments are implemented.

[0193] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned thread block configuration method embodiments are implemented.

[0194] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0195] The above describes in detail the thread block configuration method, device, readable storage medium, and program product provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only intended to help understand the method and core concept of the present application. It should be noted that, for those skilled in the art, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A thread block configuration method, characterized in that: include: Collect the operating status indicators of each system and obtain the preset bottleneck characteristics; Calculate the feature scores of each bottleneck feature according to the operating status indicators of each system to obtain the feature scores; Calculate the feature weight of each feature score to obtain the feature weight information corresponding to each bottleneck feature; Get the preset thread block size candidates; Selecting a current thread block size candidate from each thread block size candidate according to each system operation status indicator and each feature weight information; Obtaining a system performance feedback value during actual system operation based on the current thread block size candidate; According to the system performance feedback value, The strategy performs adaptive trade-off reinforcement learning between exploration and utilization, obtains a target thread block size, and configures the thread blocks of the system to the target thread block size.

2. The thread block configuration method according to claim 1, wherein: Collect operating status indicators of each system, including: The operating status indicators of each system are obtained by asynchronously sampling the indicators.

3. The thread block configuration method according to claim 1, wherein: After collecting the operating status indicators of each system, it also includes: Get the preset indicator weights corresponding to each preset indicator dimension; Normalize the operating status indicators of each system to obtain normalized indicator values; Calculate the dimension scores corresponding to each preset indicator dimension based on each normalized indicator value; The scores of each dimension are weighted and summed according to the preset indicator weights to obtain a comprehensive score; The indicator sampling frequency is adjusted according to the comprehensive score.

4. The thread block configuration method according to claim 3, wherein: Adjust the sampling frequency of indicators based on the comprehensive score, including: Get the preset upper and lower limits of the comprehensive score; When the comprehensive score exceeds the comprehensive score upper limit, adjusting the indicator sampling frequency to a first preset frequency; When the comprehensive score is lower than the comprehensive score lower limit, adjusting the indicator sampling frequency to a second preset frequency; The first preset frequency is lower than the second preset frequency.

5. The thread block configuration method according to claim 1, wherein: After collecting the operating status indicators of each system, it also includes: Smoothing of the operating status indicators of each system.

6. The thread block configuration method according to claim 1, wherein: After collecting the operating status indicators of each system, it also includes: Obtain the indicator threshold corresponding to each system operation status indicator; When the system is determined to be in a bottleneck scenario based on various system operation status indicators and thresholds of various indicators, the thread block size is adjusted.

7. The thread block configuration method according to claim 6, wherein: Obtain the indicator thresholds corresponding to each system operation status indicator, including: Obtain the historical indicator sets corresponding to each system operation status indicator; Calculate the mean of each historical indicator set respectively to obtain the mean of the historical indicators corresponding to each historical indicator set; Calculate the variance of each historical indicator set respectively to obtain the historical indicator variance corresponding to each historical indicator set; According to the operating status indicators of each system, the mean value of each historical indicator and the variance of each historical indicator, the indicator threshold corresponding to each system operating status indicator is calculated.

8. The thread block configuration method according to claim 6, wherein: When the system is determined to be in a bottleneck scenario based on the system operating status indicators and the thresholds of each indicator, the thread block size is adjusted, including: When the memory bandwidth utilization in each system operation status indicator exceeds the current memory bandwidth utilization threshold and the first-level cache hit rate is lower than the current first-level cache hit rate threshold, the current bottleneck scenario is determined to be a memory-limited scenario; The thread block size is reduced according to the memory-constrained scenario.

9. The thread block configuration method according to claim 6, wherein: When the system is determined to be in a bottleneck scenario based on the system operating status indicators and the thresholds of each indicator, the thread block size is adjusted, including: When the stream processor occupancy rate in each system operation status indicator is lower than the current stream processor occupancy rate threshold, and the thread bundle efficiency exceeds the current memory bandwidth utilization threshold, the current bottleneck scenario is determined to be a computing-limited scenario; The thread block size is increased according to the computationally limited scenario.

10. The thread block configuration method according to claim 6, wherein: When the system is determined to be in a bottleneck scenario based on the system operating status indicators and the thresholds of each indicator, the thread block size is adjusted, including: When it is determined according to each system operation status indicator and each indicator threshold that the current bottleneck scenario is a computationally limited and high computationally dense scenario, the thread block size is increased.

11. The thread block configuration method according to claim 1, wherein: Get the preset thread block size candidates, including: Get system hardware resource constraints; Determine each thread block size candidate according to the system hardware resource constraints.

12. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the thread block configuration method according to any one of claims 1 to 11 when executing the computer program.

13. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the thread block configuration method according to any one of claims 1 to 11 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the thread block configuration method according to any one of claims 1 to 11 are implemented.

Images