Gpu and hbm memory fusion management method and system

Abstract

The application relates to the technical field of computer hardware resource management, and discloses a GPU and HBM memory fusion management method and system, which comprises the following steps: collecting historical operation data, extracting hardware features, and constructing an initial resource management set; analyzing the parameter matching of historical management content and the initial set, and screening a second resource management set; analyzing the cooperative operation mode of the GPU and the HBM memory; analyzing the fusion adaptability of the cooperative operation mode and the second set, and screening a third resource management set; and according to real-time operation feedback records, obtaining corresponding resource information in the third set to optimize fusion management. The system comprises a data collection and feature extraction module, a historical data analysis and screening module, a fusion adaptation screening module, and a real-time optimization module. Through mining of historical data, combination of hardware features and the cooperative operation mode, dynamic optimization management of the GPU and the HBM memory resources is realized, and resource utilization efficiency and system performance are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer hardware resource management, specifically to a GPU and HBM memory fusion management method and system. BACKGROUND

[0002] In a computer system, GPU (Graphics Processing Unit) is the core component of high-performance computing, and its computing capability depends largely on the cooperation with memory. HBM (High Bandwidth Memory) has become the ideal memory choice for GPU due to its high bandwidth and low latency. However, with the increasing complexity and diversity of GPU computing tasks, the resource management between GPU and HBM memory faces many challenges.

[0003] Traditional GPU and HBM memory resource management methods often manage them as independent individuals without fully considering their cooperative working mode. This management method often leads to unreasonable resource allocation in actual operation. For example, when GPU performs high-load computing, the bandwidth of HBM memory may be insufficient or the storage delay may be too high, which may affect the performance of the entire system.

[0004] At the same time, existing management methods are mostly based on fixed parameter settings and lack effective use of historical running data. When facing different application scenarios and workloads, they cannot dynamically adjust resource management strategies according to actual conditions, making the system less adaptable. For example, when processing graphics rendering tasks and scientific computing tasks, the resource requirements of GPU and HBM memory are very different, and traditional methods cannot meet the needs of different tasks.

[0005] In addition, with the continuous development of technology, the number of computing cores, cache capacity of GPU, and interface bandwidth of HBM memory are constantly upgraded, and traditional resource management methods cannot fully utilize the performance advantages of these new hardware. For example, new GPUs may have more computing cores and larger cache capacity, and HBM memories may have higher interface bandwidth, and traditional methods cannot optimize resource management according to these hardware features.

[0006] Moreover, in actual operation, the cooperative running mode of GPU and HBM memory is dynamically changing. Traditional management methods cannot monitor and analyze this change in real time, so they cannot adjust resource management strategies in a timely manner, leading to problems such as resource waste or performance bottlenecks in system operation. For example, when the cooperative update frequency of GPU and HBM memory changes, traditional methods cannot adjust resource allocation in a timely manner, affecting the running efficiency of the system. SUMMARY

[0007] The application aims to provide a GPU and HBM memory fusion management method and system to solve the problems in the background art.

[0008] To achieve the above-mentioned purpose, the application provides the following technical solutions: a GPU and HBM memory fusion management method and system, the method comprising:

[0009] S1, collecting historical running data of GPU and HBM memory, extracting hardware features, and constructing an initial resource management set;

[0010] S2, analyzing the parameter matching of historical management content and the content in the initial resource management set according to the historical running data, and screening a second resource management set;

[0011] S3, analyzing the cooperative running mode of GPU and HBM memory according to the historical running data;

[0012] S4, analyzing the fusion adaptability of GPU and HBM memory and the content in the second resource management set according to the cooperative running mode of GPU and HBM memory, and screening a third resource management set;

[0013] S5, obtaining the corresponding resource information in the third resource management set according to the real-time running feedback record of GPU and HBM memory, and optimizing the fusion management in the running process.

[0014] Preferably, the S1 comprises the following specific steps:

[0015] S101, inputting the historical running data of GPU and HBM memory in the running data collection module, extracting hardware features in the historical running data, and the hardware features comprising hardware type features and resource demand features;

[0016] S102, screening the content containing hardware type features in the resource management module according to the hardware type features and outputting in the form of an initial resource management set.

[0017] Preferably, the S2 comprises the following specific steps:

[0018] S201, obtaining the historical running data of GPU and HBM memory, analyzing the features of historical management content according to the historical running data, the features of historical management content comprising resource identification features and parameter features, screening the historical management content containing resource demand features according to the resource identification features and the resource demand features, and obtaining the parameter features of the content in the initial resource management set;

[0019] S202, analyzing the parameter matching degree of the screened historical management content and the content in the initial resource management set according to the parameter features of the screened historical management content and the parameter features of the content in the initial resource management set.

[0020] S203, sort the parameter matching degrees in descending order, filter out the corresponding content within the set serial number, and output in the form of a second resource management set.

[0021] Preferably, the S3 comprises the following specific steps:

[0022] S301, extract the running time and resource occupation depth in the filtered historical running data, obtain the running time interval of the adjacent two historical running data, and then take the average of the obtained several running time intervals to obtain the average running time interval, and take the average of the obtained several resource occupation depths to obtain the average resource depth;

[0023] S302, compare the average running time interval with the first time interval threshold and the second time interval threshold, and determine the cooperative running mode of the GPU and the HBM memory, if the average running time interval is less than or equal to the first time interval threshold, it is determined that the GPU and the HBM memory have a high cooperative update frequency, if the average running time interval is greater than the first time interval threshold and the running time interval is less than the second time interval threshold, it is determined that the GPU and the HBM memory have a medium cooperative update frequency, and if the average running time interval is greater than or equal to the second time interval threshold, it is determined that the GPU and the HBM memory have a low cooperative update frequency;

[0024] S303, obtain a cooperative habit score value according to the cooperative running mode of the GPU and the HBM memory.

[0025] Preferably, the S4 comprises the following specific steps:

[0026] S401, obtain the fusion feature and resource depth of the content in the second resource management set, and analyze the fusion adaptability of the GPU and the HBM memory to the content in the second resource management set according to the cooperative habit score value, the average resource depth, the fusion feature and the resource depth of the content in the second resource management set;

[0027] S402, sort the fusion adaptability in descending order, filter out the corresponding content within the set serial number, and output in the form of a third resource management set.

[0028] Preferably, the S5 comprises the following specific steps:

[0029] Obtain the real-time running feedback record of the GPU and the HBM memory to analyze whether the GPU and the HBM memory have a comparison management habit, if the GPU and the HBM memory do not have a comparison management habit, obtain the resource information with the highest fusion matching degree in the third resource management set, and optimize the fusion management during running, if the GPU and the HBM memory have a comparison management habit, obtain the corresponding resource information within the set serial number in the third resource management set, and optimize the fusion management during running.

[0030] The analysis method of whether the GPU and the HBM memory have the comparison management habit is that the proportion of the number of times of simultaneously calling multiple resources in the total running times in the real-time running feedback record of the GPU and the HBM memory is counted, and if the proportion exceeds a preset comparison management habit threshold, it is determined that the GPU and the HBM memory have the comparison management habit.

[0031] Preferably, the hardware type features include specific hardware features of a computing core, a cache capacity and an interface bandwidth, and the resource demand features include demand features of a computing load, a memory bandwidth and a storage delay.

[0032] Preferably, the fusion adaptation analysis of the GPU and the HBM memory and the content in the second resource management set includes a matching degree of a cooperative habit score value and a content fusion feature and a fitting degree of an average resource depth and a content resource depth.

[0033] Preferably, the analysis method of the parameter matching degree is that the parameter features of the screened historical management content and the parameter features of the content in the initial resource management set are analyzed item by item, and the proportion of the number of matched parameter items to the total number of parameter items is counted.

[0034] Preferably, the application further includes a GPU and HBM memory fusion management system applied to the GPU and HBM memory fusion management method, and the system includes:

[0035] A data acquisition and feature extraction module is configured to acquire historical running data of the GPU and the HBM memory, extract hardware features and construct an initial resource management set.

[0036] A historical data analysis and screening module is configured to analyze the parameter matching of historical management content and content in the initial resource management set according to the historical running data, screen out a second resource management set and analyze a cooperative running mode of the GPU and the HBM memory.

[0037] A fusion adaptation screening module is configured to analyze the fusion adaptation of the GPU and the HBM memory and the content in the second resource management set according to the cooperative running mode of the GPU and the HBM memory, and screen out a third resource management set.

[0038] A real-time optimization module is configured to acquire corresponding resource information in the third resource management set according to real-time running feedback records of the GPU and the HBM memory, and optimize the fusion management in the running process.

[0039] Compared with the prior art, the application has the following beneficial effects:

[0040] By collecting historical running data of GPU and HBM memory and extracting hardware features to construct an initial resource management set, the management content that matches the hardware features can be mined from the historical running situation, providing a more targeted basis for subsequent resource management, avoiding the problem of blind management in traditional methods, and making resource management better adapt to the characteristics of hardware.

[0041] According to the parameter matching of historical management content and the content in the initial resource management set, the second resource management set is screened out, which fully utilizes the historical management experience, and through the analysis of parameter matching degree, the management content that better meets the actual running demand is screened out, improving the quality of the resource management set, making the subsequent management strategy more feasible and effective, and better coping with various situations in actual operation.

[0042] The cooperative operation mode of GPU and HBM memory is analyzed, and the fusion adaptability of the content in the second resource management set is analyzed according to the mode, and the third resource management set is screened out, which fully considers the cooperative relationship between GPU and HBM memory. Under different cooperative operation modes, the requirements for resource management are different, and through this way, the most suitable resource management content for the current cooperative operation mode can be screened out, so that the resource management is more accurate, and the cooperative work of GPU and HBM memory can be better promoted, and the overall performance of the system is improved.

[0043] During operation, the corresponding resource information in the third resource management set is obtained according to the real-time running feedback record and optimized fusion management, realizing the dynamic adjustment of resource management. It can adjust the management strategy according to the real-time running state of the system in time, and when the system running state changes, such as the habit of comparative management, it can respond quickly, avoiding the problem that the fixed management strategy in traditional methods cannot adapt to dynamic changes, making the resource management always in the optimal state, and further improving the running efficiency and stability of the system.

[0044] By clearly defining the hardware type features and resource demand features, including computing core, cache capacity, interface bandwidth, and computing load, memory bandwidth, storage delay, resource management can more comprehensively consider various aspects of hardware, so as to develop a more reasonable resource allocation scheme, fully utilize the performance advantages of hardware, and avoid waste of hardware resources.

[0045] Detailed analysis and screening of parameter matching degree, fusion adaptability, etc. The parameter matching degree is analyzed by corresponding analysis of each item to obtain the proportion value of the matching parameter item, and the fusion adaptability considers the matching degree of the cooperative habit score value and the content fusion feature, and the matching degree of the average resource depth and the content resource depth, etc. The screened resource management content has higher accuracy and adaptability, the resource management is more scientific and reasonable, and the cooperative operation requirements of the GPU and the HBM memory can be better met. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 The working principle diagram of the GPU and HBM memory fusion management method is described.

[0047] Figure 2 The design diagram for screening the second resource management set is described.

[0048] Figure 3 The design diagram for analyzing the cooperative operation mode is described.

[0049] Figure 4 The design diagram for real-time optimization of fusion management is described. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0051] Please refer to Figures 1-4 The GPU and HBM memory fusion management method according to the application includes the following specific implementation steps:

[0052] Collecting historical operation data of the GPU and the HBM memory, extracting hardware features, and constructing an initial resource management set.

[0053] According to the historical operation data, the parameter matching of the historical management content and the content in the initial resource management set is analyzed, and a second resource management set is screened out.

[0054] According to the historical operation data, the cooperative operation mode of the GPU and the HBM memory is analyzed.

[0055] According to the cooperative operation mode of the GPU and the HBM memory, the fusion adaptability of the GPU and the HBM memory and the content in the second resource management set is analyzed, and a third resource management set is screened out.

[0056] According to the real-time running feedback record of the GPU and the HBM memory, corresponding resource information in the third resource management set is acquired, and fusion management is optimized in the running process.

[0057] Embodiment 1:

[0058] This embodiment details step S1. The running data acquisition module needs to establish a connection with the hardware system of the GPU and the HBM memory through a specific data interface to realize continuous acquisition of historical running data. These data interfaces can be physical interfaces at the hardware level or API interfaces at the software level, ensuring that the running data of the device under different working conditions can be stably and reliably acquired. The acquired historical running data covers the running information of the device under various application scenarios, including but not limited to the workload types during device operation, such as scientific computing, graphics rendering, data processing, and different task scenarios; various performance indicators during device operation, such as the utilization rate of computing cores, the read / write rate of memory, storage delay time, etc.; and timestamp information of device operation, used to record the generation time of each data point.

[0059] After obtaining the historical running data, in-depth analysis and processing of these data are needed to extract the hardware features. Hardware features mainly include two categories: hardware type features and resource demand features. For the extraction of hardware type features, the specific information of computing cores needs to be accurately identified first. As a key component of the GPU, the model, architecture, and number of computing cores directly affect the computing power of the device. For example, when identifying the computing core model, specific information such as NVIDIA's Ampere architecture, AMD's RDNA architecture, etc. needs to be extracted from the running data, and the number of cores needs to be determined, such as a certain GPU containing 5120 CUDA cores. The extraction of cache capacity is also crucial, and the specific capacity of each level of cache such as L1 cache and L2 cache needs to be determined. In the running data, there are usually records about cache configuration and usage, and through the analysis of these records, specific values such as L1 cache of 64KB and L2 cache of 4MB can be obtained. The extraction of interface bandwidth needs to focus on the relevant parameters of data transmission, including the rate and number of channels. For example, the interface between HBM memory and GPU may use a specific bus protocol, and the running data will contain information such as a bandwidth of 4096GB / s and a channel number of 8. Through the extraction of these information, the hardware features of the interface bandwidth can be accurately described.

[0060] The extraction of resource requirement characteristics is to analyze the resource requirements of the device under different tasks from the running data. The type and intensity of the computing load are important components of the resource requirement characteristics. In different application scenarios, the computing load presents different characteristics, such as scientific computing tasks may require a large number of floating-point operations, and graphics rendering tasks have a high requirement for parallel computing capability. Through the analysis of the type and operation amount of the computing task in the running data, the demand characteristics of the computing load can be determined, for example, in a certain task, the computing load is characterized by high-intensity floating-point operation, which requires the GPU to have strong single-precision computing capability. The demand for memory bandwidth is determined according to the data transmission rate requirement. In the running data, the rate of data transmission between the GPU and the HBM memory is recorded, and through the analysis of these data, the demand for memory bandwidth of the device under certain tasks can be obtained, such as when processing large amounts of data, the memory bandwidth needs to reach 2000 GB / s or more to meet the data transmission requirements. The demand evaluation of storage delay needs to pay attention to the delay time of data reading and writing. In the running data, there are records about data access delay, and through the analysis of these records, the sensitivity of the device to storage delay under different tasks can be determined, for example, in tasks with high real-time response requirements, the storage delay needs to be controlled within 100 ns.

[0061] After the extraction of the hardware type characteristics is completed, a filtering operation needs to be performed in the resource management module. The resource management module is a database or data structure that stores a large amount of hardware-related management content, including various management strategies and configuration parameters for different hardware characteristics. In the filtering process, the extracted computing core, cache capacity, interface bandwidth, and other specific hardware type characteristics are used as query conditions for accurate or fuzzy matching in the resource management module. For example, when the extracted computing core model is NVIDIA A100 SM86 architecture, the resource management module will retrieve all management content for this architecture computing core; when the cache capacity is 4MB, the management strategy matching the cache capacity will be filtered out. In this way, the content in the resource management module containing these hardware type characteristics is filtered out to form a preliminary candidate set.

[0062] The selected candidate content needs to be output in the form of an initial resource management set. The output form of the initial resource management set needs to meet certain data structure requirements so that subsequent steps can conveniently process and use it. The set can adopt data structures such as lists, dictionaries, etc., where each element contains specific information of the selected management content, such as the name of the management policy, specific configuration parameters, applicable hardware conditions, etc. During output, the integrity and accuracy of the data need to be ensured to avoid loss or errors of information. At the same time, in order to improve the efficiency of subsequent processing, the output of the initial resource management set may also need to be sorted or indexed, for example, sorted according to the matching degree of hardware features, so that subsequent steps can preferentially process management content with high matching degree.

[0063] Through the above detailed implementation, the collection of GPU and HBM memory historical running data, the extraction of hardware features, and the construction of the initial resource management set are realized, which provides a solid data foundation for subsequent steps such as resource management set screening, collaborative running mode analysis, etc.

[0064] Embodiment 2:

[0065] This embodiment describes step S2 in detail. When obtaining the historical running data of GPU and HBM memory, the system preset data collection channel needs to continuously collect the management records of the device in different running stages. These historical running data contain all the operation logs related to resource management from the start to the running process of the device, such as the time point of resource allocation, the specific strategy of memory scheduling, the configuration change of hardware parameters, etc. The data collection channel needs to ensure the integrity and time sequence of the data to avoid affecting subsequent analysis due to data loss or sequence disorder.

[0066] After obtaining the historical running data, the features of the historical management content need to be analyzed, including resource identification features and parameter features. The resource identification feature is an identifier used to uniquely define different management policies, such as the code of a certain memory allocation strategy, the hash value of a specific cache scheduling rule, etc. These identifiers usually exist in the form of fixed fields in system logs. The parameter feature is a quantifiable configuration item in the management content, such as the block size of memory allocation, the priority coefficient of cache replacement, the weight value of bandwidth allocation, etc. These parameters will change dynamically with different management scenarios.

[0067] According to the resource identification feature and the resource demand feature extracted in step S1, the historical management content is screened. The demand for computing load, memory bandwidth, storage delay, etc. in the resource demand feature will form a screening condition. For example, when the resource demand feature shows that the current task requires 3000 GB / s of memory bandwidth, the system will retrieve all strategies in the historical management content that meet this bandwidth requirement. The screening process needs to be implemented through a database query language, and the resource identification feature and the resource demand feature are matched in the field. For example, in the SQL query, the records with "memory bandwidth requirement = 3000 GB / s" are limited through the WHERE clause, and the unique index of the resource identification feature is combined to quickly locate the historical management content that meets the condition.

[0068] When obtaining the parameter features of the contents in the initial resource management set, the specific parameters need to be parsed from the storage structure of the set. The initial resource management set is usually stored in a structured data format, such as JSON or XML, and each management strategy contains a parameter list. For example, the parameter features of a certain management strategy may include "compute core scheduling threshold = 80%" and "cache prefetch depth = 16KB". The system needs to extract these parameters through a parser and form a feature vector.

[0069] The analysis of the parameter matching degree needs to compare the parameter features of the screened historical management content with the parameter features of the contents in the initial resource management set item by item. The comparison process uses field alignment. For example, for the parameter "memory allocation block size", if the parameter value of the historical management content is "256MB" and the corresponding parameter value in the initial resource management set is "256MB" or within a preset error range (such as ±5%), it is determined to be matched. The system needs to traverse all parameter items and calculate the proportion of the number of matched parameter items to the total number of parameter items. For example, if a certain historical management content contains 10 parameter items, 8 of which match the parameters in the initial resource management set, the proportion is 80%.

[0070] The sorting of the parameter matching degree needs to establish a comparison function to convert the proportion value into a sortable numerical data. The sorting algorithm uses efficient algorithms such as quicksort or mergesort to ensure processing efficiency under large data volume. After sorting, the content is screened according to the preset serial number threshold (such as the top 10% or the top 20 records). For example, the top 10 records with the highest matching degree are selected. The screened content needs to be reorganized into a second resource management set, and the storage structure of this set needs to be compatible with the initial resource management set, but the parameter matching degree information will be added as a new field for subsequent steps.

[0071] When analyzing parameter characteristics, compatibility issues of different data types need to be handled. For example, numerical parameters such as bandwidth values need to be compared numerically, enumeration parameters such as scheduling strategy types need to be matched with enumeration values, and text parameters such as strategy descriptions need to be calculated for keyword similarity. For text parameters, the edit distance algorithm (such as Levenshtein distance) can be used to calculate similarity, and a threshold (such as 0.7) is set to be considered as a match.

[0072] The analysis of historical running data may involve aggregation processing across time periods, for example, when historical management content is scattered in different time periods, it needs to be grouped according to the type of resource demand characteristics, and then the parameter matching degree of each group is calculated respectively. For example, the historical management content of scientific computing tasks and graphics rendering tasks is divided into two groups, and the matching degree is calculated with the corresponding strategy in the initial resource management set.

[0073] The output of the second resource management set needs to include complete parameter matching degree information for subsequent verification. The output format can be in table form or key-value pair structure, for example, each management content contains "strategy ID", "matching degree proportion", "parameter list" and other fields. When outputting, the consistency of the data needs to be ensured to avoid loss or error of parameter values due to format conversion.

[0074] Throughout the implementation process, data verification mechanisms need to be implemented throughout. When obtaining historical running data, the integrity check code (such as CRC check) of the data needs to be checked; when analyzing parameter characteristics, the value range of the parameters needs to be verified to meet the hardware specifications; when calculating the matching degree, the consistency of the comparison logic needs to be ensured. For example, the unit of memory bandwidth parameter must be unified to GB / s to avoid matching errors due to different units.

[0075] Example 3:

[0076] This embodiment describes step S3 in detail. The filtered historical running data refers to the historical running records with high parameter matching degree with the initial resource management set after step S2. These data contain the running state information of the device under a specific management strategy. When extracting the running time and resource occupation depth, the corresponding timestamp and resource occupation related indicators of each data point need to be parsed from these historical running data. The timestamp can be accurate to the millisecond level to ensure the accuracy of the time interval calculation; the resource occupation depth needs to be determined according to the specific hardware resource type, for example, for the computing core of GPU, the core utilization rate can be used to represent the resource occupation depth; for HBM memory, the memory bandwidth utilization rate or memory space occupation rate can be used to represent it.

[0077] When obtaining the running time intervals of the two adjacent historical running data, the data points need to be sorted in chronological order, and then the time difference of each two adjacent data points is calculated in turn. For example, assuming that there are three data points A, B, and C arranged in chronological order, and the time stamps are t1, t2, and t3 respectively, then the adjacent time intervals are t2-t1 and t3-t2. For the obtained several running time intervals, their average value needs to be calculated to obtain the average running time interval. The method of calculating the average value is to add all the time intervals and then divide by the number of time intervals. Similarly, for the resource occupation depth, the resource occupation depth values of multiple data points also need to be averaged to obtain the average resource depth.

[0078] The preset first time interval threshold and second time interval threshold are set according to the hardware performance of the GPU and HBM memory and the actual application scenario. The setting of these thresholds needs to consider the normal working frequency of the device and the requirement of data update frequency in different application scenarios. For example, in the high-performance computing scenario, a higher data update frequency may be required, so the first time interval threshold and the second time interval threshold are set to be smaller; while in some scenarios that do not require real-time performance, the thresholds can be set to be larger.

[0079] When comparing the average running time interval with the preset first time interval threshold and second time interval threshold, the following logic needs to be used for judgment: if the average running time interval is less than or equal to the first time interval threshold, it means that the data update between the GPU and the HBM memory is very frequent, so the collaborative update frequency can be judged to be high; if the average running time interval is greater than the first time interval threshold and less than the second time interval threshold, it means that the data update frequency is at a medium level, so the collaborative update frequency is judged to be medium; if the average running time interval is greater than or equal to the second time interval threshold, it means that the data update frequency is low, so the collaborative update frequency is judged to be low.

[0080] When obtaining the collaborative habit score value according to the collaborative running mode of the GPU and the HBM memory, a scoring model needs to be established. This model can set different scoring ranges according to different collaborative running modes. For example, when the collaborative update frequency is high, the score value can be set in a higher range, such as 80-100 points; when the collaborative update frequency is medium, the score value is set in the range of 50-80 points; when the collaborative update frequency is low, the score value is set in the range of 0-50 points. The specific scoring standard can be adjusted according to the actual situation, but the rationality and consistency of the score need to be ensured.

[0081] In actual implementation, the following points need to be noted: the selection of historical operation data must be accurate to ensure the reliability of the basis data for subsequent analysis; secondly, the recording of timestamps must be accurate to avoid errors in the calculation of time intervals due to time errors; thirdly, the selection of resource occupation depth indicators must be reasonable to accurately reflect the actual occupation of hardware resources; finally, the setting of thresholds needs to be tested and verified to adapt to different application scenarios.

[0082] The amount of data also needs to be considered. If the amount of historical operation data is too small, the calculated average running time interval and average resource depth may not be accurate enough, affecting the judgment of collaborative running mode and the acquisition of collaborative habit score value. Therefore, in actual application, it is necessary to ensure that there is enough historical operation data to support analysis.

[0083] The acquisition of collaborative habit score value can also consider more factors, such as the trend of resource occupation depth, the difference in running mode under different application scenarios, etc. However, in this embodiment, the collaborative running mode is mainly determined based on the average running time interval, and then the collaborative habit score value is obtained to simplify the processing flow.

[0084] In the data processing process, efficient data structures and algorithms need to be used to improve processing efficiency. For example, when storing historical operation data, a database table structure can be used to facilitate fast querying and sorting; when calculating the average value, a streaming calculation method can be used to avoid loading a large amount of data into memory at once.

[0085] The entire implementation process needs to be strictly tested and verified to ensure the accuracy and reliability of step S3. Testing can use a combination of simulation data and actual data to check whether the judgment of collaborative running mode is correct and whether the acquisition of collaborative habit score value is reasonable under different conditions.

[0086] Embodiment 4:

[0087] This embodiment describes step S4 in detail. The contents in the second resource management set are management strategies that have been screened in step S2 and have a high degree of match with the initial resource management set parameters. These contents contain specific strategies and configuration parameters for GPU and HBM memory fusion management. When obtaining the fusion features and resource depth of the contents in the second resource management set, relevant feature information needs to be parsed from each management strategy. Fusion features mainly reflect the characteristics of management strategies in promoting the collaborative work of GPU and HBM memory, such as whether it supports dynamic allocation of data, whether it has a cache prefetch mechanism, whether it can adaptively adjust memory bandwidth allocation, etc.; resource depth refers to the level and range of resource management involved in the management strategy, such as the scheduling granularity of computing cores, the minimum unit of memory allocation, the level of cache management, etc.

[0088] The cooperative habit score is obtained by step S3 according to the cooperative running mode of the GPU and the HBM memory, and is used to quantify the cooperative running habit of the device. The average resource depth is obtained by averaging the resource occupation depth of the historical running data in step S3, and reflects the average occupation degree of the device to the resource in the historical running process. When analyzing the fusion adaptability of the GPU and the HBM memory and the content in the second resource management set, the four factors need to be considered comprehensively: the cooperative habit score, the average resource depth, the fusion feature, and the resource depth.

[0089] The matching degree of the cooperative habit score and the content fusion feature is analyzed. Different cooperative habit scores correspond to different cooperative running modes of the device. For example, a high cooperative update frequency may mean that the device needs to frequently exchange data and schedule resources, so the fusion feature of the management strategy needs to support efficient data transmission and fast resource response. Specifically, when the cooperative habit score is high, the fusion feature of the management strategy should have a high-speed data transmission channel, a low-latency resource scheduling mechanism, etc.; when the cooperative habit score is low, the fusion feature of the management strategy can be relatively simplified to reduce resource consumption.

[0090] The matching degree of the average resource depth and the content resource depth is analyzed. The average resource depth reflects the resource occupation level of the device in the historical running, and the content resource depth represents the resource management level that the management strategy can handle. If the average resource depth is high, it means that the device often runs under high load, so the resource depth of the management strategy needs to be deep enough to support fine-grained resource scheduling and complex resource allocation; if the average resource depth is low, the resource depth of the management strategy can be correspondingly reduced to improve management efficiency.

[0091] In the specific implementation process, an evaluation system needs to be established for each index of the fusion feature and the resource depth. For the fusion feature, it can be decomposed into multiple specific indexes, such as data transmission rate, resource scheduling delay, adaptive adjustment capability, etc., and each index is assigned a different weight. According to the matching of the cooperative habit score and these indexes, a score is given. For example, the weight of the data transmission rate index is 30%, and if the index of the management strategy matches the requirement of the high cooperative habit score, a higher score is given.

[0092] For the resource depth, it can also be decomposed into indexes such as calculation core scheduling granularity, memory allocation unit, cache management level, etc., and a score is given according to the matching of the average resource depth and these indexes. For example, the weight of the calculation core scheduling granularity index is 25%, and if the average resource depth is high and the calculation core scheduling granularity of the management strategy is fine enough to meet the fine scheduling demand under high load, the index gets a higher score.

[0093] The scores of each indicator are weighted and summed according to the weights to obtain the matching total score of the synergy habit score value and the fusion feature, and the matching total score of the average resource depth and the resource depth. Then, the two total scores are added to obtain the fusion adaptation total score of each management policy.

[0094] The fusion adaptation total score needs to be sorted in descending order. During the sorting process, attention needs to be paid to maintaining the accuracy and consistency of the data to ensure that the score and sorting result of each management policy correctly reflect its fusion adaptation with the device. The sorting algorithm can use common efficient sorting algorithms such as quicksort or heapsort to improve processing efficiency.

[0095] According to the set serial number, the content in the corresponding serial number is filtered to form a third resource management set. The set serial number can be adjusted according to actual needs, such as filtering the top 10% or top 20 management policies with the highest fusion adaptation. The filtered content needs to be stored and output in a specific data structure to ensure that subsequent steps can be easily called and used.

[0096] In actual implementation, the following points need to be paid attention to: first, the index decomposition and weight setting of fusion features and resource depth need to be reasonably designed according to specific hardware architecture and application scenarios to ensure the scientificity and rationality of the evaluation system; second, the accuracy and consistency of the data need to be guaranteed during the scoring process to avoid distortion of the scoring results caused by human factors or data errors; third, the sorting and filtering process needs to be efficient and reliable to handle a large amount of management policy data; finally, the output format of the third resource management set needs to be compatible with the processing flow of subsequent steps to ensure the smooth implementation of the entire fusion management method.

[0097] The dynamic updating of management policies also needs to be considered. As the device operating environment and application scenarios change, the synergy running mode and resource demand of GPU and HBM memory may change, so the analysis of fusion adaptation and the filtering of the third resource management set also need to be updated regularly or in real time to ensure the effectiveness and adaptability of the management policies.

[0098] In terms of data processing, appropriate data storage and management techniques need to be used to ensure that the content in the second resource management set can be efficiently retrieved and processed. For example, database indexing techniques can be used to speed up the query and filtering of fusion features and resource depth, improving the efficiency of the entire implementation process.

[0099] The entire implementation process needs to be strictly tested and verified to ensure the accuracy and reliability of step S4. The test can include fusion adaptation analysis under different synergy running modes, matching degree evaluation under different average resource depths, etc., to check whether the filtering result of the third resource management set meets the expectations and can provide effective support for subsequent real-time optimization.

[0100] Embodiment 5:

[0101] This embodiment details step S5. When obtaining the real-time running feedback record of the GPU and the HBM memory, the system needs to continuously collect device running state data through the built-in monitoring module. The monitoring module can listen to the hardware registers, driver layer logs and application layer calling interfaces in real time, and the collected data includes but is not limited to the instruction execution sequence of the computing core, the read-write operation record of the memory controller, the data packet timing of the bus transmission, etc. The data collection frequency needs to be synchronized with the hardware clock period, for example, recording the running state once every 100 microseconds to ensure the time accuracy of the real-time feedback record.

[0102] When analyzing whether there is a contrast management habit between the GPU and the HBM memory, the determination standard of "simultaneous invocation of multiple resources" needs to be defined first. The system defines simultaneous invocation of multiple resources as: in the same time window (such as 500 microseconds), the GPU core initiates at least two different types of resource requests at the same time, such as simultaneously requesting read-write operation and cache data prefetch of the HBM memory. The type, timestamp and associated hardware unit of each resource invocation are marked in the real-time running feedback record, and the system traverses all records through the sliding time window algorithm to count the number of invocations that meet the conditions.

[0103] The preset contrast management habit threshold needs to be set according to the hardware architecture characteristics. For example, for a GPU system using multi-channel HBM2e, its parallel resource scheduling capability is strong, and the threshold can be set to 30% of the number of simultaneous invocations over the total number of running times; while the threshold of a single-channel HBM system can be set to 15%. The threshold is stored in the system configuration file and can be dynamically adjusted through the management interface.

[0104] When the number of simultaneous invocations exceeds the preset threshold, it is determined that there is a contrast management habit. At this time, the system needs to obtain the resource information in the set number from the third resource management set. The value of the set number needs to be combined with the number of management strategies and the hardware complexity, for example, when the third resource management set contains 50 strategies, the top 10 strategies with higher matching degree can be set to be obtained to cover the scene requirements of multi-resource collaborative management.

[0105] If there is no contrast management habit, the system directly obtains the resource information with the highest fusion matching degree in the third resource management set. The sorting of the fusion matching degree has been completed in step S4, and at this time the highest matching item can be quickly located through the index. For example, the third resource management set is stored in a priority queue, and the highest matching item is located at the head of the queue, which can be directly read.

[0106] In the process of optimizing the fusion management, the obtained resource information needs to be converted into specific hardware configuration instructions. For example, if the resource information is "memory bandwidth dynamic allocation strategy", the system needs to generate the corresponding register configuration value and send it to the HBM memory controller through the PCIe bus to adjust the bandwidth allocation ratio of each channel. The sending of optimization instructions needs to follow the hardware protocol specification, such as conforming to the configuration request format of PCIe4.0.

[0107] The storage of real-time running feedback records adopts a ring buffer structure, ensuring that the latest data covers the oldest data and maintaining a fixed time window of historical records (such as retaining the last 10 seconds of data). The size of the buffer needs to be calculated based on the data acquisition frequency and hardware bandwidth, for example, if 10,000 records are collected per second, a 10-second buffer needs to reserve 100KB of storage space.

[0108] The analysis algorithm of the comparison management habit needs to have real-time performance, and parallel computing framework can be used to speed up the statistics. For example, use the stream multi-processor of GPU to process records of different time windows in parallel, and distribute the total running time statistics and simultaneous call time statistics to different thread blocks, exchange intermediate results through shared memory, and reduce global memory access delay.

[0109] The logic of obtaining resource information needs to handle boundary conditions, for example, when the third resource management set is empty, the system needs to switch to the default management strategy and trigger an alarm to indicate the exception. The default strategy is the general configuration preset by the hardware manufacturer, such as evenly distributing memory bandwidth and computing core load, to ensure that the device can still run normally in abnormal situations.

[0110] The execution process of optimizing the fusion management needs to include an error checking mechanism. After sending each configuration instruction, the system waits for the confirmation signal returned by the hardware (such as ACK response packet), and if no confirmation is received within the preset timeout time (such as 500 microseconds), the instruction is resent and an error log is recorded. The error log includes the instruction content, sending time, and hardware response state, which is used for subsequent fault analysis.

[0111] Real-time monitoring module and resource management module communicate through message queue to ensure the reliability of data transmission. The message queue uses the producer-consumer mode, the monitoring module as the producer writes real-time feedback records into the queue, and the resource management module as the consumer reads data from the queue for analysis, and the queue capacity is dynamically adjusted to adapt to different load scenarios.

[0112] When the device running state changes abruptly (such as a sudden high-load computing task), the system needs to shorten the statistical period of the comparison management habit, for example, from the default 1 second to 200 milliseconds, to respond more quickly to hardware behavior changes. The adjustment of the statistical period is realized through an adaptive algorithm, when the resource occupancy rate fluctuation exceeds 20%, the period shortening mechanism is automatically triggered.

[0113] Data privacy and security are guaranteed. Real-time operation feedback records only contain hardware operation status data, and do not involve user application data. Data transmission in the memory adopts an encrypted channel (such as AES-128 encryption) to prevent external eavesdropping. The sending of configuration instructions needs to pass through hardware identity verification to ensure that only authorized management modules can modify the hardware configuration.

[0114] It should be noted that, in the present text, relational terms such as first and second are used merely to distinguish one entity or action from another, without necessarily requiring or implying that there is any such actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof are intended to cover non-exclusive inclusions, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0115] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements, and variations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A GPU and HBM memory fusion management method, characterized in that, The method comprises the following specific steps: S1, collecting historical running data of GPU and HBM memory, extracting hardware features, and constructing an initial resource management set; S2, analyzing the parameter matching of historical management content and the content in the initial resource management set according to the historical running data, and screening a second resource management set; S3, analyzing the cooperative running mode of GPU and HBM memory according to the historical running data; S4, analyzing the fusion adaptability of GPU and HBM memory and the content in the second resource management set according to the cooperative running mode of GPU and HBM memory, and screening a third resource management set; S5, obtaining corresponding resource information in the third resource management set according to real-time running feedback records of GPU and HBM memory, and optimizing fusion management in the running process; The S1 comprises the following specific steps: S101, inputting the historical running data of GPU and HBM memory in the running data collection module, and extracting hardware features in the historical running data, wherein the hardware features comprise hardware type features and resource demand features; S102, screening the content containing the hardware type features in the resource management module according to the hardware type features, and outputting the content in the form of an initial resource management set; The S2 comprises the following specific steps: S201, obtaining the historical running data of GPU and HBM memory, analyzing the features of historical management content according to the historical running data, wherein the features of the historical management content comprise resource identification features and parameter features, screening the historical management content containing the resource demand features according to the resource identification features and the resource demand features, and obtaining the parameter features of the content in the initial resource management set, wherein the resource identification features are identifiers for uniquely defining different management strategies; S202, analyzing the parameter matching degree of the screened historical management content and the content in the initial resource management set according to the parameter features of the screened historical management content and the parameter features of the content in the initial resource management set; S203, sorting the parameter matching degrees in descending order, screening the corresponding content in the set serial number, and outputting the content in the form of a second resource management set; The S3 comprises the following specific steps: S301, extracting the running time and resource occupation depth in the screened historical running data, obtaining the running time interval of adjacent two historical running data, taking the average of the obtained several running time intervals to obtain an average running time interval, taking the average of the obtained several resource occupation depths to obtain an average resource depth, and the screened historical running data refers to the historical running records with high parameter matching degree with the initial resource management set screened in step S2; S302, compare the average running time interval with the preset first time interval threshold and the second time interval threshold, and determine the cooperative running mode of the GPU and the HBM memory, if the average running time interval is less than or equal to the first time interval threshold, it is judged that the GPU and the HBM memory have high cooperative update frequency, if the average running time interval is greater than the first time interval threshold and the running time interval is less than the second time interval threshold, it is judged that the GPU and the HBM memory have medium cooperative update frequency, if the average running time interval is greater than or equal to the second time interval threshold, it is judged that the GPU and the HBM memory have low cooperative update frequency; S303, obtaining the cooperative habit score value according to the cooperative running mode of the GPU and the HBM memory; The S4 includes the following specific steps: S401, obtaining the fusion feature and resource depth of the content in the second resource management set, and analyzing the fusion adaptability of the GPU and the HBM memory and the content in the second resource management set according to the cooperative habit score value, the average resource depth, the fusion feature and the resource depth of the content in the second resource management set, the resource depth refers to the level and range of resource management involved in the management strategy; S402, sort the fusion adaptability in descending order, select the corresponding content in the set serial number, and output in the form of a third resource management set.

2. The GPU and HBM memory fusion management method of claim 1, wherein, The S5 includes the following specific steps: Obtain the real-time running feedback record of the GPU and the HBM memory to analyze whether the GPU and the HBM memory have comparative management habits, if the GPU and the HBM memory do not have comparative management habits, obtain the resource information with the highest fusion matching in the third resource management set, and optimize the fusion management in the running process, if the GPU and the HBM memory have comparative management habits, obtain the resource information corresponding to the set serial number in the third resource management set, and optimize the fusion management in the running process; The analysis method of whether the GPU and the HBM memory have comparative management habits is to count the proportion of the number of times that multiple resources are called simultaneously in the total running time in the real-time running feedback record of the GPU and the HBM memory, if the proportion exceeds the preset comparative management habit threshold, it is determined that the GPU and the HBM memory have comparative management habits. 3.The GPU and HBM memory fusion management method of claim 1, wherein, The hardware type features include specific hardware features of computing core, cache capacity and interface bandwidth, and the resource demand features include demand features of computing load, memory bandwidth and storage delay.

4. The GPU-HBM memory fusion management method of claim 1, wherein, The fusion adaptability analysis of the GPU and the HBM memory and the content in the second resource management set includes the matching degree of the cooperative habit score value and the content fusion feature, and the fitting degree of the average resource depth and the content resource depth.

5. The GPU and HBM memory fusion management method of claim 1, wherein, The analysis method of the parameter matching degree is to analyze the parameter features of the filtered historical management content and the parameter features of the content in the initial resource management set item by item, and count the proportion of the number of matching parameter items to the total number of parameter items.

6. A GPU and HBM memory fusion management system applied to the GPU and HBM memory fusion management method of any one of claims 1 to 5, characterized in that, It includes: A data acquisition and feature extraction module is used to collect historical running data of the GPU and the HBM memory, extract hardware features, and construct an initial resource management set; The historical data analysis and screening module is configured to analyze parameter matching between historical management content and content in the initial resource management set according to historical operation data, screen out a second resource management set, and analyze a cooperative operation mode of the GPU and the HBM memory. The fusion adaptation screening module is configured to analyze fusion adaptation of the GPU and the HBM memory and content in the second resource management set according to the cooperative operation mode of the GPU and the HBM memory, and screen out a third resource management set. The real-time optimization module is configured to obtain corresponding resource information in the third resource management set according to real-time operation feedback records of the GPU and the HBM memory, and optimize fusion management in a running process.

Images