Configurable GPU tile-type architecture based on deep learning, computing performance enhancement method and device

Abstract

The embodiment of the invention provides a GPU (Graphics Processing Unit) architecture, a configuration method, equipment and a storage medium, the architecture comprises at least one processing module, and the processing module comprises a computing resource and a storage resource; the configuration controller is connected with the at least one processing module and used for configuring target computing resources and target storage resources required for executing tasks for the at least one processing module; through the configuration controller, the target computing resource and the target storage resource can be accurately allocated to the processing module according to task requirements, and resource configuration can be dynamically adjusted according to task characteristics, so that the GPU architecture can adapt to diversified tasks, resource idling or overload can be avoided, and the overall utilization rate is improved; according to the method, the GPU architecture can quickly respond to various types of tasks, a hardware architecture does not need to be redesigned, and the flexibility and the expandability of the architecture are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a GPU architecture, configuration method, device and storage medium. Background Art

[0002] With the rapid development of deep learning technology, it has been widely used in many fields such as image recognition, natural language processing, and speech recognition. In existing technologies, deep learning tasks are generally processed through GPU architecture. However, when dealing with diverse deep learning tasks, although the traditional GPU architecture has powerful parallel computing capabilities, its computing resources and memory layout are pre-designed, which has certain limitations and low flexibility. Summary of the Invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a GPU architecture, configuration method, device, and storage medium that overcome the above problems or at least partially solve the above problems.

[0004] In order to solve the above problems, an embodiment of the present invention discloses a GPU architecture, which includes:

[0005] at least one processing module, the processing module comprising computing resources and storage resources;

[0006] A configuration controller is connected to the at least one processing module and is used to configure the target computing resources and target storage resources required for the at least one processing module to execute the task.

[0007] Optionally, the configuration controller is used to obtain computing power information required for the task; determine the target processing module in the at least one processing module based on the computing power information, and configure the target computing resources and target storage resources of the target processing module.

[0008] Optionally, the configuration controller is used to split the task into multiple subtasks; determine the target processing module corresponding to the subtask based on the computing power information required by the subtask, and configure the target computing resources and target storage resources of multiple target processing modules.

[0009] Optionally, the computing resource includes a computing unit; the storage resource includes a storage unit;

[0010] The configuration controller is used to configure a target computing unit and a target storage unit adapted to the computing power information required by the subtask for the target processing module corresponding to the subtask.

[0011] Optionally, the task includes a model processing task, and the subtask includes various processing steps involved in the model processing task.

[0012] Optionally, the configuration controller is further used to determine the data caching strategy of the target processing module based on the computing power information required for the task, and control the target processing module to cache the data involved in the task according to the data caching strategy.

[0013] Optionally, the configuration controller is further configured to monitor the load status of the at least one processing module, and when the load of the target processing module is too high, select a processing module from the remaining processing modules to replace the target processing module.

[0014] Optionally, the task includes one of a matrix multiplication optimization algorithm task and a convolution operation algorithm task.

[0015] The present invention also discloses a configuration method based on a GPU architecture, wherein the method applies the above-mentioned GPU architecture and comprises:

[0016] The at least one processing module is configured with target computing resources and target storage resources required to execute the task.

[0017] Optionally, configuring target computing resources and target storage resources required for executing the task for the at least one processing module includes:

[0018] Obtaining computing power information required for the task;

[0019] According to the computing power information, a target processing module in the at least one processing module is determined, and target computing resources and target storage resources of the target processing module are configured.

[0020] Optionally, determining a target processing module in the at least one processing module based on the computing power information, and configuring target computing resources and target storage resources for the target processing module includes:

[0021] Splitting the task into multiple subtasks;

[0022] According to the computing power information required by the subtask, the target processing module corresponding to the subtask is determined, and target computing resources and target storage resources of multiple target processing modules are configured.

[0023] Optionally, the computing resource includes a computing unit; the storage resource includes a storage unit;

[0024] The configuring of target computing resources and target storage resources of the plurality of target processing modules includes:

[0025] For the target processing module corresponding to the subtask, a target computing unit and a target storage unit adapted to the computing power information required by the subtask are configured.

[0026] Optionally, the task includes a model processing task, and the subtask includes various processing steps involved in the model processing task.

[0027] The present invention also discloses an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the configuration method based on the GPU architecture as described above.

[0028] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the configuration method based on the GPU architecture are implemented.

[0029] The embodiments of the present invention include the following advantages:

[0030] The present invention discloses a GPU architecture, configuration method, device and storage medium. The present invention can accurately allocate target computing resources and target storage resources to processing modules according to task requirements through a configuration controller, and can dynamically adjust resource configuration according to task characteristics, so that the GPU architecture can adapt to diverse tasks, avoid idle or overloaded resources, and improve overall utilization. The GPU architecture can be enabled to respond quickly to multiple types of tasks without redesigning the hardware architecture, thereby improving the flexibility and scalability of the architecture. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a structural block diagram of a GPU architecture provided by an embodiment of the present invention;

[0032] Figure 2 This is a structural block diagram of another GPU architecture provided by an embodiment of the present invention;

[0033] Figure 3 This is a structural block diagram of a processing module provided by an embodiment of the present invention;

[0034] Figure 4 This is a flowchart of a task processing method provided by an embodiment of the present invention;

[0035] Figure 5 This is a flowchart of the steps of a GPU configuration method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0037] One of the core concepts of the embodiments of the present invention is that the present invention can accurately allocate target computing resources and target storage resources to the processing module according to task requirements by configuring the controller, and can dynamically adjust resource configuration according to task characteristics, so that the GPU architecture can adapt to diverse tasks, avoid idle or overloaded resources, and improve overall utilization; it can enable the GPU architecture to respond quickly to multiple types of tasks without redesigning the hardware architecture, thereby improving the flexibility and scalability of the architecture.

[0038] Reference Figure 1 , shows a block diagram of a structure 10 of a GPU architecture provided by an embodiment of the present invention, which may include:

[0039] At least one processing module 101, the processing module including computing resources and storage resources;

[0040] The configuration controller 102 is connected to the at least one processing module and is used to configure the target computing resources and target storage resources required for the at least one processing module to execute a task.

[0041] In the embodiment of the present invention, GPU architecture: GPU is a graphics processing unit, and GPU architecture refers to the hardware organization and logical structure inside the GPU, which determines how the GPU processes data, performs computing tasks, and how the various components work together.

[0042] The processing module 101 is a functional unit in the GPU architecture. It contains computing resources and storage resources. The computing resources are used to perform various computing tasks, such as floating-point operations and integer operations; the storage resources are used to store data, including input data, intermediate calculation results, and final output results. Figure 1 The processing module 101 may include a processing module 1011, a processing module 1012, a processing module 1013, a processing module 1014, a processing module 1015, a processing module 1016, a processing module 1017... a processing module 101n.

[0043] The configuration controller 102 is a component responsible for managing and allocating processing module resources. It can reasonably configure the computing resources and storage resources in the processing module according to the requirements of different tasks.

[0044] Target computing resources: refers to the computing power required by the processing module to complete a specific task, such as the number of computing cores required to participate in the operation and the operating frequency of each computing core.

[0045] Target storage resources: refers to the storage space required by the processing module to complete a specific task, which is used to store the data involved in the task execution process.

[0046] The configuration controller 102 can first obtain relevant information about the task to be performed, including the type and complexity of the task and the approximate requirements for computing and storage resources. For example, if it is a large-scale deep learning model training task, a large amount of computing resources and storage resources are required to store model parameters and intermediate calculation results.

[0047] Furthermore, based on the acquired task requirements, the configuration controller 102 can evaluate the existing computing resources and storage resources of at least one processing module. It will check the computing power, number of idle computing cores, available storage capacity and other information of each processing module. For example, some processing modules may have some resources occupied by other tasks, and the configuration controller needs to accurately understand these situations.

[0048] Based on the task requirements and the resource evaluation results of the processing module, the configuration controller 102 can allocate target computing resources and target storage resources to the processing module. For example, for a task with high computing requirements, the configuration controller 102 can allocate more computing cores to the processing module; for tasks that need to store a large amount of data, a larger storage area will be allocated.

[0049] After obtaining the configured target computing resources and target storage resources, the processing module 101 starts to execute the task. During the task execution process, the processing module 101 uses the allocated computing resources to perform data calculations and uses the allocated storage resources to store related data.

[0050] It should be noted that the GPU architecture in the present invention is equivalent to a configurable tile architecture. Each processing module in the GPU architecture is equivalent to a tile. Through the configurable tile architecture, the computing resource allocation of each tile can be flexibly adjusted according to different deep learning tasks, so that the computing resources of the GPU can be fully utilized, avoiding the problem of some processing modules being idle in the traditional GPU architecture.

[0051] The present invention discloses a GPU architecture. By configuring a controller, the present invention can accurately allocate target computing resources and target storage resources to a processing module according to task requirements. The present invention can dynamically adjust resource allocation according to task characteristics, so that the GPU architecture can adapt to diverse tasks, avoid idle or overloaded resources, and improve overall utilization. The present invention can enable the GPU architecture to respond quickly to multiple types of tasks without redesigning the hardware architecture, thereby improving the flexibility and scalability of the architecture.

[0052] In one embodiment of the present invention, a controller is configured to obtain computing power information required for a task; based on the computing power information, a target processing module in at least one processing module is determined, and target computing resources and target storage resources of the target processing module are configured.

[0053] In the embodiment of the present invention, the computing power information includes information such as the type of task, the algorithm used, and the data size. The configuration controller 102 can extract parameters related to the computing power from the computing power information.

[0054] In order to improve the processing efficiency of tasks, the configuration controller can use multiple processing modules to process tasks. Specifically, the configuration controller 102 can match and evaluate the computing power information required for the task with the current status of each processing module. The evaluation indicators include computing power matching, storage capacity matching and load balancing. Based on the results of the matching evaluation, the configuration controller 102 can select the target processing module that is most suitable for executing the task from at least one processing module.

[0055] For example, for a deep learning model training task, the configuration controller will estimate computing power indicators such as the number of floating-point operations and memory read and write times required during the training process based on parameters such as the number of model layers, the number of neurons in each layer, and the dimension of the input data, and then determine the required target processing module among multiple processing modules.

[0056] After determining multiple target processing modules, the configuration controller 102 can configure the target computing resources and target storage resources of each target processing module respectively. The configuration controller can allocate a corresponding number of computing cores and storage resources to the target processing modules. For example, if the task requires a large amount of parallel computing, the configuration controller will allocate multiple computing cores to work simultaneously.

[0057] like Figure 2 , showing a structural block diagram of a GPU architecture provided by an embodiment of the present invention, the GPU architecture may include multiple processing modules RT, U2U interfaces, C2V interfaces, the U2U interfaces can transmit data in the same GPU cluster, the C2V interfaces can transmit data in different GPU clusters, the storage module DDR can store data, and when a task is obtained, the corresponding target processing module RT can be determined according to the computing power information of the task.

[0058] By accurately allocating computing and storage resources according to task requirements, this invention avoids the problem of idle or insufficient resources that may be caused by fixed resource allocation in traditional GPU architecture. This architecture can flexibly respond to tasks of different types and complexities. Whether it is a simple image recognition task or a complex deep learning model training task, it can be efficiently completed by rationally allocating resources, thereby improving the adaptability of the GPU architecture to diverse tasks.

[0059] In one embodiment of the present invention, a controller is configured to split a task into multiple subtasks; based on the computing power information required by the subtasks, the target processing modules corresponding to the subtasks are determined, and the target computing resources and target storage resources of the multiple target processing modules are configured.

[0060] In the embodiment of the present invention, in order to more finely manage and allocate computing resources and improve the execution efficiency of the entire task, the configuration controller 102 can decompose a complex task into multiple relatively simple and interrelated subtasks.

[0061] Suppose you want to process a large-scale image recognition task, which includes operations such as feature extraction and classification of a large number of images. The configuration controller can split it into multiple subtasks. For example, divide the image set into several subsets and process each subset as a subtask, or separate feature extraction and classification into different subtasks.

[0062] For each subtask, the configuration controller can evaluate the current status of each processing module based on the required computing power information, such as computing power and memory requirements, including the idle status of computing resources and the remaining space of storage resources, and then select the most suitable target processing module for each subtask.

[0063] In the above-mentioned image recognition task, the feature extraction subtask may require a large amount of computing resources to perform complex convolution operations, while the classification subtask may have high requirements for storage resources for storing classification models and intermediate results. The configuration controller can assign the feature extraction subtask to the processing module with abundant and idle computing resources, and assign the classification subtask to the processing module with sufficient storage resources based on these different requirements.

[0064] After determining the target processing modules for each subtask, the configuration controller allocates specific computing and storage resources to these target processing modules. This includes allocating the number of computing cores, adjusting the computing frequency to meet the computing requirements of the subtask, allocating sufficient video memory space, and properly managing cache resources to meet the data storage and access requirements of the subtask.

[0065] For the processing module responsible for feature extraction, the configuration controller can allocate more computing cores and increase the computing frequency to speed up the convolution operation; at the same time, allocate an appropriate amount of video memory to store image data and intermediate feature maps; for the processing module responsible for classification, it will ensure that sufficient video memory is allocated to store the classification model and the data to be classified, and reasonably configure cache resources according to the characteristics of the classification algorithm to improve data access efficiency.

[0066] The present invention splits tasks and accurately allocates resources according to the characteristics of subtasks, so that each subtask can be efficiently executed on the most suitable processing module, reducing resource waiting and waste, thereby improving the execution speed of the entire task.

[0067] In one embodiment of the present invention, computing resources include computing units; storage resources include storage units; and a configuration controller is used to configure a target computing unit and a target storage unit for adapting computing power information required for the subtask to the target processing module corresponding to the subtask.

[0068] In an embodiment of the present invention, it is assumed that there is a complex deep learning model training task, such as training a generative adversarial network (GAN) model for image generation. This task can be divided into multiple subtasks, such as the forward propagation of the generator network, the forward propagation of the discriminator network, the backpropagation of the generator and discriminator, and the parameter update subtasks.

[0069] For the forward propagation subtask of the generator network, it mainly involves a large number of matrix multiplications and activation function calculations, which require high floating-point computing capabilities. Assume that after analysis, this subtask needs to perform 10 trillion floating-point operations per second (TFLOPS).

[0070] The forward propagation subtask of the discriminator network also requires a lot of calculations. It may need to frequently read image data in the training dataset. Assume that this subtask requires 5TFLOPS of computing power.

[0071] The backpropagation subtask involves gradient calculation, which requires precise calculation and a certain amount of storage to save the intermediate gradient values. Assume that the computing power requirement is 8 TFLOPS.

[0072] The parameter update subtask requires updating the parameters of the model, and the computing power requirement is relatively low, assuming 2 TFLOPS.

[0073] Assume that there are 12 processing modules in the GPU architecture, namely processing module 1, processing module 2, processing module 3, processing module 4, processing module 5, processing module 6, processing module 7, processing module 8, processing module 9, processing module 10, processing module 11, and processing module 12, and the maximum computing power system of each processing module is 3 TFLOPS. Processing module 1, processing module 2, processing module 3, and processing module 4 can be used as the target processing modules corresponding to the forward propagation subtask of the generator network, processing module 5 and processing module 6 can be used as the target processing modules for the forward propagation subtask of the discriminator network, processing module 7, processing module 8, and processing module 9 can be used as the target processing modules for the backpropagation subtask, and processing module 10 can be used as the target processing module for the parameter update subtask.

[0074] For the forward propagation subtask of the generator network, it is known that this subtask is the responsibility of processing modules 1-4, and the maximum computing power of each processing module is 3TFLOPS. Due to the high computing requirements of this subtask, the computing resources of these processing modules can be fully utilized as much as possible. For example, computing units corresponding to approximately 2.5TFLOPS computing power are allocated from each processing module, and these computing units are combined to meet the computing power requirements of the subtask. In actual operation, a specific number of computing units PE, tensor cores, etc. within the processing module may be selected. For example, there are 100 computing units PE in each processing module. According to the corresponding relationship of computing power, 80 computing units PE may be selected from each processing module to participate in the calculation.

[0075] A large amount of intermediate data is generated during the forward propagation of the generator, such as feature maps after passing through the convolution layer and activation layer, which require a large storage space. Assume that this subtask requires 5GB of storage to store the intermediate results. Processing modules 1-4 have their own storage units, which can be allocated according to certain rules. For example, based on the storage performance and idle status of the processing modules, processing modules 1 to 4 are allocated 1GB, 1GB, 1.5GB, and 1.5GB of storage units as target storage units, respectively.

[0076] For the forward propagation subtask of the discriminator network, this subtask is the responsibility of processing modules 5 and 6. The computational amount of the discriminator forward propagation is relatively small. Assuming that 4TFLOPS of computing power is required, computing units can be allocated from processing modules 5 and 6. For example, computing units corresponding to 2.2TFLOPS computing power are allocated from processing module 5, and computing units corresponding to 1.8TFLOPS computing power are allocated from processing module 6. Specifically, there are 100 computing units PE in processing module 5, and 73 of them are allocated as target computing units; there are 100 computing units PE in processing module 6, and 60 of them are allocated as target computing units.

[0077] Intermediate data is also generated during the forward propagation of the discriminator, but it is less than that of the generator. Assuming that 2GB of storage is required, 1GB of storage units can be allocated to processing modules 5 and 6 as target storage units respectively.

[0078] For the back propagation subtask, this subtask is the responsibility of processing modules 7-9. Back propagation requires gradient calculation, which has high requirements on calculation accuracy and computing power. Assuming that 8TFLOPS of computing power is required, computing units can be reasonably allocated from processing modules 7-9. For example, processing modules 7, 8, and 9 are respectively allocated computing units corresponding to approximately 3TFLOPS, 3TFLOPS, and 2TFLOPS computing capabilities. There are 100 computing units PE in processing module 7, and 100 of them are allocated as target computing units; there are 100 computing units PE in processing module 8, and 100 of them are allocated as target computing units; there are 100 computing units PE in processing module 9, and 66 of them are allocated as target computing units.

[0079] Backpropagation requires storage of intermediate results such as gradient information. Assume 3GB of storage is required. This can be allocated based on the storage availability of the processing modules. For example, processing modules 7, 8, and 9 can be allocated 1GB, 1GB, and 1GB of storage units as target storage units, respectively.

[0080] Regarding the target computing unit allocation, the subtask is the responsibility of the processing module 10, and computing units with corresponding computing capabilities can be allocated from the processing module 10. For example, there are 100 computing units PE in the processing module 10, and 66 computing units PE are allocated to participate in the calculation.

[0081] Parameter update mainly updates model parameters, which requires less storage. Assuming that 0.5 GB of storage is required, the 0.5 GB storage unit in the processing module 10 is used as the target storage unit.

[0082] like Figure 3 A structural block diagram of a processing module provided by an embodiment of the present invention is shown. The processing module may include multiple computing units PE, and a target computing unit PE for an adaptation subtask may be determined from the multiple computing units.

[0083] In the present invention, when the type or scale of a task changes, the configuration controller can reconfigure the target computing unit and target storage unit based on the new subtask computing power information. Different subtasks run independently under their respective appropriate processing modules and resource configurations, reducing computing waiting time and data transmission delay, and improving task execution efficiency.

[0084] In one embodiment of the present invention, the task includes a model processing task, and the subtask includes various processing steps involved in the model processing task.

[0085] In the embodiments of the present invention, a model processing task refers to a process of performing a series of operations and processing on a machine learning or deep learning model to achieve a specific goal. Subtasks refer to the individual steps in a model processing task. For example, when the task is a deep learning model processing task for image classification, the subtasks may be:

[0086] Data preprocessing subtask: including reading image data, adjusting image size, normalization and other operations.

[0087] Model forward propagation subtask: Input the preprocessed image data into the convolutional neural network (CNN), and obtain the classification result after calculations such as the convolution layer, pooling layer, and fully connected layer.

[0088] Loss function calculation subtask: Calculate the loss function value such as cross entropy loss based on the model's classification results and true labels.

[0089] Backpropagation subtask: Based on the value of the loss function, the gradient is calculated through the backpropagation algorithm to update the parameters of the model.

[0090] Parameter update subtask: Use the calculated gradient to update the weight and bias parameters of the model.

[0091] In one embodiment of the present invention, the controller is configured to determine the data caching strategy of the target processing module based on the computing power information required for the task, and control the target processing module to cache the data involved in the task according to the data caching strategy.

[0092] In an embodiment of the present invention, the configuration controller 102 can determine the data caching strategy based on the computing power information required for the task. First, the configuration controller will obtain the computing power parameters required for the task during execution, such as computational complexity, data throughput, processing time, etc. These computing power information reflects the degree of demand for computing resources of the task.

[0093] The data caching strategy includes the type of cached data (such as original data, intermediate calculation results, etc.), cache location (such as memory cache, disk cache, etc.), cache capacity and cache replacement algorithm. Based on this computing power information, the configuration controller will determine the cache strategy that is adapted to the computing power information to ensure that during the task execution, the data can be quickly and accurately acquired by the processing module, reducing data reading delays and improving overall processing efficiency.

[0094] After determining the data caching strategy, the configuration controller can control the target processing module to cache the data involved in the task according to the strategy.

[0095] In one example, the task is an AI image rendering task scenario. When performing high-precision 3D animation rendering, the rendering task needs to process massive image data, which requires extremely high computing power. The configuration controller can obtain the computing power information of the rendering task, and know that it needs to frequently read model texture data, lighting parameters, etc., and the data processing speed is required to be fast. Based on this, the configuration controller determines to cache commonly used texture data and lighting parameters in high-speed memory, and adopts the LRU cache replacement algorithm to give priority to retaining the most recently used data. In this way, during the rendering process, the rendering engine can quickly obtain data from the memory cache, greatly shortening the rendering time and improving rendering efficiency.

[0096] The present invention reduces data reading time through a reasonable data caching strategy, enables the processing module to obtain data for processing more quickly, thereby accelerating the overall execution speed of the task and shortening the task completion time.

[0097] In one embodiment of the present invention, the configuration controller is further configured to monitor the load status of the at least one processing module, and when the load of the target processing module is too high, a processing module is selected from the remaining processing modules to replace the target processing module.

[0098] In an embodiment of the present invention, the configuration controller can monitor the load status of the processing module and dynamically replace it, which is an important mechanism for maintaining efficient operation of the system. First, the configuration controller will collect load data of at least one processing module in real time. These data include CPU usage, memory occupancy, task queue length and other indicators, so as to judge the workload of each processing module.

[0099] When it is detected that the target processing module is overloaded, it means that the module is under too much working pressure, which may cause task processing delays or even system crashes. At this time, the configuration controller will start the replacement mechanism to screen out a suitable processing module from the remaining processing modules to replace the target processing module. The screening process will comprehensively consider factors such as the current load, processing capacity, and adaptability to the task of the remaining processing modules to ensure that the selected processing module can efficiently undertake the tasks of the target processing module, achieve smooth transition and processing of tasks, and avoid system performance degradation due to excessive load on a certain module.

[0100] Assume that in a deep learning model training task, there are 10 processing modules (processing modules 1-10) in the GPU architecture. Currently, the configuration controller has determined processing module 3 as the target processing module for executing the model forward propagation subtask.

[0101] The configuration controller continuously monitors the load of these 10 processing modules. During the training process, it found that the computing core utilization of processing module 3 reached 90%, the video memory utilization was also close to saturation, and there were multiple tasks waiting to be processed in the task queue. Based on the preset load threshold (computing core utilization exceeded 80%, and video memory utilization exceeded 70%), it was judged that the load of processing module 3 was too high.

[0102] The configuration controller checks the load status of the remaining nine processing modules (processing modules 1, 2, 4-10) and finds that the current computing core utilization of processing module 7 is only 30%, the video memory utilization is 40%, and its computing power is comparable to that of processing module 3. At the same time, it also has good support for the computing type of the forward propagation subtask of the model. Therefore, the configuration controller decides to select processing module 7 to replace processing module 3.

[0103] The configuration controller migrates the unfinished forward propagation task-related data and computing status on processing module 3 to processing module 7, and allocates computing resources and storage resources suitable for the task to processing module 7. Then, processing module 7 starts to continue executing the model forward propagation subtask.

[0104] The present invention avoids the situation where the performance of the entire system is degraded or malfunctions due to the overload of a single processing module by timely replacing the processing module with an overload. The new processing module can perform tasks in a better state, ensuring the smooth progress of the tasks and improving the stability and reliability of the system.

[0105] In one embodiment of the present invention, the task includes one of a matrix multiplication optimization algorithm task and a convolution operation algorithm task.

[0106] In an embodiment of the present invention, matrix multiplication is a basic operation in linear algebra. When the task is a matrix multiplication optimization algorithm task, the large matrix in the matrix multiplication can be decomposed into multiple small matrix blocks. Each processing module is responsible for calculating the product of one or more small matrix blocks, and then the final result is obtained through communication and data merging between processing modules. In this way, the parallel computing capability of the GPU architecture can be fully utilized to reduce the computing time.

[0107] For convolution operation algorithm tasks, the convolution kernel can be divided into multiple sub-convolution kernels according to the computing resources of the computing module and the size of the convolution kernel. Each computing module is responsible for processing the convolution operation of a sub-convolution kernel. At the same time, the local storage and shared storage mechanism of the tile is used to reduce the repeated transmission of data during the convolution operation and improve computing efficiency.

[0108] like Figure 4, shows a flow chart of a task processing provided by an embodiment of the present invention, which is a convolution operation algorithm task. The convolution operation algorithm task may include a first convolution layer subtask Conv1, a pooling layer subtask Pool, a second convolution layer subtask Conv2, and a fully connected layer subtask FC. The processing module RT in the first convolution layer subtask processes the data Data4, and then sends Data4 to the processing module RT corresponding to the Pool layer subtask for processing to obtain Data3, and then sends Data3 to the processing module RT corresponding to the second convolution layer subtask Conv2 for processing to obtain Data2, and then sends Data3 to the processing module RT corresponding to the fully connected layer subtask FC for processing to obtain Data1, and finally outputs Data1.

[0109] The present invention discloses a GPU architecture. By configuring a controller, the present invention can accurately allocate target computing resources and target storage resources to a processing module according to task requirements. The present invention can dynamically adjust resource allocation according to task characteristics, so that the GPU architecture can adapt to diverse tasks, avoid idle or overloaded resources, and improve overall utilization. The present invention can enable the GPU architecture to respond quickly to multiple types of tasks without redesigning the hardware architecture, thereby improving the flexibility and scalability of the architecture.

[0110] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0111] Reference Figure 5 , shows a flowchart of the steps of a configuration method based on a GPU architecture provided by an embodiment of the present invention. The method applies the above-mentioned GPU architecture and includes:

[0112] Step 201: configure target computing resources and target storage resources required for executing a task for at least one processing module.

[0113] It should be noted that the configuration method of the GPU architecture in the present invention can optimize and enhance the computing performance of different algorithms, so that the GPU of the present invention can adapt to different types of deep learning tasks, such as convolutional neural networks, recurrent neural networks, etc., and has greater versatility and flexibility.

[0114] When the algorithm is matrix multiplication optimization, the matrix multiplication optimization algorithm is implemented in the computing unit of each tile. First, the input large matrix is ​​decomposed into multiple small matrix blocks according to certain rules, and these small matrix blocks are distributed to each tile. Each tile uses the corresponding computing core to calculate the product of the small matrix blocks according to its own configuration information. Finally, the calculation results are merged through the communication interface between tiles to obtain the final matrix product.

[0115] When the algorithm is a convolution operation optimization algorithm, the convolution operation optimization algorithm can be implemented in the computing unit of the tile. According to the size of the convolution kernel and the computing resources of the tile, the convolution kernel is divided into multiple sub-convolution kernels. Each tile is responsible for processing the convolution operation of a sub-convolution kernel. The local storage is used to store the intermediate calculation results to reduce repeated data transmission. After completing the calculation of all sub-convolution kernels, the results are merged to obtain the final convolution result.

[0116] The present invention discloses a configuration method for a GPU architecture. By configuring a controller, the present invention can accurately allocate target computing resources and target storage resources to a processing module according to task requirements. The present invention can dynamically adjust resource configuration according to task characteristics, so that the GPU architecture can adapt to diverse tasks, avoid idle or overloaded resources, and improve overall utilization. The present invention can enable the GPU architecture to respond quickly to multiple types of tasks without redesigning the hardware architecture, thereby improving the flexibility and scalability of the architecture.

[0117] In one embodiment of the present invention, configuring target computing resources and target storage resources required for executing a task for at least one processing module includes:

[0118] Obtain computing power information required for the task;

[0119] According to the computing power information, a target processing module in at least one processing module is determined, and target computing resources and target storage resources of the target processing module are configured.

[0120] In one embodiment of the present invention, determining a target processing module in at least one processing module based on computing power information and configuring target computing resources and target storage resources for the target processing module include:

[0121] Split the task into multiple subtasks;

[0122] Based on the computing power information required by the subtask, determine the target processing module corresponding to the subtask, and configure the target computing resources and target storage resources of multiple target processing modules.

[0123] In one embodiment of the present invention, the computing resource includes a computing unit; the storage resource includes a storage unit;

[0124] Configure target computing resources and target storage resources for multiple target processing modules, including:

[0125] For the target processing module corresponding to the subtask, configure the target computing unit and target storage unit that adapt to the computing power information required by the subtask.

[0126] In one embodiment of the present invention, the task includes a model processing task, and the subtask includes various processing steps involved in the model processing task.

[0127] In one embodiment of the present invention, the method further includes: determining a data caching strategy of a target processing module according to computing power information required for the task, and controlling the target processing module to cache data involved in the task according to the data caching strategy.

[0128] In one embodiment of the present invention, the method further includes: monitoring the load status of at least one processing module, and when the load of a target processing module is too high, selecting a processing module from the remaining processing modules to replace the target processing module.

[0129] In one embodiment of the present invention, the task includes one of a matrix multiplication optimization algorithm task and a convolution operation algorithm task.

[0130] The present invention discloses a configuration method for a GPU architecture. By configuring a controller, the present invention can accurately allocate target computing resources and target storage resources to a processing module according to task requirements. The present invention can dynamically adjust resource configuration according to task characteristics, so that the GPU architecture can adapt to diverse tasks, avoid idle or overloaded resources, and improve overall utilization. The present invention can enable the GPU architecture to respond quickly to multiple types of tasks without redesigning the hardware architecture, thereby improving the flexibility and scalability of the architecture.

[0131] It should be noted that for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0132] An embodiment of the present invention further provides an electronic device, including:

[0133] The present invention includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various processes of the configuration method embodiment of the above-mentioned GPU architecture are implemented, and the same technical effects can be achieved. To avoid repetition, they are not described here.

[0134] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned GPU architecture configuration method embodiment are implemented, and the same technical effects can be achieved. To avoid repetition, they will not be described here.

[0135] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0136] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatus, or computer program products. Thus, embodiments of the present invention may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0137] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0138] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0139] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0140] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0141] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0142] The above describes in detail a GPU architecture, a configuration method, a device, and a medium for the GPU architecture provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only intended to help understand the method and core concept of the present invention. At the same time, for those skilled in the art, according to the concept of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. A GPU architecture, characterized in that: The architecture includes: at least one processing module, the processing module comprising computing resources and storage resources; A configuration controller is connected to the at least one processing module and is used to configure the target computing resources and target storage resources required for the at least one processing module to execute the task.

2. The architecture according to claim 1, wherein: The configuration controller is used to obtain the computing power information required for the task; determine the target processing module in the at least one processing module based on the computing power information, and configure the target computing resources and target storage resources of the target processing module.

3. The architecture according to claim 2, characterized in that The configuration controller is used to split the task into multiple subtasks; determine the target processing module corresponding to the subtask based on the computing power information required by the subtask, and configure the target computing resources and target storage resources of multiple target processing modules.

4. The architecture according to claim 3, characterized in that The computing resource includes a computing unit; the storage resource includes a storage unit; The configuration controller is used to configure a target computing unit and a target storage unit adapted to the computing power information required by the subtask for the target processing module corresponding to the subtask.

5. The architecture according to claim 3, wherein: The task includes a model processing task, and the subtask includes various processing steps involved in the model processing task.

6. The architecture according to claim 2, wherein: The configuration controller is further configured to determine a data caching strategy for the target processing module based on computing power information required for the task, and to control the target processing module to cache data involved in the task according to the data caching strategy.

7. The architecture according to claim 2, wherein: The configuration controller is further configured to monitor the load status of the at least one processing module, and when the load of the target processing module is too high, select a processing module from the remaining processing modules to replace the target processing module.

8. The architecture according to claim 1, wherein: The task includes one of a matrix multiplication optimization algorithm task and a convolution operation algorithm task.

9. A configuration method based on a GPU architecture, the method applying the GPU architecture according to any one of claims 1 to 8, the method comprising: The at least one processing module is configured with target computing resources and target storage resources required to execute the task.

10. The configuration method based on GPU architecture according to claim 9, characterized in that: The configuring the target computing resources and target storage resources required for the at least one processing module to execute the task includes: Obtaining computing power information required for the task; According to the computing power information, a target processing module in the at least one processing module is determined, and target computing resources and target storage resources of the target processing module are configured.

11. The configuration method based on GPU architecture according to claim 10, characterized in that: The determining, based on the computing power information, a target processing module in the at least one processing module, and configuring target computing resources and target storage resources for the target processing module includes: Splitting the task into multiple subtasks; According to the computing power information required by the subtask, the target processing module corresponding to the subtask is determined, and target computing resources and target storage resources of multiple target processing modules are configured.

12. The configuration method based on GPU architecture according to claim 11, characterized in that: The computing resource includes a computing unit; the storage resource includes a storage unit; The configuring of target computing resources and target storage resources of the plurality of target processing modules includes: For the target processing module corresponding to the subtask, a target computing unit and a target storage unit adapted to the computing power information required by the subtask are configured.

13. The configuration method based on GPU architecture according to claim 12, characterized in that: The task includes a model processing task, and the subtask includes various processing steps involved in the model processing task.

14. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the configuration method based on the GPU architecture as described in any one of claims 9 to 13 are implemented.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the configuration method based on the GPU architecture as described in any one of claims 9 to 13 are implemented.

Images