Matrix computation optimization method and device, electronic equipment and storage medium

Abstract

The application provides a matrix calculation optimization method and device, electronic equipment and storage medium, and belongs to the technical field of data processing. The method comprises the following steps: determining the data reading amount of each matrix calculation task to be processed; grouping a plurality of calculation cores based on the physical distance between each calculation core and a plurality of high-bandwidth memories, to obtain a plurality of calculation core groups; matching the plurality of matrix calculation tasks to be processed with the plurality of calculation core groups based on the data reading amount of each matrix calculation task, to obtain a task allocation scheme; and allocating each matrix calculation task to a corresponding calculation core group based on the task allocation scheme. The matrix calculation optimization method and device, electronic equipment and storage medium provided by the application can effectively avoid performance bottlenecks and significantly improve the overall operation efficiency and hardware resource utilization of a plurality of matrix calculation tasks by comprehensively considering the data reading amount of the matrix calculation tasks and the physical layout characteristics of the calculation cores.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a matrix calculation optimization method, apparatus, electronic device, and storage medium. Background Technology

[0002] In artificial intelligence, deep learning, and other high-performance computing fields, matrix computation, especially matrix multiplication, is one of the most prevalent and core computational tasks. Currently, existing matrix computation tasks are typically allocated statically or randomly, resulting in low hardware resource utilization and an inability to achieve optimal computational performance. Therefore, improving system computational performance and hardware resource utilization is a critical issue that urgently needs to be addressed in matrix computation optimization. Summary of the Invention

[0003] This invention provides a matrix computation optimization method, apparatus, electronic device, and storage medium to address the shortcomings of existing technologies that use static or random allocation methods to allocate matrix computation tasks, resulting in low computational performance and low hardware resource utilization.

[0004] This invention provides a matrix computation optimization method applied to an artificial intelligence chip, the artificial intelligence chip including multiple computing cores and multiple high-bandwidth memories, the method comprising:

[0005] Determine the amount of data to be read for each matrix calculation task to be processed;

[0006] Based on the physical distance between each computing core and the plurality of high-bandwidth memories, the plurality of computing cores are grouped to obtain a plurality of computing core groups;

[0007] Based on the data reading volume of each matrix calculation task, the multiple matrix calculation tasks to be processed are matched with the multiple computing core groups to obtain a task allocation scheme;

[0008] Based on the task allocation scheme, each matrix calculation task is assigned to the corresponding computing core group.

[0009] In some embodiments, grouping the plurality of computing cores based on the physical distance between each computing core and the plurality of high-bandwidth memories includes:

[0010] Determine the physical distance between each computing core and the plurality of high-bandwidth memories;

[0011] Computational cores whose physical distance from the same high-bandwidth memory is less than a preset distance threshold are grouped into the same type of computational core;

[0012] Based on the physical distance between multiple computing cores in each type of computing core and the target high-bandwidth memory, each type of computing core is divided into multiple computing core groups.

[0013] In some embodiments, determining the physical distance between each computing core and the plurality of high-bandwidth memories includes:

[0014] Obtain the physical number of each computing core and determine the physical numbers of multiple high-bandwidth memories;

[0015] Based on the physical number of each computing core, the corresponding first coordinate position is obtained by querying the preset hardware topology mapping table, and based on the physical number of the multiple high-bandwidth memories, the corresponding second coordinate position is obtained by querying the hardware topology mapping table.

[0016] Based on the first coordinate position of each computing core and the second coordinate position of the plurality of high-bandwidth memories, the physical distance between each computing core and the plurality of high-bandwidth memories is calculated.

[0017] In some embodiments, determining the physical distance between each computing core and the plurality of high-bandwidth memories includes:

[0018] Obtain the physical number of each computing core and determine the physical numbers of multiple high-bandwidth memories;

[0019] The application programming interface is invoked to query a preset hardware topology mapping table based on the physical number of any computing core and the physical number of any high-bandwidth memory, so as to obtain the physical distance between the computing core and the high-bandwidth memory.

[0020] In some embodiments, the step of matching multiple matrix computation tasks to be processed with multiple computation core groups based on the data read volume of each matrix computation task to obtain a task allocation scheme includes:

[0021] Based on the amount of data read for each matrix calculation task, the multiple matrix calculation tasks are prioritized and sorted to obtain a sorting result.

[0022] The performance levels of the plurality of computing core groups are determined based on the physical distance between the plurality of computing core groups and the corresponding high-bandwidth memory.

[0023] Based on the sorting results and the performance levels of the multiple computing core groups, the multiple matrix calculation tasks are matched with the multiple computing core groups to obtain a task allocation scheme.

[0024] In some embodiments, matching the plurality of matrix computation tasks with the plurality of computational core groups to obtain a task allocation scheme includes:

[0025] With the goal of minimizing the completion time of the multiple matrix calculation tasks, the multiple matrix calculation tasks are matched with the multiple computing core groups to obtain the optimal task allocation scheme.

[0026] In some embodiments, after assigning each matrix calculation task to a corresponding computing core group based on the task allocation scheme, the method further includes:

[0027] Real-time monitoring of the load status of the multiple computing core groups;

[0028] Based on the load status of the multiple computing core groups, the task allocation scheme is adjusted to obtain the adjusted task allocation scheme;

[0029] Based on the adjusted task allocation scheme, the multiple matrix calculation tasks are reallocated.

[0030] The present invention also provides a matrix calculation optimization device applied to an artificial intelligence chip, the artificial intelligence chip including multiple computing cores and multiple high-bandwidth memories, the device comprising:

[0031] The determination unit is used to determine the amount of data to be read for each matrix calculation task to be processed.

[0032] A grouping unit is used to group the multiple computing cores based on the physical distance between each computing core and the multiple high-bandwidth memories, thereby obtaining multiple computing core groups;

[0033] The matching unit is used to match the multiple matrix calculation tasks to be processed with the multiple calculation core groups based on the data reading volume of each matrix calculation task, so as to obtain a task allocation scheme;

[0034] The allocation unit is used to allocate each matrix calculation task to the corresponding computing core group based on the task allocation scheme.

[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the matrix calculation optimization method as described above.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the matrix calculation optimization method as described above.

[0037] The matrix computation optimization method, apparatus, electronic device, and storage medium provided by this invention determine the data read volume of each matrix computation task to be processed; group multiple computation cores based on the physical distance between each computation core and multiple high-bandwidth memories to obtain multiple computation core groups; match the multiple matrix computation tasks to be processed with the multiple computation core groups based on the data read volume of each matrix computation task to obtain a task allocation scheme; and allocate each matrix computation task to the corresponding computation core group based on the task allocation scheme. This achieves load-aware task allocation, effectively avoids performance bottlenecks, and significantly improves the overall computational efficiency and hardware resource utilization of multiple matrix computation tasks. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of the structure of the artificial intelligence chip provided in an embodiment of the present invention.

[0040] Figure 2 This is a flowchart illustrating the matrix calculation optimization method provided in this embodiment of the invention.

[0041] Figure 3 This is a schematic diagram of the process of grouping multiple computing cores according to an embodiment of the present invention.

[0042] Figure 4 This is a flowchart illustrating the process of determining the task allocation scheme provided in an embodiment of the present invention.

[0043] Figure 5 This is a schematic diagram of the matrix calculation optimization device provided in an embodiment of the present invention.

[0044] Figure 6 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention.

[0045] Among them, 500 is a matrix calculation optimization device, 510 is a determination unit, 520 is a grouping unit, 530 is a matching unit, and 540 is an allocation unit; 610 is a processor, 620 is a communication interface, 630 is a memory, and 640 is a communication bus. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0047] The terms "first," "second," etc., used in this invention are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, the first object can be one or more. Furthermore, in this invention, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0048] Figure 1 This is a schematic diagram of the structure of an artificial intelligence chip provided in an embodiment of the present invention. Figure 1 As shown, the artificial intelligence chip in this embodiment of the invention includes multiple computing cores and multiple high-bandwidth memories, such as a first high-bandwidth memory and a second high-bandwidth memory. The multiple computing cores are physically arranged in an array on the artificial intelligence chip. Each computing core is an independent computing unit. The computing cores located at different positions on the artificial intelligence chip are at different physical distances relative to the first high-bandwidth memory and the second high-bandwidth memory.

[0049] It should be noted that high-performance computing chips typically integrate hundreds or even thousands of computing cores to support large-scale parallel computing. To meet the high throughput requirements of matrix computation, the chip is usually equipped with multiple high-bandwidth memory stacks, providing extremely high memory bandwidth and low power consumption.

[0050] Optionally, the AI ​​chip also includes an on-chip interconnect network, through which multiple computing cores and multiple high-bandwidth memories transmit data. The greater the physical distance, the more routing nodes the data transmission needs to pass through, resulting in higher access latency and potential bandwidth contention. The AI ​​chip also includes a task scheduler, which can be located in the host driver layer or the firmware on the device side, and is used to assign matrix calculation tasks to the matching computing core group.

[0051] Figure 2 This is a flowchart illustrating the matrix calculation optimization method provided in an embodiment of the present invention. Figure 2As shown, a matrix calculation optimization method is provided, applied to an artificial intelligence chip, including the following steps: steps 210, 220, 230, and 240. This method's steps are merely one possible implementation of the invention.

[0052] Step 210: Determine the amount of data to be read for each matrix calculation task to be processed.

[0053] Matrix computation tasks refer to sequences of instructions involving matrix operations executed on a processor, including but not limited to general matrix multiplication and convolution operations. Each matrix computation task typically involves operations on two or more input matrices; data read volume refers to the total amount of data that a matrix computation task needs to read from high-bandwidth memory to the computation core during execution.

[0054] Optionally, the data reading volume for each matrix calculation task can be calculated based on the dimension and data precision of the input matrix for each matrix calculation task.

[0055] Step 220: Based on the physical distance between each computing core and multiple high-bandwidth memories, group the multiple computing cores to obtain multiple computing core groups.

[0056] In this context, physical distance specifically refers to the wiring length or signal transmission path length of the computing core relative to high-bandwidth memory on the chip layout. This physical distance is directly related to memory access latency and actual available bandwidth.

[0057] Optionally, the chip layout information of the artificial intelligence chip is obtained, and based on the chip layout information, the physical distance between each computing core and multiple high-bandwidth memories is determined. From the multiple high-bandwidth memories, the high-bandwidth memory corresponding to each computing core is determined, that is, the high-bandwidth memory with the closest physical distance.

[0058] Optionally, for multiple computing cores corresponding to the same high-bandwidth memory, computing cores that are physically closer to the high-bandwidth memory are divided into near-end computing core groups, and computing cores that are physically farther from the high-bandwidth memory are divided into far-end computing core groups.

[0059] Understandably, by grouping multiple computing cores based on the physical distance between each computing core and multiple high-bandwidth memories, the performance of hardware computing resources can be predicted and classified, providing a clear and reliable basis for subsequent task matching.

[0060] Step 230: Based on the data reading volume of each matrix calculation task, match the multiple matrix calculation tasks to be processed with multiple computing core groups to obtain a task allocation scheme.

[0061] Optionally, the matrix computation task with the largest data reading volume can be assigned to the computing core group with the highest bandwidth; the matrix computation task with the second largest data reading volume can be assigned to the computing core group with the second highest bandwidth, and so on, to generate a clear task allocation scheme.

[0062] Optionally, multiple matrix computation tasks are matched with multiple computing core groups based on the data reading volume of each matrix computation task and the physical location of multiple computing core groups. This enables dynamic perception of computational load and hardware heterogeneity, making allocation decisions more intelligent.

[0063] Step 240: Based on the task allocation scheme, assign each matrix calculation task to the corresponding computing core group.

[0064] Optionally, the task allocation scheme is parsed to determine the mapping relationship between multiple matrix calculation tasks and multiple computing core groups, and the instruction stream corresponding to each matrix calculation task is distributed to each computing core in the corresponding computing core group.

[0065] Optionally, the chip's on-chip network and direct memory access engine can be configured according to the task allocation scheme.

[0066] In this embodiment of the invention, the data read volume of each matrix computation task to be processed is determined; multiple computation cores are grouped based on the physical distance between each computation core and multiple high-bandwidth memories to obtain multiple computation core groups; based on the data read volume of each matrix computation task, the multiple matrix computation tasks to be processed are matched with multiple computation core groups to obtain a task allocation scheme; based on the task allocation scheme, each matrix computation task is assigned to the corresponding computation core group, thereby realizing load-aware task allocation, which can effectively avoid performance bottlenecks and significantly improve the overall computing efficiency and hardware resource utilization of multiple matrix computation tasks.

[0067] Figure 3 This is a schematic diagram illustrating the process of grouping multiple computing cores according to an embodiment of the present invention. Figure 3 As shown, in some embodiments, step 220 involves grouping multiple computing cores based on the physical distance between each computing core and multiple high-bandwidth memories, including:

[0068] Step 221: Determine the physical distance between each computing core and multiple high-bandwidth memories.

[0069] Optionally, the physical distance between each computing core and multiple high-bandwidth memories can be determined by querying a preset hardware topology mapping table.

[0070] In some embodiments, determining the physical distance between each computing core and multiple high-bandwidth memories includes:

[0071] Obtain the physical number of each computing core and determine the physical numbers of multiple high-bandwidth memories;

[0072] Based on the physical number of each computing core, the corresponding first coordinate position is obtained by querying the preset hardware topology mapping table, and based on the physical numbers of multiple high-bandwidth memories, the corresponding second coordinate position is obtained by querying the hardware topology mapping table.

[0073] Based on the first coordinate position of each computing core and the second coordinate positions of multiple high-bandwidth memories, the physical distance between each computing core and multiple high-bandwidth memories is calculated.

[0074] Here, the first coordinate position refers to the spatial coordinate position of the computing core on the chip layout, and the second coordinate position refers to the spatial coordinate position of the high-bandwidth memory on the chip layout; the preset hardware topology mapping table refers to a data structure or lookup table pre-stored in the computer system. The hardware topology mapping table includes the mapping relationship between the physical numbers of the computing cores and high-bandwidth memory and their coordinate positions.

[0075] Optionally, the physical identifiers of each computing core and each high-bandwidth memory in the processor can be obtained through the system configuration file. These physical identifiers are unique identifiers for hardware components on the chip.

[0076] Optionally, a preset distance formula, such as the Manhattan distance formula or the Euclidean distance formula, can be used to calculate the physical distance between the first coordinate position and the second coordinate position.

[0077] In some embodiments, determining the physical distance between each computing core and multiple high-bandwidth memories includes:

[0078] Obtain the physical number of each computing core and determine the physical numbers of multiple high-bandwidth memories;

[0079] The application programming interface is invoked to query the preset hardware topology mapping table based on the physical number of any computing core and the physical number of any high-bandwidth memory, so as to obtain the physical distance between any computing core and any high-bandwidth memory.

[0080] The Application Programming Interface (API) is a standardized function or call entry point provided by the underlying driver or firmware specifically for querying hardware topology information. It encapsulates the underlying logic for accessing and parsing the hardware topology mapping table. Upper-layer applications do not need to concern themselves with the specific storage location, data format, or distance calculation method of the mapping table; they only need to call this API and pass in specified parameters, such as the physical number of the computing core and the physical number of the high-bandwidth memory, to directly obtain the physical distance between them.

[0081] Optionally, for all combinations of computing cores and high-bandwidth memory, the application programming interface is called repeatedly to query and output the physical distance corresponding to each combination.

[0082] Understandably, querying the physical distance between the computing core and high-bandwidth memory through the application programming interface (API) improves query efficiency, significantly reduces scheduling decision latency, and improves system response speed. When the physical layout of the underlying hardware changes, only the internal implementation of the API and its dependent hardware topology mapping table need to be updated, while the upper-layer application code that calls the API does not need to be modified, greatly enhancing the portability and scalability of the software solution.

[0083] Step 222: Group computing cores whose physical distance from the same high-bandwidth memory is less than a preset distance threshold into the same type of computing core.

[0084] Optionally, a preset distance threshold can be determined based on user input or empirical data; the preset distance threshold can be dynamically adjusted.

[0085] Optionally, the computational cores adjacent to each high-bandwidth memory are traversed, and computational cores whose physical distance to the same high-bandwidth memory is less than a preset distance threshold are grouped into the same category of computational cores.

[0086] It should be noted that if a computing core is located in the vicinity of multiple high-bandwidth memories, it will be assigned to the category of the high-bandwidth memory with the closest physical distance according to preset rules, such as the principle of proximity, to ensure that each computing core belongs to only one category.

[0087] Understandably, by grouping computing cores that are physically closer to the same high-bandwidth memory than a preset distance threshold into the same type of computing core, it is possible to significantly reduce data reading across high-bandwidth memory and long-distance transmission of on-chip networks, thereby reducing the possibility of network congestion and access conflicts.

[0088] Step 223: Divide each type of computing core into multiple computing core groups based on the physical distance between multiple computing cores in each type of computing core and the target high-bandwidth memory.

[0089] Optionally, for multiple computing cores in each type of computing core, they are precisely sorted from near to far according to their physical distance from the target high-bandwidth memory; based on the sorting results, each type of computing core is divided into multiple computing core groups.

[0090] For example, a percentile partitioning method can be used to divide the nearest 50% of computational kernels into a near-end group and the far-end group into a far-end group, or a multi-level threshold partitioning method can be used to divide each type of computational kernel into multiple computational kernel groups.

[0091] Optionally, the number and size of computing cores can be dynamically adjusted to accommodate different combinations of matrix computing tasks of varying quantities and scales, making resource allocation strategies more flexible and intelligent.

[0092] Understandably, by dividing multiple computing cores into two levels, matrix computation tasks with different data reading volumes can be more accurately matched to the most suitable computing core group, thereby avoiding conflicts caused by resource competition and improving overall computing efficiency.

[0093] Figure 4 This is a flowchart illustrating the process of determining a task allocation scheme according to an embodiment of the present invention. Figure 4 As shown, in some embodiments, step 230 matches multiple matrix computation tasks to be processed with multiple computation core groups based on the data read volume of each matrix computation task to obtain a task allocation scheme, including:

[0094] Step 231: Based on the amount of data read for each matrix calculation task, prioritize the multiple matrix calculation tasks and obtain the ranking results.

[0095] Optionally, multiple matrix calculation tasks can be sorted in descending order, that is, the matrix calculation task with the largest amount of data read gets the highest priority, and an ordered task queue is output.

[0096] Step 232: Determine the performance level of multiple computing core groups; the performance level is determined based on the physical distance between the multiple computing core groups and the corresponding high-bandwidth memory.

[0097] Optionally, a performance level can be assigned to each computing core group, which is inversely proportional to the physical distance; that is, the closer the physical distance, the higher the performance level. If a computing core group contains multiple computing cores, the average physical distance between the multiple computing cores and the corresponding high-bandwidth memory can be used as the evaluation criterion.

[0098] Optionally, multiple computing core groups can be sorted based on their performance levels and load conditions to generate an ordered hardware resource pool.

[0099] Step 233: Based on the sorting results and the performance levels of multiple computing core groups, match multiple matrix computation tasks with multiple computing core groups to obtain a task allocation scheme.

[0100] Optionally, during the task matching process, a strategy is followed to prioritize matching high-priority tasks with high-performance computing core groups.

[0101] In some embodiments, multiple matrix computation tasks are matched with multiple computational core groups to obtain a task allocation scheme, including:

[0102] With the goal of minimizing the completion time of multiple matrix computation tasks, multiple matrix computation tasks are matched with multiple computing core groups to obtain the optimal task allocation scheme.

[0103] Optionally, with the goal of minimizing the completion time of multiple matrix calculation tasks, an objective function is constructed, constraints are determined, and the objective function is solved under the constraints to obtain the optimal task allocation scheme.

[0104] Optionally, for cases with a small number of tasks and computational cores, an integer linear programming solver or exhaustive search can be used to find the theoretically optimal solution. For real-time scenarios that require fast decision-making, an efficient heuristic algorithm can be used to obtain an approximate optimal solution. For example, all tasks can be sorted in descending order of their estimated execution time, and then the tasks at the head of the queue can be assigned to the computational core with the smallest current cumulative load.

[0105] It is understandable that by minimizing the completion time of multiple matrix computation tasks, matching multiple matrix computation tasks with multiple computing core groups can seek the overall optimal coordination between all tasks and all computing core groups, achieve load balancing, and thus shorten the total processing time of the entire task set.

[0106] In some embodiments, after assigning each matrix computation task to a corresponding computational core group based on a task allocation scheme, the method further includes:

[0107] Real-time monitoring of the load status of multiple computing core groups;

[0108] Based on the load status of multiple computing core groups, the task allocation scheme is adjusted to obtain the adjusted task allocation scheme.

[0109] Based on the adjusted task allocation scheme, multiple matrix calculation tasks are redistributed.

[0110] Load status refers to a set of multi-dimensional indicators describing the workload and resource utilization saturation of a computing core group at a specific point in time. Load status includes, but is not limited to: task queue depth, estimated remaining task completion time, resource utilization, power consumption, and temperature.

[0111] Optionally, it can be determined whether the load status of multiple computing core groups meets the preset reallocation triggering conditions. The reallocation triggering conditions include, but are not limited to: detecting that the estimated remaining completion time of a certain computing core group is significantly higher than that of other computing core groups; discovering that the actual execution time of a certain matrix calculation task far exceeds the initial prediction value, causing the computing core group to become an unexpected performance bottleneck; matrix calculation tasks on a certain computing core group are completed ahead of schedule, causing it to be idle for a long time; a new matrix calculation task with the highest priority suddenly arrives, requiring the immediate freeing up of optimal resources for it.

[0112] Optionally, an adjustment strategy can be generated based on the load status of multiple computing core groups to adjust the task allocation scheme.

[0113] In this embodiment of the invention, the load status of multiple computing core groups is monitored in real time; the task allocation scheme is adjusted based on the load status of multiple computing core groups to obtain an adjusted task allocation scheme; based on the adjusted task allocation scheme, multiple matrix calculation tasks are redistributed, which can correct resource mismatch caused by inaccurate prediction in real time, so that the system can still maintain efficient operation when facing various unknown and dynamically changing workloads, and can achieve continuous fine-grained load balancing.

[0114] The matrix calculation optimization device provided in the embodiments of the present invention is described below. The matrix calculation optimization device described below and the matrix calculation optimization method described above can be referred to in correspondence.

[0115] Figure 5 This is a schematic diagram of the matrix calculation optimization device provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the matrix computation optimization device 500 is applied to an artificial intelligence chip, which includes multiple computing cores and multiple high-bandwidth memories. The matrix computation optimization device 500 includes:

[0116] The determining unit 510 is used to determine the amount of data read for each matrix calculation task to be processed;

[0117] Grouping unit 520 is used to group multiple computing cores based on the physical distance between each computing core and multiple high-bandwidth memories, to obtain multiple computing core groups;

[0118] The matching unit 530 is used to match multiple matrix calculation tasks to be processed with multiple computing core groups based on the amount of data read for each matrix calculation task, so as to obtain a task allocation scheme.

[0119] Allocation unit 540 is used to allocate each matrix calculation task to the corresponding computing core group based on the task allocation scheme.

[0120] Optionally, multiple computing cores are grouped based on the physical distance between each computing core and multiple high-bandwidth memories, including:

[0121] Determine the physical distance between each computing core and multiple high-bandwidth memories;

[0122] Computational cores whose physical distance from the same high-bandwidth memory is less than a preset distance threshold are grouped into the same type of computational core;

[0123] Based on the physical distance between multiple computing cores in each type of computing core and the target high-bandwidth memory, each type of computing core is divided into multiple computing core groups.

[0124] Optionally, the physical distance between each computing core and multiple high-bandwidth memories is determined, including:

[0125] Obtain the physical number of each computing core and determine the physical numbers of multiple high-bandwidth memories;

[0126] Based on the physical number of each computing core, the corresponding first coordinate position is obtained by querying the preset hardware topology mapping table, and based on the physical numbers of multiple high-bandwidth memories, the corresponding second coordinate position is obtained by querying the hardware topology mapping table.

[0127] Based on the first coordinate position of each computing core and the second coordinate positions of multiple high-bandwidth memories, the physical distance between each computing core and multiple high-bandwidth memories is calculated.

[0128] Optionally, the physical distance between each computing core and multiple high-bandwidth memories is determined, including:

[0129] Obtain the physical number of each computing core and determine the physical numbers of multiple high-bandwidth memories;

[0130] The application programming interface is invoked to query the preset hardware topology mapping table based on the physical number of any computing core and the physical number of any high-bandwidth memory, so as to obtain the physical distance between any computing core and any high-bandwidth memory.

[0131] Optionally, based on the data read volume of each matrix computation task, multiple matrix computation tasks to be processed are matched with multiple computation core groups to obtain a task allocation scheme, including:

[0132] Based on the amount of data read for each matrix calculation task, the multiple matrix calculation tasks are prioritized and sorted to obtain the sorting results.

[0133] Determine the performance levels of multiple computing core groups; the performance levels are determined based on the physical distance between the multiple computing core groups and their respective high-bandwidth memories.

[0134] Based on the sorting results and the performance levels of multiple computing core groups, multiple matrix computation tasks are matched with multiple computing core groups to obtain a task allocation scheme.

[0135] Optionally, multiple matrix computation tasks are matched with multiple computational core groups to obtain a task allocation scheme, including:

[0136] With the goal of minimizing the completion time of multiple matrix computation tasks, multiple matrix computation tasks are matched with multiple computing core groups to obtain the optimal task allocation scheme.

[0137] Optionally, the matrix computation optimization device 500 further includes:

[0138] The monitoring unit is used to monitor the load status of multiple computing core groups in real time.

[0139] The adjustment unit is used to adjust the task allocation scheme based on the load status of multiple computing core groups to obtain the adjusted task allocation scheme.

[0140] The reallocation unit is used to reallocate multiple matrix calculation tasks based on the adjusted task allocation scheme.

[0141] It should be noted that the matrix calculation optimization device provided in this embodiment of the invention can implement all the method steps implemented in the above matrix calculation optimization method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0142] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 6 As shown, the electronic device may include a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a matrix calculation optimization method, which includes: determining the data read volume of each matrix calculation task to be processed; grouping multiple calculation cores based on the physical distance between each calculation core and multiple high-bandwidth memories to obtain multiple calculation core groups; matching the multiple matrix calculation tasks to be processed with the multiple calculation core groups based on the data read volume of each matrix calculation task to obtain a task allocation scheme; and assigning each matrix calculation task to the corresponding calculation core group based on the task allocation scheme.

[0143] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0144] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the matrix computation optimization method provided by the methods described above. The method includes: determining the data read volume of each matrix computation task to be processed; grouping multiple computation cores based on the physical distance between each computation core and multiple high-bandwidth memories to obtain multiple computation core groups; matching the multiple matrix computation tasks to be processed with the multiple computation core groups based on the data read volume of each matrix computation task to obtain a task allocation scheme; and assigning each matrix computation task to a corresponding computation core group based on the task allocation scheme.

[0145] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0146] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a read-only memory / random access memory, a magnetic disk, an optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A matrix computation optimization method applied to an artificial intelligence chip, the artificial intelligence chip comprising multiple computation cores and multiple high-bandwidth memories, characterized in that, The method includes: Determine the amount of data to be read for each matrix calculation task to be processed; Based on the physical distance between each computing core and the plurality of high-bandwidth memories, the plurality of computing cores are grouped to obtain a plurality of computing core groups; Based on the data reading volume of each matrix calculation task, the multiple matrix calculation tasks to be processed are matched with the multiple computing core groups to obtain a task allocation scheme; Based on the task allocation scheme, each matrix calculation task is assigned to the corresponding computing core group; The grouping of the multiple computing cores based on the physical distance between each computing core and the multiple high-bandwidth memories includes: Determine the physical distance between each computing core and the plurality of high-bandwidth memories; Computational cores whose physical distance from the same high-bandwidth memory is less than a preset distance threshold are grouped into the same type of computational core; Based on the physical distance between multiple computing cores in each type of computing core and the target high-bandwidth memory, each type of computing core is divided into multiple computing core groups.

2. The matrix calculation optimization method according to claim 1, characterized in that, Determining the physical distance between each computing core and the plurality of high-bandwidth memories includes: Obtain the physical number of each computing core and determine the physical numbers of multiple high-bandwidth memories; Based on the physical number of each computing core, the corresponding first coordinate position is obtained by querying the preset hardware topology mapping table, and based on the physical number of the multiple high-bandwidth memories, the corresponding second coordinate position is obtained by querying the hardware topology mapping table. Based on the first coordinate position of each computing core and the second coordinate position of the plurality of high-bandwidth memories, the physical distance between each computing core and the plurality of high-bandwidth memories is calculated.

3. The matrix calculation optimization method according to claim 1, characterized in that, Determining the physical distance between each computing core and the plurality of high-bandwidth memories includes: Obtain the physical number of each computing core and determine the physical numbers of multiple high-bandwidth memories; The application programming interface is invoked to query a preset hardware topology mapping table based on the physical number of any computing core and the physical number of any high-bandwidth memory, so as to obtain the physical distance between the computing core and the high-bandwidth memory.

4. The matrix calculation optimization method according to claim 1, characterized in that, The step of matching multiple matrix calculation tasks to be processed with multiple computing core groups based on the data read volume of each matrix calculation task to obtain a task allocation scheme includes: Based on the amount of data read for each matrix calculation task, the multiple matrix calculation tasks are prioritized and sorted to obtain a sorting result. The performance levels of the plurality of computing core groups are determined based on the physical distance between the plurality of computing core groups and the corresponding high-bandwidth memory. Based on the sorting results and the performance levels of the multiple computing core groups, the multiple matrix calculation tasks are matched with the multiple computing core groups to obtain a task allocation scheme.

5. The matrix calculation optimization method according to claim 4, characterized in that, The step of matching the multiple matrix calculation tasks with the multiple computing core groups to obtain a task allocation scheme includes: With the goal of minimizing the completion time of the multiple matrix calculation tasks, the multiple matrix calculation tasks are matched with the multiple computing core groups to obtain the optimal task allocation scheme.

6. The matrix calculation optimization method according to claim 1, characterized in that, After assigning each matrix calculation task to a corresponding computing core group based on the task allocation scheme, the process further includes: Real-time monitoring of the load status of the multiple computing core groups; Based on the load status of the multiple computing core groups, the task allocation scheme is adjusted to obtain the adjusted task allocation scheme; Based on the adjusted task allocation scheme, the multiple matrix calculation tasks are reallocated.

7. A matrix computation optimization device applied to an artificial intelligence chip, the artificial intelligence chip comprising multiple computation cores and multiple high-bandwidth memories, characterized in that, The device includes: The determination unit is used to determine the amount of data to be read for each matrix calculation task to be processed. A grouping unit is used to group the multiple computing cores based on the physical distance between each computing core and the multiple high-bandwidth memories, thereby obtaining multiple computing core groups; The matching unit is used to match the multiple matrix calculation tasks to be processed with the multiple calculation core groups based on the data reading volume of each matrix calculation task, so as to obtain a task allocation scheme; An allocation unit is used to allocate each matrix calculation task to a corresponding computing core group based on the task allocation scheme. The grouping of the multiple computing cores based on the physical distance between each computing core and the multiple high-bandwidth memories includes: Determine the physical distance between each computing core and the plurality of high-bandwidth memories; Computational cores whose physical distance from the same high-bandwidth memory is less than a preset distance threshold are grouped into the same type of computational core; Based on the physical distance between multiple computing cores in each type of computing core and the target high-bandwidth memory, each type of computing core is divided into multiple computing core groups.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the matrix calculation optimization method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the matrix calculation optimization method as described in any one of claims 1 to 6.

Images