Memory access optimization method and device, storage medium and product
By splitting memory access addresses and performing multi-level hash calculations, the problem of uneven utilization of memory channel resources in the GPGPU architecture is solved, memory access load balancing is achieved, and chip performance and memory bandwidth utilization are improved.
Patent Information
- Application Number
- CN202610101633.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-23
- Publication Date
- 2026-05-12
AI Technical Summary
In the GPGPU architecture, the direct mapping method leads to uneven utilization of memory channel resources, which affects chip performance.
By splitting the memory access address and performing multi-level hash calculations, the target channel index is obtained, and memory access requests are evenly mapped to multiple memory channels to achieve memory access load balancing.
It improves the utilization of the memory controller, reduces memory access latency, and enhances the overall performance of the AI chip, especially in memory-intensive applications where performance is improved by 25-35% and latency is reduced by 30%.
Smart Images
Figure CN122019411A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence chip technology, and in particular to a memory access optimization method, device, storage medium and product. Background Technology
[0002] Artificial intelligence chip architectures (such as general-purpose graphics processing units (GPGPUs)) typically include multi-level cache hierarchies. Among these, the access performance for high-bandwidth memory (HBM) is crucial to the overall computing performance of the artificial intelligence chip.
[0003] In the context of related technologies, HBM in GPGPU architecture typically employs a multi-channel architecture, meaning it includes multiple memory channels. The compute core uses a direct mapping method to allocate memory access requests to specific memory channels.
[0004] However, direct mapping selects the memory channel based on specific bits of the memory address. When the memory address has a fixed step size, direct mapping can cause consecutive memory access requests to be mapped to the same memory channel. This can lead to some memory channels being overloaded while others are idle, resulting in uneven resource utilization and affecting the overall performance of the chip. Summary of the Invention
[0005] This application provides a memory access optimization method, device, and storage medium for uniformly mapping memory access requests to multiple memory channels to achieve memory access load balancing, thereby improving resource utilization and overall chip performance.
[0006] On one hand, embodiments of this application provide a memory access optimization method, the method comprising: Receive a memory access request sent by the computing core, the memory access request carrying the memory access address; Based on the hardware hierarchy of the artificial intelligence chip, the memory access address is segmented to obtain multiple hierarchical address segments. The hardware hierarchy includes multiple hardware layers, and each hardware layer corresponds to a hierarchical address segment. Hash calculations are performed on the multiple address segments to obtain the corresponding target hash results; The target channel index is obtained by combining the multiple target hash results. According to the target channel index, the memory access request is sent to the corresponding memory channel for execution, and the execution result is obtained.
[0007] On one hand, embodiments of this application provide a memory access optimization apparatus, the apparatus comprising: The acquisition module is used to receive memory access requests sent by the computing core, wherein the memory access requests carry memory access addresses; The segmentation module is used to segment the memory access address based on the hardware hierarchy of the artificial intelligence chip to obtain multiple hierarchical address segments. The hardware hierarchy includes multiple hardware layers, and each hardware layer corresponds to a hierarchical address segment. The matching module is used to perform hash calculations on the multiple address segments to obtain corresponding target hash results; combine the obtained target hash results to obtain a target channel index; and send the memory access request to the corresponding memory channel for execution according to the target channel index to obtain the execution result.
[0008] Optionally, the hardware hierarchy of the artificial intelligence chip includes: multiple hardware layers; each hardware layer corresponds to a layer address segment.
[0009] Optionally, the matching module is specifically used for: Hash calculations are performed on the multiple address segments in descending order of hierarchy to obtain the corresponding target hash results.
[0010] Optionally, the matching module is specifically used for: The following hash calculation process is performed on the multiple address segments in descending order of hierarchy: A hash calculation is performed on the reference address segment carried by the memory access address and the hierarchical address segment to obtain a preliminary hash result; Based on the preliminary hash result and the target hash results of other hierarchical address segments already obtained, the target hash result of the hierarchical address segment is obtained.
[0011] Optionally, the matching module is specifically used for: The initial hash result and the target hash result of other obtained hierarchical address segments are XORed to obtain the target hash result of the hierarchical address segments.
[0012] Optionally, the memory access address further includes: a reference address range; The matching module is specifically used for: For each level address segment, a hash calculation is performed on the reference address segment and the level address segment to obtain the corresponding target hash result.
[0013] Optionally, the matching module is specifically used for: The multiple target hash results are combined in descending order of hierarchy to obtain the target channel index.
[0014] Optionally, the matching module is specifically used for: The target channel index includes sub-indexes for each of the multiple hardware layers; For each hardware layer, based on the sub-index of the hardware layer, determine the matching hardware resources under that hardware layer; Based on the hardware resources matched by each of the multiple hardware layers, the memory channel matched by the memory access request is determined. The memory access request is sent to the memory channel for execution, and the execution result is obtained.
[0015] Optionally, the matching module is specifically used for: The memory access request is sent to the memory channel for execution through the memory controller, and the execution result is obtained. The memory channel is located in the video memory, which includes multiple memory channels, and each memory channel corresponds to a memory controller.
[0016] On one hand, embodiments of this application provide a computer device, including a memory, an artificial intelligence chip, and a computer program stored on the memory and running on the artificial intelligence chip, wherein the artificial intelligence chip implements the steps of the above-described memory access optimization method when executing the computer program.
[0017] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the memory access optimization method described above.
[0018] On one hand, embodiments of this application provide a computer program product, the computer program product including a computer program stored on a computer-readable storage medium, the computer program including program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the above-described memory access optimization method.
[0019] In this embodiment, firstly, a memory access request sent by the computing core is received, carrying the memory access address. Based on the hardware hierarchy of the AI chip, the memory access address is segmented to obtain multiple hierarchical address segments. Multi-level hash calculations are performed on the multiple hierarchical address segments to obtain corresponding target hash results. Then, the obtained target hash results are combined to obtain a target channel index. The target channel index obtained in this way will not be concentrated in a few memory channels, but will be evenly distributed across different memory channels, thereby achieving load balancing of memory access.
[0020] Secondly, since each memory channel corresponds to an independent memory controller, distributing multiple memory access requests evenly across different memory channels effectively reduces the situation where some memory controllers are overloaded while others are idle, thereby improving the utilization rate of memory controllers and thus increasing the overall memory bandwidth utilization rate.
[0021] In addition, load balancing of memory access can effectively reduce memory access latency, especially in memory-intensive applications, and can improve the overall performance of artificial intelligence chips. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This is a schematic diagram of the structure of an artificial intelligence chip provided in an embodiment of this application; Figure 2 A flowchart illustrating a memory access optimization method provided in an embodiment of this application; Figure 3 A schematic diagram of a memory access address provided for an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a memory access optimization device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0024] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0025] refer to Figure 1 This is a structural diagram of an artificial intelligence chip applicable to embodiments of this application. The artificial intelligence chip 100 includes at least: a video memory 101 and multiple computing cores 102; the computing cores 102 can be computing units (CUs). The computing cores 102 have artificial intelligence-related computing capabilities, such as matrix multiplication, convolution operations, etc., which are common computational operations in artificial intelligence models.
[0026] The video memory 101 can be HBM or other types of memory. The video memory 101 adopts a multi-channel architecture, that is, the video memory 101 includes multiple memory channels; the computing core 102 accesses the video memory 101 through the memory controller, and each memory channel corresponds to an independent memory controller.
[0027] In this embodiment, the computing core 102 issues a memory access request, which carries the memory access address. Based on the hardware hierarchy of the AI chip 100, the memory access address is segmented to obtain multiple hierarchical address segments, each corresponding to a hardware hierarchy of the AI chip 100. There is a hierarchical relationship between the hardware resources of the multiple hardware hierarchies. These hardware resources can be ports, routing lines, etc., of a module within the AI chip.
[0028] Hash calculations are performed on multiple address segments to obtain the corresponding target hash results; then, the multiple target hash results are combined to obtain the target channel index; then, according to the target channel index, the memory access request is sent to the corresponding memory channel in video memory 101 for execution to obtain the execution result.
[0029] In this embodiment, firstly, multi-level hash calculations are performed on multiple address segments to obtain corresponding target hash results. Then, the multiple target hash results are combined to obtain the target channel index. The target channel index obtained in this way will not be concentrated in a part of the memory channel, but will be evenly distributed on different memory channels, thereby achieving load balancing of memory access.
[0030] Secondly, since each memory channel corresponds to an independent memory controller, distributing multiple memory access requests evenly across different memory channels effectively reduces the situation where some memory controllers are overloaded while others are idle, thereby improving the utilization rate of memory controllers and thus increasing the overall memory bandwidth utilization rate.
[0031] In addition, load balancing of memory access can effectively reduce memory access latency, thereby improving the overall performance of AI chips; especially in memory-intensive applications, GPGPU can improve overall performance by 25-35% and reduce memory access latency by 30%.
[0032] Furthermore, the complexity of multi-level hash algorithms is linearly related to the number of memory channels, supporting HBM architecture expansion with more channels; moreover, multi-level hash algorithms are simple to implement, with hardware resource consumption increasing by only 5-8%, and will not significantly affect the area and power consumption budget of artificial intelligence chips, i.e., low hardware overhead.
[0033] In addition to the structure described above, the artificial intelligence chip 100 in this application may also include other structures, and this application does not specifically limit such structures.
[0034] Artificial intelligence chips 100 can be: Graphics Processing Unit (GPU), General-Purpose Graphics Processing Units (GPGPU), Domain Specific Architecture (DSA), etc.
[0035] The following is based on Figure 1 The diagram shown illustrates the architecture of an AI chip and details a process for optimizing memory access. (See attached image.) Figure 2 This method is executed by an artificial intelligence chip and includes the following steps: Step 201: Receive the memory access request sent by the computing core. The memory access request carries the memory access address.
[0036] Specifically, the memory access address includes an M-bit high-order address segment and an N-bit low-order address segment, where the N-bit low-order address segment uniquely identifies the memory channel to be allocated.
[0037] In this embodiment, the artificial intelligence chip is a multi-layered system. Specifically, the hardware hierarchy of the AI chip includes multiple hardware layers; the low-order address segment includes multiple hierarchical address segments, with each hardware layer corresponding to one hierarchical address segment. There is a hierarchical relationship between the hardware resources across the multiple hardware layers; that is, higher-level hardware resources can be further divided into multiple lower-level hardware resources. These hardware resources can be ports, routing lines, etc., of a module within the AI chip.
[0038] When the hardware hierarchy of an AI chip differs, the multiple levels of address segments included in the lower-order address segment also differ. In practical applications, the bits corresponding to multiple address segments do not overlap; furthermore, the higher the hardware hierarchy, the higher the corresponding bit count. The number of bits occupied by multiple address segments can be the same or different.
[0039] For example, the structure of an artificial intelligence chip can be defined by three hardware layers, namely the first hardware layer, the second hardware layer, and the third hardware layer, which are in descending order of hierarchy.
[0040] The artificial intelligence chip can be divided into four first-level hardware resources, each of which can be further divided into four second-level hardware resources, and each of which can be further divided into four third-level hardware resources.
[0041] Accordingly, see Figure 3The memory access address includes a high-order address segment and a low-order address segment, where the high-order address segment is M bits and the low-order address segment is N bits, where M and N are positive integers.
[0042] The low-level address segment includes: the first hardware level address segment (L bit), the second hardware level address segment (K bit), and the third hardware level address segment (J bit), where L, K, and J are positive integers.
[0043] Step 202: Based on the hardware hierarchy of the artificial intelligence chip, the memory access address is segmented to obtain multiple hierarchical address segments.
[0044] Specifically, based on the hardware hierarchy of the AI chip, multiple hardware layers, the hierarchical relationship between the multiple hardware layers, and the number of bits occupied by the hierarchical address segment of each hardware layer are determined. Then, according to the hierarchical relationship between the multiple hardware layers and the number of bits occupied by the hierarchical address segment of each hardware layer, the low-order address segment of the memory access address is divided into multiple hierarchical address segments. Each hierarchical address segment corresponds to a hardware layer in the AI chip.
[0045] For example, Figure 3 The low-order address segment shown is divided into: the first hardware level address segment (L bit), the second hardware level address segment (K bit), and the third hardware level address segment (J bit).
[0046] Step 203: Perform hash calculations on the multiple address segments to obtain the corresponding target hash results.
[0047] Specifically, a hash algorithm requires two parameters: a reference address range and the address range to be hashed. The least significant bit of the reference address range determines the hash algorithm's sensitivity to the address range; the larger the least significant bit of the reference address range, the lower the hash algorithm's sensitivity to the address range.
[0048] In this embodiment of the application, the high-order address segment in the memory access address can be used as the reference address segment, and the low-order address segment in the memory access address can be used as the address segment to be hashed.
[0049] However, as the number of memory channels in HBM increases, the low-order address segment of the memory access address becomes longer; correspondingly, the least significant bit of the high-order address segment becomes higher, thus making the hash algorithm less sensitive to the address segment that needs to be hashed.
[0050] In this scenario, if the entire low-order address segment of the memory access address is used as the address segment to be hashed, the hash algorithm becomes insensitive to the hardware hierarchy represented by the low-order address segment. That is, when mapping and determining the memory channel, the hardware hierarchy of the AI chip cannot be taken into account. This makes it difficult for access requests to be evenly mapped at each hardware level (e.g., concentrated mapping to memory channels under a certain hardware level), resulting in uneven resource utilization.
[0051] Based on this, in the embodiments of this application, the high-order address segment in the memory access address is used as the reference address segment, and each level address segment is used as the address segment that needs to be hashed.
[0052] That is, for each level address segment, a hash calculation is performed on the reference address segment and the level address segment to obtain the corresponding target hash result.
[0053] In some embodiments, hash calculations are performed on multiple address segments in descending order of hierarchy to obtain the corresponding target hash results. The hash calculation process for each address segment includes the following operations: The reference address segment and hierarchical address segment carried by the memory access address are hashed to obtain a preliminary hash result; then, based on the preliminary hash result and the target hash results of other hierarchical address segments already obtained, the target hash result of that hierarchical address segment is obtained.
[0054] Specifically, the hardware level of the other address segments mentioned above is greater than the hardware level of the address segment currently being hashed. An XOR operation is performed between the preliminary hash result and the target hash result of the other address segments to obtain the target hash result of the address segment.
[0055] In practical applications, the address segments of multiple levels are first sorted in descending order of hierarchy to obtain the sorting result.
[0056] For the first level address segment in the sorting results, perform hash calculations on the reference address segment and the address segment at that level to obtain a preliminary hash result; use the preliminary hash result directly as the target hash result for the first level address segment.
[0057] For the second-level address segment in the sorting results, perform hash calculations on the reference address segment and the address segment at this level to obtain a preliminary hash result; perform an XOR operation on the preliminary hash result and the target hash result of the first-level address segment to obtain the target hash result of the second-level address segment.
[0058] For the third level address segment in the sorting results, perform a hash calculation on the reference address segment and the address segment at this level to obtain a preliminary hash result; perform an XOR operation on the preliminary hash result, the target hash result of the first level address segment, and the target hash result of the second level address segment to obtain the target hash result of the third level address segment.
[0059] The reasoning continues until the target hash result of the last level address segment is obtained.
[0060] For example, the high-order address segment AH in the memory access address is set as the reference address segment, and the low-order address segment is set to be divided into: the first hardware level address segment A1, the second hardware level address segment A2, and the third hardware level address segment A3.
[0061] The address segments of the above levels are sorted in order of increasing hierarchy to obtain the sorting results, namely A1, A2, and A3.
[0062] Perform a hash calculation on the high-order address segment AH and the hierarchical address segment A1 of the first hardware level to obtain the first hash result. Specifically, as shown in the following formula (1): (1) Here, Hash represents a hash function.
[0063] Perform a hash calculation on the high-order address segment AH and the second hardware level address segment A1, and then compare the obtained hash result with the first hash result. Perform an XOR operation to obtain the second hash result. Specifically, as shown in the following formula (2): (2) Perform hash calculations on the high-order address segment AH and the third hardware level address segment A3, and then compare the resulting hash with the first hash result. Second hash result Perform an XOR operation to obtain the third hash result. Specifically, as shown in the following formula (3): (3) Step 204: Combine the obtained target hash results to obtain the target channel index.
[0064] Specifically, multiple target hash results are combined in descending order of hierarchy to obtain the target channel index.
[0065] For example, the first hash result obtained above... Second hash result and the third hash result Perform splicing to obtain the target channel index.
[0066] Step 205: Send the memory access request to the corresponding memory channel for execution according to the target channel index, and obtain the execution result.
[0067] In practical applications, the target channel index includes multiple sub-indexes for each hardware level. Based on the sub-index of each hardware level, the matching hardware resources under each hardware level can be determined. Specifically, first, based on the sub-index of the highest-level hardware level, the matching hardware resources under that highest-level hardware level are determined. Then, based on the sub-index of the next-lower-level hardware level, the matching hardware resources for the next level are determined from the matching hardware resources; and so on, until all hardware levels have been traversed.
[0068] Based on the hardware resources matched at each of the multiple hardware layers, a memory channel matching the memory access request is determined. This channel is uniquely accessible through the hardware resources (such as ports, routing lines, etc.) matched at each of the multiple hardware layers. The memory access request is then sent to the memory channel for execution, and the execution result is obtained.
[0069] In some embodiments, the computing core accesses video memory (such as HBM) through a memory controller. Video memory typically employs a multi-channel architecture, comprising multiple memory channels, each corresponding to an independent memory controller. After determining the memory channel that matches the memory access request, the memory access request is sent to the memory channel for execution through the corresponding memory controller, and the execution result is obtained.
[0070] In this embodiment, firstly, a memory access request sent by the computing core is received, carrying the memory access address. Based on the hardware hierarchy of the AI chip, the memory access address is segmented to obtain multiple hierarchical address segments. Multi-level hash calculations are performed on the multiple hierarchical address segments to obtain corresponding target hash results. Then, the obtained target hash results are combined to obtain a target channel index. The target channel index obtained in this way will not be concentrated in a few memory channels, but will be evenly distributed across different memory channels, thereby achieving load balancing of memory access.
[0071] Secondly, since each memory channel corresponds to an independent memory controller, distributing multiple memory access requests evenly across different memory channels effectively reduces the situation where some memory controllers are overloaded while others are idle, thereby improving the utilization rate of memory controllers and thus increasing the overall memory bandwidth utilization rate.
[0072] Furthermore, load balancing of memory access can effectively reduce memory access latency, especially in memory-intensive applications, thereby improving the overall performance of AI chips. Through configurable parameter design, it can adapt to different access modes and system configurations, making it widely applicable.
[0073] Furthermore, the complexity of multi-level hash algorithms is linearly related to the number of memory channels, supporting HBM architecture expansion with more channels; moreover, multi-level hash algorithms are simple to implement and will not significantly affect the area and power consumption budget of artificial intelligence chips, i.e., low hardware overhead.
[0074] Based on the same technical concept, this application provides a schematic diagram of the structure of a memory access optimization device, such as... Figure 4 As shown, the memory access optimization device 400 includes: The acquisition module 401 is used to receive a memory access request sent by the computing core, wherein the memory access request carries the memory access address; The segmentation module 402 is used to segment the memory access address based on the hardware hierarchy of the artificial intelligence chip to obtain multiple hierarchical address segments; the hardware hierarchy includes multiple hardware layers, and each hardware layer corresponds to a hierarchical address segment. The matching module 403 is used to perform hash calculations on the multiple address segments to obtain corresponding target hash results; combine the obtained target hash results to obtain a target channel index; and send the memory access request to the corresponding memory channel for execution according to the target channel index to obtain the execution result.
[0075] Optionally, the hardware hierarchy of the artificial intelligence chip includes: multiple hardware layers; each hardware layer corresponds to a layer address segment.
[0076] Optionally, the matching module 403 is specifically used for: Hash calculations are performed on the multiple address segments in descending order of hierarchy to obtain the corresponding target hash results.
[0077] Optionally, the matching module 403 is specifically used for: The following hash calculation process is performed on the multiple address segments in descending order of hierarchy: A hash calculation is performed on the reference address segment carried by the memory access address and the hierarchical address segment to obtain a preliminary hash result; Based on the preliminary hash result and the target hash results of other hierarchical address segments already obtained, the target hash result of the hierarchical address segment is obtained.
[0078] Optionally, the matching module 403 is specifically used for: The initial hash result and the target hash result of other obtained hierarchical address segments are XORed to obtain the target hash result of the hierarchical address segments.
[0079] Optionally, the memory access address further includes: a reference address range; The matching module 403 is specifically used for: For each level address segment, a hash calculation is performed on the reference address segment and the level address segment to obtain the corresponding target hash result.
[0080] Optionally, the matching module 403 is specifically used for: The multiple target hash results are combined in descending order of hierarchy to obtain the target channel index.
[0081] Optionally, the matching module 403 is specifically used for: The target channel index includes sub-indexes for each of the multiple hardware layers; For each hardware layer, based on the sub-index of the hardware layer, determine the matching hardware resources under that hardware layer; Based on the hardware resources matched by each of the multiple hardware layers, the memory channel matched by the memory access request is determined. The memory access request is sent to the memory channel for execution, and the execution result is obtained.
[0082] Optionally, the matching module 403 is specifically used for: The memory access request is sent to the memory channel for execution through the memory controller, and the execution result is obtained. The memory channel is located in the video memory, which includes multiple memory channels, and each memory channel corresponds to a memory controller.
[0083] In this embodiment, firstly, a memory access request sent by the computing core is received, carrying the memory access address. Based on the hardware hierarchy of the AI chip, the memory access address is segmented to obtain multiple hierarchical address segments. Multi-level hash calculations are performed on the multiple hierarchical address segments to obtain corresponding target hash results. Then, the obtained target hash results are combined to obtain a target channel index. The target channel index obtained in this way will not be concentrated in a few memory channels, but will be evenly distributed across different memory channels, thereby achieving load balancing of memory access.
[0084] Secondly, since each memory channel corresponds to an independent memory controller, distributing multiple memory access requests evenly across different memory channels effectively reduces the situation where some memory controllers are overloaded while others are idle, thereby improving the utilization rate of memory controllers and thus increasing the overall memory bandwidth utilization rate.
[0085] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.
[0086] Based on the same technical concept, embodiments of this application provide a computer device, such as... Figure 5 As shown, it includes at least one artificial intelligence chip 100 and a memory 501 connected to at least one artificial intelligence chip 100. In this embodiment, the specific connection medium between the artificial intelligence chip 100 and the memory 501 is not limited. Figure 5 Taking the connection between the AI chip 100 and the memory 501 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.
[0087] In this embodiment of the application, the memory 501 stores instructions that can be executed by at least one artificial intelligence chip 100. By executing the instructions stored in the memory 501, the at least one artificial intelligence chip 100 can perform the steps of the memory access optimization method described above.
[0088] The artificial intelligence chip 100 serves as the control center of the computer device. It connects to various parts of the computer device via various interfaces and lines, optimizing memory access by running or executing instructions stored in the memory 501 and accessing data stored in the memory 501. Optionally, the artificial intelligence chip 100 may include one or more processing units. It may integrate an application processor and a modem processor, where the application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the artificial intelligence chip 100. In some embodiments, the artificial intelligence chip 100 and the memory 501 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.
[0089] The artificial intelligence chip 100 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.
[0090] Memory 501, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 501 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 501 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer device, but is not limited thereto. Memory 501 in the embodiments of this application may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.
[0091] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the above-described memory access optimization method.
[0092] Based on the same inventive concept, this application provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions that, when executed by a computer device, cause the computer device to perform the steps of the above-described memory access optimization method.
[0093] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0094] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer apparatus or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0095] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer device or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0096] These computer program instructions may also be loaded onto a computer device or other programmable data processing equipment to cause a series of operational steps to be performed on the computer device or other programmable equipment to produce a process implemented by the computer device, thereby providing instructions that execute on the computer device or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0097] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.
[0098] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. A memory access optimization method, characterized in that, The method includes: Receive a memory access request sent by the computing core, the memory access request carrying the memory access address; Based on the hardware hierarchy of the artificial intelligence chip, the memory access address is segmented to obtain multiple hierarchical address segments. The hardware hierarchy includes multiple hardware layers, and each hardware layer corresponds to a hierarchical address segment. Hash calculations are performed on the multiple address segments to obtain the corresponding target hash results; The target channel index is obtained by combining the multiple target hash results. According to the target channel index, the memory access request is sent to the corresponding memory channel for execution, and the execution result is obtained.
2. The method as described in claim 1, characterized in that, The hardware hierarchy of the artificial intelligence chip includes: multiple hardware layers; each hardware layer corresponds to a layer address segment.
3. The method as described in claim 1, characterized in that, The step of performing hash calculations on the multiple address segments to obtain the corresponding target hash results includes: Hash calculations are performed on the multiple address segments in descending order of hierarchy to obtain the corresponding target hash results.
4. The method as described in claim 3, characterized in that, The step of performing hash calculations on the multiple address segments in descending order of hierarchy to obtain the corresponding target hash results includes: The following hash calculation process is performed on the multiple address segments in descending order of hierarchy: A hash calculation is performed on the reference address segment carried by the memory access address and the hierarchical address segment to obtain a preliminary hash result; Based on the preliminary hash result and the target hash results of other hierarchical address segments already obtained, the target hash result of the hierarchical address segment is obtained.
5. The method as described in claim 4, characterized in that, The process of obtaining the target hash result of the hierarchical address segment based on the preliminary hash result and the target hash results of other obtained hierarchical address segments includes: The initial hash result and the target hash result of other obtained hierarchical address segments are XORed to obtain the target hash result of the hierarchical address segments.
6. The method as described in claim 1, characterized in that, The memory access address further includes: a reference address segment; the step of performing hash calculations on the multiple level address segments respectively to obtain the corresponding target hash results includes: For each level address segment, a hash calculation is performed on the reference address segment and the level address segment to obtain the corresponding target hash result.
7. The method as described in any one of claims 1 to 6, characterized in that, The step of combining multiple target hash results to obtain the target channel index includes: The multiple target hash results are combined in descending order of hierarchy to obtain the target channel index.
8. The method according to any one of claims 1 to 6, characterized in that, The target channel index includes sub-indexes for each of the multiple hardware layers; The step of sending the memory access request to the corresponding memory channel for execution according to the target channel index and obtaining the execution result includes: For each hardware layer, based on the sub-index of the hardware layer, determine the matching hardware resources under that hardware layer; Based on the hardware resources matched by each of the multiple hardware layers, the memory channel matched by the memory access request is determined. The memory access request is sent to the memory channel for execution, and the execution result is obtained.
9. The method as described in claim 8, characterized in that, Sending the memory access request to the memory channel for execution and obtaining the execution result includes: The memory access request is sent to the memory channel for execution through the memory controller, and the execution result is obtained. The memory channel is located in the video memory, which includes multiple memory channels, and each memory channel corresponds to a memory controller.
10. A computer device comprising a memory, an artificial intelligence chip, and a computer program stored on the memory and running on the artificial intelligence chip, characterized in that, When the artificial intelligence chip executes the computer program, it implements the steps of the method as described in any one of claims 1 to 9.