Memory allocation method and device, chip, electronic equipment and storage medium

Abstract

The invention provides a memory allocation method and device, a chip, electronic equipment and a storage medium. The memory allocation method comprises the steps that local first memories are arranged for a plurality of calculation modules, and each calculation module comprises a first memory and a second memory; allocating first buffer areas to first memories corresponding to the plurality of calculation modules respectively, wherein each first memory is in one-to-one correspondence with each first buffer area; a plurality of first virtual addresses are allocated for each first buffer area, the plurality of first virtual addresses corresponding to one calculation module are continuous, and each first virtual address comprises a calculation module identifier and an address offset in the first buffer area; a data access operation is performed based on a first virtual address of a first memory of the computing module and a second memory. According to the invention, the data access efficiency of the calculation module can be improved.

Description

Technical Field

[0001] This application relates to on-chip network technology, and more particularly to a memory allocation method, apparatus, chip, electronic device, and storage medium. Background Technology

[0002] In multi-processor node or multi-compute tile architectures, frequent data interaction and collaborative processing are typically required between the various computing modules. Under conventional memory allocation and mapping mechanisms, a strategy of interleaving physical addresses across modules is usually employed for global physical memory allocation. In this mechanism, the physical memory pages of the intermediate buffer required by a single communication task are discretely distributed across multiple dynamic random access memory (DRAM) nodes connected to different computing modules. When a single computing module performs data processing and initiates load or store operations on the intermediate buffer, its access requests and data often need to frequently traverse the network on-chip (NoC) and be routed to the DRAM nodes of other remote computing modules for processing.

[0003] Because the physical addresses of the intermediate buffers are discretely distributed, large amounts of cross-module data transfer increase the communication load on the on-chip network, introducing additional multi-hop routing node turnaround time and remote memory access latency. Furthermore, the memory access latency caused by cross-on-chip network access to remote storage is unpredictable, making the execution process of the communication pipeline responsible for data flow within the computing module prone to waiting or interruption. This, in turn, affects the global data throughput and storage bandwidth utilization when multiple modules work together, reducing the efficiency of data access (data storage or data loading) in the computing module. Summary of the Invention

[0004] This application provides a memory allocation method, apparatus, chip, electronic device, and storage medium that can improve the data access efficiency of computing modules.

[0005] The technical solution of this application embodiment is implemented as follows: This application provides a memory allocation method, the method comprising: A local first memory is set up for multiple computing modules, wherein each computing module includes a first memory and a second memory; A first buffer is allocated to the first memory corresponding to each of the plurality of computing modules, and each first memory corresponds one-to-one with each first buffer. Multiple first virtual addresses are allocated for each first buffer, wherein the multiple first virtual addresses corresponding to one computing module are consecutive, and the first virtual address includes a computing module identifier and an address offset within the first buffer; Data access operations are performed based on the first virtual address of the first memory of the computing module and the second memory.

[0006] This application provides a memory allocation device, including: A buffer allocation module is used to set up local first memory for multiple computing modules, wherein each computing module includes a first memory and a second memory; and to allocate a first buffer to the first memory corresponding to each of the multiple computing modules, wherein each first memory corresponds one-to-one with each first buffer. The address allocation module is used to allocate multiple first virtual addresses for each first buffer, wherein the multiple first virtual addresses corresponding to one calculation module are consecutive, and the first virtual address includes a calculation module identifier and an address offset within the first buffer. The data access module is used to perform data access operations based on the first virtual address of the first memory of the computing module and the second memory.

[0007] This application provides a chip that includes multiple computing modules and an on-chip network. Each computing module includes a first memory and a second memory. The first memory is a dynamic random access memory, and the second memory is a static random access memory. The chip is used to implement the memory allocation method described in the embodiments of this application.

[0008] This application provides an electronic device, the electronic device comprising: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the memory allocation method provided in the embodiments of this application.

[0009] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs, which, when executed by a processor, implement the memory allocation method provided in this application.

[0010] This application provides a computer program product, including a computer program or computer executable instructions, which, when executed by a processor, implements the memory allocation method provided in this application.

[0011] The embodiments of this application have the following beneficial effects: By setting a first buffer in the local first memory of each computing module, the physical carrier of data is completely confined within the computing module when the computing module performs communication or intermediate data caching. This avoids the need for remote memory access across the on-chip network to obtain data, reduces the physical routing latency of a single read / write operation, saves data packet traffic injected into the on-chip network due to cross-module access, and effectively reduces the probability of network node congestion. A virtual address space is constructed by combining the computing module identifier with the internal address offset. During address resolution, the physical node corresponding to the address can be accurately determined simply by extracting the computing module identifier, simplifying the address translation and addressing logic. Simultaneously, allocating consecutive virtual addresses to a single computing module ensures the data space locality during data throughput, effectively improving the bandwidth utilization of the storage bus for continuous batch reads and writes. Based on the aforementioned buffer and the corresponding virtual addresses, data access operations are performed between the first and second memories, ensuring that the physical path of the access operation is fixed and unique. Compared to the memory access latency jitter caused by multi-hop routing in related technologies, the data interaction between the local first memory and the second memory can effectively prevent the data processing pipeline inside the computing module from being interrupted or stalled due to waiting for remote data synchronization, thereby improving the data transmission efficiency in a multi-computing module parallel environment. Attached Figure Description

[0012] Figure 1A This is a first structural schematic diagram of the chip provided in an embodiment of this application; Figure 1B This is a schematic diagram of the second structure of the chip provided in the embodiments of this application; Figure 2A This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application; Figure 2B This is a schematic diagram of the server structure provided in an embodiment of this application; Figure 3A This is a first flowchart illustrating the memory allocation method provided in this application embodiment; Figure 3B This is a second flowchart illustrating the memory allocation method provided in the embodiments of this application; Figure 4A This is a schematic diagram illustrating the principle of memory allocation provided in an embodiment of this application; Figure 4B This is a schematic diagram of a virtual address provided in an embodiment of this application; Figure 5A This is a schematic diagram of the multi-module architecture chip provided in the embodiments of this application; Figure 5B This is a schematic diagram illustrating the principle of the round-robin mapping method.

[0013] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0016] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0017] It should be noted that the data collection and processing in this application should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0018] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0020] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0021] 1) Network-On-Chip (NoC): This is a communication method for System-on-a-Chip (SoC). As a key component of multi-core technology, the network-on-chip architecture achieves concurrent communication through distributed routing nodes and time-division multiplexing links, replacing the traditional bus architecture to solve scalability and performance bottlenecks. Network-on-chip architectures are divided into electronically based network-on-chip (ENoC) and optically based network-on-a-chip (ONoC), employing a globally asynchronous and locally synchronous clock mechanism to reduce power consumption. Network-on-chip bandwidth is higher than bus architectures, with a lower dynamic power consumption ratio, supporting scenarios such as multi-core processors and artificial intelligence accelerators.

[0022] 2) Compute Tile: A modular and scalable basic physical building block of on-chip heterogeneous or homogeneous processing arrays. It typically integrates instruction scheduling logic, arithmetic execution engines (such as matrix multiplication units), and supporting on-chip static random access memory (SRAM) and other microarchitectural resources with high spatial locality on the physical layout.

[0023] 3) Dynamic Random Access Memory (DRAM): This is a semiconductor memory that uses capacitors to store electrical charge to retain data. DRAM is volatile; data stored cannot be retained after power is lost. Data can be lost due to capacitor leakage, therefore periodic refreshing is necessary. DRAM is characterized by its simple structure, high integration, low cost, and large capacity, and is widely used as the main memory (RAM) in computers, servers, smartphones, and other devices.

[0024] 4) Static Random-Access Memory (SRAM): This is a semiconductor memory based on bistable flip-flops. As long as power is supplied, the data stored in SRAM can be stably preserved without the need for periodic refreshing like dynamic random-access memory (DRAM). The main characteristics of SRAM are high access speed and low standby power consumption, but it has lower integration density and higher cost. It is often used in scenarios with extremely high speed requirements, such as CPU cache.

[0025] 5) Virtual address mapping: This is an address translation control logic executed by the Memory Management Unit (MMU) or the underlying runtime stage in a computer architecture. It is used to accurately convert the logical address space at the software application level into the actual physical address space in the underlying hardware medium (such as dynamic random access memory) according to preset translation rules.

[0026] 6) Interleaving: This is a storage resource management or data organization and distribution strategy. The interleaving mechanism divides logically continuous data sequences or address spaces into several data sub-units according to a specific granularity (such as pages, blocks, cache lines or bytes) through a preset mapping algorithm or sharding rules, and maps these sub-units sequentially and cyclically to multiple physically independent storage entities, storage channels or logical nodes.

[0027] 7) Runtime: In computer science, runtime refers to the operating environment, typically a process-level virtual machine that provides necessary dependency support for program execution and persists throughout the program's lifecycle. As the execution context of an application, the runtime environment abstracts away the details of the underlying hardware and operating system, providing logical isolation through the virtual machine. The runtime virtual machine is responsible for core services such as memory management, exception handling, and code-level security, and provides isolation and protection for applications through a sandbox mechanism. The runtime environment includes key components such as runtime libraries, execution engines, memory management systems, and just-in-time (JIT) compilers.

[0028] 8) Communication Pipeline: In the field of computer architecture and parallel computing, this is a parallel execution architecture for instruction scheduling and data flow. The core mechanism of the communication pipeline architecture is to break down a complete end-to-end data transmission or collection interaction task into multiple consecutive sub-stages (pipelines, such as data reading, fragmentation and packaging, network delivery, and reception acknowledgment) that can overlap on the physical time axis, so that multiple discrete data segments can be concurrently promoted in a time-division multiplexing manner in different physical execution units or logical channels.

[0029] In chips with multi-computing module architectures, communication typically requires the use of intermediate buffers allocated by the runtime virtual machine. These buffers are used to transfer data within or between devices, cache pipelined segments to mask synchronization latency, and improve overall throughput. When allocating dynamic random access memory (DRAM) space (video memory), the runtime virtual machine in related technologies often employs a DRAM mapping method with contiguous virtual addresses and tiled physical addresses or interleaving of memory controllers. While interleaving memory allocation can improve overall DRAM bandwidth utilization in some architectures, for communication buffers that frequently perform load / store operations between the static random access memory (SRAM) and DRAM within the same module, this approach leads to: physical pages of the intermediate buffer being scattered across multiple modules; frequent cross-module accesses to remote DRAM when a single module accesses the communication buffer; remote memory access latency during load / store operations between SRAM and DRAM; a significant increase in on-chip network traffic; and difficulty in constructing continuous, high-bandwidth, low-latency communication pipelines. Therefore, if the communication buffer cannot maintain local module access, it will directly lead to a decrease in communication performance, a surge in communication latency, and a reduction in computing power utilization.

[0030] This application provides a memory allocation method, a memory allocation device, a chip, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the data access efficiency of the computing module.

[0031] The memory allocation method provided in this application can be applied to chips. (Refer to...) Figure 1A , Figure 1A This is a first structural schematic diagram of a chip provided in an embodiment of this application. The chip 100 includes multiple computing modules, such as computing modules 101 to 10N, where N is an integer greater than 1, and each computing module has the same structure. Computing module 101 includes a first memory 1011 and a second memory 1012. A first buffer 10111 is allocated in the first memory 1011. Computing module 10N includes a first memory 10N1 and a second memory 10N2. A first buffer 10N11 is allocated in the first memory 10N1. In each computing module, in response to a local data access request, a data access operation is performed between the first memory and the second memory of the computing module according to the first virtual address of the first buffer of the first memory of the computing module.

[0032] In some embodiments, reference Figure 1B , Figure 1B This is a schematic diagram of the second structure of the chip provided in the embodiments of this application; Figure 1Byes Figure 1A In another embodiment of chip 100, chip 100 further includes an on-chip network 110, and computing module 101 further includes a computing unit 1013 and a load / store unit (LSU). In each computing module, in response to a local data access request, data access operations are performed between the first and second memories of the computing module via the load / store unit, based on a first virtual address of a first buffer in the first memory of the computing module. In response to a remote data access request, data access operations are performed between the first memories of different computing modules via the on-chip network, based on a first virtual address of a first buffer in the first memory of the computing module and a first virtual address of a first buffer in the first memory of another computing module.

[0033] The following describes exemplary applications of the electronic devices provided in the embodiments of this application. These electronic devices can be implemented as terminal devices, such as laptops, tablets, desktop computers, set-top boxes, smart TVs, in-vehicle terminals, virtual reality (VR) devices, augmented reality (AR) devices, and other various types of terminals. They can also be implemented as servers. The following will describe exemplary applications when the electronic device is implemented as a terminal device or a server.

[0034] See Figure 2A , Figure 2A This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Figure 2A The terminal device 400 shown includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2A The general labeled all buses as Bus System 440.

[0035] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0036] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0037] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0038] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0039] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0040] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks; The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc. Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with user interface 430 (e.g., a display screen, a speaker, etc.). The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0041] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2AA memory allocation device 455 stored in memory 450 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a buffer allocation module 4551, an address allocation module 4552, and a data access module 4553. These modules are logically linked and can therefore be arbitrarily combined or further divided according to their implemented functions. The functions of each module will be described below.

[0042] See Figure 2B , Figure 2B This is a schematic diagram of the server structure provided in an embodiment of this application. Figure 2B The server 200 shown includes at least one processor 210, memory 250, and at least one network interface 220. The various components of server 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2B The general labeled all buses as Bus System 240.

[0043] Processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0044] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 250 may optionally include one or more storage devices physically located away from the processor 210.

[0045] The memory 250 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 250 described in this application embodiment is intended to include any suitable type of memory.

[0046] In some embodiments, memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0047] Operating system 251 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks; The network communication module 252 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc. In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2B A memory allocation device 255 stored in memory 250 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a buffer allocation module 2551, an address allocation module 2552, and a data access module 2553. These modules are logically linked and can therefore be arbitrarily combined or further divided according to their implemented functions. The functions of each module will be described below.

[0048] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment.

[0049] In other embodiments, the memory allocation device provided in this application can be implemented in hardware. As an example, the memory allocation device provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the memory allocation method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0050] The memory allocation method provided in this application will be described in conjunction with exemplary applications and implementations of the terminal devices or servers provided in the embodiments of this application.

[0051] The memory allocation method provided in the embodiments of this application will be described below. As mentioned above, the electronic device implementing the memory allocation method of the embodiments of this application can be a terminal device or a server, or a combination of both. Therefore, the executing entity of each step will not be described again below.

[0052] See Figure 3A , Figure 3A This is a flowchart illustrating the memory allocation method provided in the embodiments of this application, which will be combined with... Figure 3A The steps shown are explained. Figure 3A The execution entity for the steps is either a server or a terminal device.

[0053] In step 301, a local first memory is set up for multiple computing modules.

[0054] Here, each computing module includes a first memory and a second memory.

[0055] For example, a computing module refers to a hardware processing unit in a system-on-a-chip or multi-core processor architecture that has the ability to independently execute computing instructions and route local data; the first memory is usually dynamic random access memory (DRAM), which is used to provide off-chip or in-package physical memory space for the corresponding computing module; the second memory is usually static random access memory (SRAM), which is located inside the computing module and is used as a cache to meet the high-frequency data temporary storage requirements.

[0056] For example, the chip includes multiple mesh nodes interconnected via an on-chip network. Each node serves as a computing module, which integrates a computing engine and a local memory controller. The first memory of the computing module is configured by connecting its local memory controller to a separate physical storage medium via a dedicated high-bandwidth physical interface through a point-to-point hardwired connection. This creates multiple physically isolated Non-Uniform Memory Access (NUMA) local memory domains at the hardware level, with the physical storage medium serving as the first memory.

[0057] In step 302, a first buffer is allocated to the first memory corresponding to each of the multiple computing modules.

[0058] Here, each first memory corresponds one-to-one with each first buffer.

[0059] For example, the first buffer is a local communication intermediate buffer. It serves as an intermediate storage area for data sending, receiving, or pipeline segmentation buffering when multiple computation modules perform collaborative processing (such as ensemble communication or data splitting and transfer during large model training). Computation modules write data that needs to be synchronized or interacted with into this buffer, or read data from other modules from this buffer.

[0060] In some embodiments, step 302 can be implemented by: determining a first data volume of the first buffer according to the first communication task; determining a contiguous physical storage space of the first data volume from the free physical pages of the first memory of each computing module; and using the contiguous physical storage space of the first data volume as the first buffer of the first memory.

[0061] For example, the first communication task refers to the underlying data flow process triggered between various computing modules in the process of collaborative processing of multiple computing modules (such as distributed artificial intelligence model training or large-scale data parallel computing) for data exchange, parameter synchronization or intermediate result aggregation.

[0062] For example, during the initialization or runtime virtual machine startup phase of the terminal device (or server), the underlying driver obtains the compute module identifiers (Tile IDs) of all compute modules participating in the task; iterates through the compute module identifiers, disables the global memory interleaving mapping attribute, and sends a targeted allocation instruction to the local storage controller of each compute module. For instance, it calls the underlying memory allocation interface with node affinity constraints (such as the alloc_pages_node function with modified attributes), allocates a pre-set first data amount (e.g., 64MB) of contiguous physical storage space in the first physical page frame of the memory directly connected to the compute module with Tile ID_1, registers and locks the allocated contiguous physical storage space as a one-to-one corresponding first buffer entity exclusively for the compute module Tile ID_1, and locking means preventing the operating system from performing page swapping on the allocated contiguous physical storage space.

[0063] In some embodiments, after using a contiguous physical storage space containing a first amount of data as a first buffer of a first memory, in response to receiving a second communication task, a second amount of data is determined according to the second communication task; in response to the second amount of data being greater than the first amount of data and the first buffer containing no data, the first buffer is revoked; for each computing module, a contiguous physical storage space containing the second amount of data is determined from the free physical pages of the first memory of the computing module; and the contiguous physical storage space containing the second amount of data is used as the updated first buffer.

[0064] For example, the second communication task refers to the subsequent data interaction or state synchronization operation triggered after the first communication task within the lifecycle of multi-computing module collaborative processing; the second data volume corresponds to the total number of effective payload bytes required to be transferred by the second communication task, and the second data volume being greater than the first data volume indicates that there is a need for expansion of the first memory.

[0065] For example, the parameters of the second communication task and the state of the first buffer can be obtained in the following ways: extract the communication operator features of the second communication task to be called to calculate the required second data amount; by maintaining a reference counter for the first buffer, or by polling the status register of the direct memory access (DMA) engine, detect in real time whether there is still valid data in the first buffer that has not been consumed. If there is no valid data, the first buffer can be canceled.

[0066] For example, revoking the first buffer and allocating an updated first buffer can be achieved as follows: When the second data volume is greater than the first data volume, the reference counter is reset to zero, the page release interface is called to release the first buffer, and the physical contiguous page frames of the first buffer are released and returned to the free memory pool of the local first memory directly connected to the computing module. A contiguous physical storage space is then found and locked in the memory pool and re-registered as the updated first buffer of the computing module, thereby dynamically adapting to larger-scale communication loads. For example: the calculated value of the second data volume is 128MB. The first data volume is 64MB. Since the second data volume is greater than the first data volume, the original 64MB physical contiguous page frames are released, and a new 128MB contiguous physical storage space is acquired as the updated first buffer.

[0067] In this embodiment, if there is a need for expansion, a new contiguous physical storage space can be found for each individual computing module. Compared with related technologies that require cross-module buffer modification, this reduces cross-module operations and saves network bandwidth on the on-chip network.

[0068] In step 303, multiple first virtual addresses are allocated for each first buffer.

[0069] Here, the multiple first virtual addresses corresponding to a computing module are consecutive, and the first virtual address includes the computing module identifier and the address offset inside the first buffer.

[0070] For example, the first virtual address consists of a high-order field and a low-order field, with the calculation module identifier as the high-order field and the address offset inside the first buffer as the low-order field.

[0071] In some embodiments, the first virtual address consists of a high-order field and a low-order field; step 303 is achieved through... Figure 3B Steps 3031 to 3033 are implemented, and the details are explained below.

[0072] In step 3031, the following processing is performed for each first buffer: the computation module identifier of the computation module where the first buffer is located is used as the high-order field of each first virtual address.

[0073] For example, a compute module identifier (Tile ID) refers to a unique physical topology coordinate or logical route number assigned to each individual compute module at the underlying hardware architecture level. A compute module identifier can be represented as Tile ID = N, where N is a positive integer.

[0074] In step 3032, multiple address offsets are determined based on the upper limit of the data capacity of the first buffer, and the multiple address offsets are respectively used as the low-order fields of the first virtual address.

[0075] For example, the upper limit of the data capacity of the first buffer is determined based on the size of the contiguous storage space actually physically allocated to the first buffer. Converting the upper limit of the data capacity into the corresponding byte span range yields multiple address offsets.

[0076] In step 3033, each high-order address is combined with a low-order address to obtain multiple first virtual addresses.

[0077] Here, the low-order fields of multiple first virtual addresses of multiple computing modules are encoded in the same way.

[0078] refer to Figure 4B , Figure 4B This is a schematic diagram of a virtual address provided in an embodiment of this application. The buffer of calculation module 1 includes address offsets from [0] to [7]. Any address in the buffer of calculation module 1 is represented as: [Calculation module identifier = 1, buffer address 1 + address offset i], where the value of address offset i is an integer from 0 to 7. Similarly, any address in the buffer of calculation module 2 is represented as: [Calculation module identifier = 2, buffer address 2 + address offset i], where the value of address offset i is an integer from 0 to 7. The lower-order fields of both calculation module 1 and calculation module 2 can be represented as address offsets i, where the value is an integer from 0 to 7.

[0079] In some embodiments, the memory allocation method provided in this application is applied to a chip, which includes an on-chip network and multiple computing modules; each computing module is interconnected through the on-chip network, and each computing module shares the same virtual address space. (Reference) Figure 1B , Figure 1BThis is a schematic diagram of the second structure of the chip provided in this application embodiment; the chip 100 includes: an on-chip network 110 and multiple computing modules, such as computing modules 101 to 10N, where N is an integer greater than 1. Computing module 101 includes a first memory 1011 and a second memory 1012, and a first buffer 10111 is allocated in the first memory 1011. In response to a remote data access request, data access operations are performed between the first memories of different computing modules through the on-chip network, based on the first virtual address of the first buffer of the first memory of the computing module and the first virtual addresses of the first buffers of the first memories of other computing modules.

[0080] In some embodiments, the write virtual address of the first computing module, the read virtual address of the second computing module, and the requested first target data carried in the first data access request are obtained, wherein the write virtual address is a first virtual address used to write the first target data, and the read virtual address is a first virtual address used to read the first target data; if the identifier of the first computing module in the write virtual address is different from the identifier of the second computing module in the read virtual address, the first target data is read from the read virtual address; the first target data is transmitted to the first computing module through the on-chip network, and the first target data is stored in the write virtual address of the first computing module.

[0081] For example, the first data access request refers to the low-level memory access control instruction initiated by the hardware communication engine or the main control scheduling unit during multi-module distributed processing or aggregated communication, used to move or synchronize payloads between different physical nodes; the on-chip network is used to realize cross-module message exchange and high-bandwidth data routing.

[0082] In some embodiments, when the identifier of the first computing module is determined to be inconsistent with the identifier of the second computing module, cross-module remote data reading or writing is triggered. The network interface of the second computing module reads the first target data from the first memory of the second computing module based on the low-order field of the read virtual address; the network interface of the second computing module slices the read pure data block (Payload) and adds a header containing the identifier of the first computing module, encapsulating it into a standard on-chip network transmission fragment; the transmission fragment is forwarded hop-by-hop to the first computing module through routing nodes at various levels in the on-chip network; the network receiving interface of the first computing module decapsulates the transmission fragment, recovers the original first target data, and writes the data into the first memory of the first computing module through the storage controller of the first computing module according to the address offset of the written virtual address, thereby completing the cross-module data transmission.

[0083] In step 304, data access operations are performed based on the first virtual address of the first memory of the computing module and the second memory.

[0084] For example, data access operations refer to data loading and storage operations performed between different storage levels within a computing module. For instance, intermediate results received from network communication are cached from the first memory (dynamic random access memory) to the second memory (static random access memory) for operator execution, or data that has been computed locally is written from the second memory to the first buffer of the first memory.

[0085] In some embodiments, the first memory is a dynamic random access memory (DRAM), and the second memory is a static random access memory (SRAM). Step 304 can be implemented as follows: For each computing module, perform the following processing: obtain the target virtual address carried by the second data access request and the requested second target data, wherein the target virtual address includes a target computing module identifier and a target address offset; if the target computing module identifier is the same as the computing module identifier in the first virtual address of the computing module identifier, determine the target physical address in the first memory based on the target address offset; establish a second transmission path between the physical space pointed to by the target physical address and the second memory, and transmit the second target data between the first memory and the second memory based on the second transmission path.

[0086] For example, the second data access request refers to a low-level memory access instruction initiated by the controller within the computing module, which aims to locally transfer communication temporary data or local computing results between dynamic random access memory and static random access memory.

[0087] For example, the second data access request and its target virtual address can be obtained in the following way: the load / store unit inside the computing module resolves that the second target data stored in the first buffer needs to be extracted into the static random access memory for computation; the load / store unit generates a direct memory access (DMA) task descriptor containing the above-mentioned target virtual address, and submits the task descriptor to the on-chip bus master to generate the second data access request.

[0088] In some embodiments, when the identifiers are the same, it indicates that the data transmission occurs within the same technical module. The data transmission is implemented in the following specific manner: the target address offset of the target virtual address is added to the physical base address of the first buffer in the first memory to accurately calculate the corresponding target physical address. A dedicated internal data transmission bus path (i.e., a second transmission path) is established between the controller of the first memory and the interface of the second memory via a system-level interconnect bus (such as Advanced Extensible Interface AXI or Consistent Hub Interface CHI) within the computing module. Based on a direct memory access transmission protocol, the second target data is transmitted between the first memory and the second memory.

[0089] In this embodiment, by setting a first buffer in the local first memory of each computing module, the physical carrier of data is completely confined within the computing module when the computing module performs communication or intermediate data caching. This avoids the need for remote memory access across the on-chip network to obtain data, reduces the physical routing latency of a single read / write operation, saves data packet traffic injected into the on-chip network due to cross-module access, and effectively reduces the probability of network node congestion. By constructing a virtual address space by combining the computing module identifier with the internal address offset, during address resolution, the physical node corresponding to the address can be accurately determined simply by extracting the computing module identifier of the address, simplifying the address translation and addressing logic. At the same time, allocating consecutive virtual addresses to a single computing module ensures the data space locality of the computing module during data throughput, effectively improving the bandwidth utilization of the storage bus for continuous batch read / write operations. Based on the above buffer and the corresponding virtual address, data access operations are performed between the first memory and the second memory, making the physical path of the access operation fixed and unique. Compared to the memory access latency jitter caused by multi-hop routing in related technologies, the data interaction between the local first memory and the second memory can effectively prevent the data processing pipeline inside the computing module from being interrupted or stalled due to waiting for remote data synchronization, thereby improving the data transmission efficiency in a multi-computing module parallel environment.

[0090] The following will describe an exemplary application of the memory allocation method of this application in a real-world application scenario.

[0091] In chips with multi-computing module architectures, communication typically requires the use of intermediate buffers allocated by the runtime virtual machine. These buffers are used to transfer data within or between devices, cache pipelined segments to mask synchronization latency, and improve overall throughput. When allocating dynamic random access memory (DRAM) space, also known as video memory, the runtime virtual machine in related technologies often employs a DRAM mapping method where virtual addresses are contiguous and physical addresses are allocated by tile or memory controller interleaving.

[0092] While this approach can improve overall dynamic random access memory (DRAM) bandwidth utilization in some architectures, for communication buffers that frequently perform load / store operations between their local static random access memory (SRAM) and DRAM, this mapping method leads to several problems: the physical pages of the buffer are scattered across multiple modules; a single module frequently accesses remote DRAM across modules when accessing the communication buffer, resulting in remote memory access latency between SRAM and DRAM; a significant increase in on-chip network traffic; and difficulty in constructing continuous, high-bandwidth, low-latency communication pipelines. Therefore, if the communication buffer cannot maintain tile-local access, it will directly lead to degraded communication performance, a surge in communication latency, and reduced computational efficiency.

[0093] In multi-module computing chips, dynamic random access memory (DRAM) typically allocates physical addresses in the following way: DRAM physical pages are rotated among multiple modules at a fixed granularity, resulting in contiguous virtual addresses but dispersed physical addresses. This method is barely usable for computing cores, but it has the following drawbacks for communication buffers: (1) Physical memory pages are distributed across dynamic random access memory in different modules; buffer loading / storage no longer has module locality; a large number of remote DRAM accesses are inevitable.

[0094] (2) Cross-module access leads to high latency and bandwidth degradation. This is because the communication buffer serves the following purposes in communication: segmented caching; on-chip pipelined processing; and inter-node staging buffer. Cross-module access involves loading / storing very frequently. If the buffer is distributed across modules, then: each access may access the dynamic random access memory of the remote module through the on-chip network; on-chip network congestion has an amplifying effect; and the communication pipeline is damaged, making it impossible to mask synchronization delays.

[0095] (3) Lack of a dedicated tile-local allocation mechanism optimized for communication paths. The allocation strategies of most runtime virtual machines are designed to improve global bandwidth rather than to improve the consistent access latency of a single-module communication pipeline. Therefore, related technologies cannot support: allocating an exclusive buffer in dynamic random access memory for each tile to temporarily store the geometry data, parameters, or intermediate results required for module rendering; and tile-local access during load / store operations between static random access memory and dynamic random access memory.

[0096] In this application embodiment, addressing the aforementioned problems of related technologies, a memory allocation method is provided. This method constructs a dynamic random access memory (DRAM) memory allocation process for a multi-computing module chip architecture, managing internal communication buffers, and performing global virtual address mapping on the DRAM buffers. By providing tile-local buffers, the cross-module access overhead between static random access memory and DRAM is reduced, on-chip network pressure is decreased, and communication pipeline efficiency and data access locality are improved. The aim is to solve the problem in multi-computing module chips where the internal buffers used for communication are distributed across different modules in DRAM, leading to increased access latency, heavier internal communication load, and decreased communication pipeline efficiency.

[0097] To facilitate understanding, we will first introduce the basic storage structure of multi-module chips. (Reference) Figure 5A , Figure 5A This is a schematic diagram of the multi-module architecture chip provided in the embodiments of this application. Figure 5A This demonstrates the basic memory and memory access path structure of a multi-module architecture chip. The multi-module architecture chip includes multiple computing modules, an on-chip network 505A, and dynamic random access memory. Figure 5A Taking a scenario with four computing modules as an example, such as computing module 501A, computing module 502A, computing module 503A, and computing module 504A. Assume the on-chip network has four dynamic random access memories (DRAMs), such as DRAM 5051A, DRAM 5052A, DRAM 5053A, and DRAM 5054A. DRAMs serve as the chip's main memory, used to store large-scale model data and runtime virtual machine buffers. However, DRAM access latency is higher than on-chip static random access memory (SRAM), and access requires traversing the on-chip network.

[0098] Each computing module has the same structure. Taking computing module 501A as an example, computing module 501A includes: a computing unit 5011A, static random access memory (SRAM) 5012A, and a load / store unit (LSU) 5013A. Computing unit 5011A is used to execute operators and communication tasks. Static random access memory 5012A is local on-chip storage used for high-speed working cache within the computing module. Load / store unit 5013A is used to transfer data between static random access memory and dynamic random access memory (DRAM).

[0099] Computing module 501A corresponds to dynamic random access memory 5051A, computing module 502A corresponds to dynamic random access memory 5052A, computing module 503A corresponds to dynamic random access memory 5053A, and computing module 504A corresponds to dynamic random access memory 5054A; that is, each computing module corresponds to an independent dynamic random access memory, and the dynamic random access memory spaces of all computing modules are interconnected through an on-chip network.

[0100] In related technologies, memory allocation mostly adopts a round-robin mapping strategy, where each fixed-size address block of a logical address is sequentially allocated to the local dynamic random access memory of different modules, resulting in logically contiguous data physically spanning multiple computing modules. The round-robin mapping method causes a large number of cross-module remote dynamic random access memory accesses when a single module accesses this type of memory, leading to high access latency and on-chip network load.

[0101] refer to Figure 5B , Figure 5B This is a schematic diagram illustrating the principle of the round-robin mapping method. When a program requests a contiguous block of virtual memory ( Figure 5B When using 1024KB as an example for explanation, the address will be in a fixed granularity ( Figure 5B (Using 32KB as an example for illustration) in multiple modules (e.g.) Figure 5A The rotational mapping between the dynamic random access memories of computing modules 501A, 502A, 503A, and 504A. Figure 5B It can be seen that virtual addresses [0] to

[31] are logically continuous, with each address corresponding to 32KB of storage space. However, the physical memory pages are mapped to the dynamic random access memory of multiple computing modules, such as computing module 501A, computing module 502A, computing module 503A, and computing module 504A. For example, virtual address [0] is mapped to address [0] of computing module 501A, virtual address [1] is mapped to address [0] of computing module 502A, virtual address [2] is mapped to address [0] of computing module 503A, virtual address [3] is mapped to address [0] of computing module 504A, and so on.

[0102] When a computing module needs to perform data transmission and reception or network communication, the communication library software or task scheduler running on the computing module will send a request to the underlying memory management unit (MMU) or operating system kernel. If the requested buffer size exceeds the fixed granularity (32KB), for example, if the buffer size requested by computing module 501A is 96KB, and the virtual address [0] corresponding to computing module 501A is only 32KB, then it is necessary to access the consecutive virtual addresses [1] to [3] corresponding to computing modules 502A, 503A and 504A respectively to obtain a buffer of 96KB size with consecutive physical addresses. Every time the computing module loads / stores the buffer, it will access the remote dynamic random access memory of different modules, and the on-chip network is forced to frequently transfer between modules, which will affect the performance.

[0103] In this embodiment, a symmetrical module-local memory allocation mechanism is adopted to synchronously allocate local dynamic random access memory storage space of consistent size and symmetrical address structure across multiple computing modules. (Reference) Figure 4A , Figure 4A This is a schematic diagram illustrating the memory allocation principle provided in an embodiment of this application. The multi-computing module chip includes four computing modules: computing module 401A, computing module 402A, computing module 403A, and computing module 404A. Each computing module includes a local dynamic random access memory (DRAM). For example, computing module 401A includes DRAM 405A, computing module 402A includes DRAM 406A, computing module 403A includes DRAM 407A, and computing module 404A includes DRAM 408A. A unified memory allocation operation is performed on each DRAM. DRAM 405A is allocated buffer 4051A, DRAM 406A is allocated buffer 4061A, DRAM 407A is allocated buffer 4071A, and DRAM 408A is allocated buffer 4081A.

[0104] Through a local memory allocation mechanism, each computing module receives an independent buffer for intermediate data storage and transfer within the communication pipeline. The addresses of these independent buffers are encoded using both the computing module address and its internal address, with the computing module address used as the high-order address and the internal address as the low-order address. The allocated buffers have a consistent low-order address layout across the global virtual address space, differing only in the module identification information at the high-order addresses. Therefore, each computing module can access its corresponding buffer locally without needing to access the dynamic random access memory (DRAM) bound to other modules via the on-chip network, thereby reducing communication latency and on-chip network load.

[0105] In this embodiment of the application, under the local memory allocation mechanism, a single memory allocation operation can obtain the buffer space of the number of modules. Since each computing module is an independent parallel computing resource, when multiple modules need to participate in collaborative processing at the same time in certain computing or communication stages, each computing module can directly use the local buffer corresponding to its own module generated in this application, ensuring that all modules have locally accessible low-latency storage space when executing in parallel, and avoiding performance loss caused by remote access.

[0106] In multi-computing module chip architectures, communication often requires frequent reading and writing of data between the static random access memory (SRAM) and dynamic random access memory (DRAM) of the corresponding module. If the physical memory of the intermediate buffer is not located within the module but distributed across other modules, each access must traverse the chip's internal data channel, resulting in increased latency, unstable bandwidth, and impacting the continuity and overall efficiency of the communication process. In this embodiment, the communication intermediate buffer is allocated as much as possible within the module's local DRAM, achieving better locality of memory access and avoiding unnecessary cross-module accesses. This local module allocation method enables stable and fast data reading and writing of the intermediate buffer, avoiding increased latency caused by accessing remote modules; reduces the occupancy of internal data paths, mitigates mutual interference between different modules, and improves the overall smoothness of communication; and enhances the pipelined capability of aggregated communication, making data transfer and queuing stages more continuous and efficient.

[0107] In some embodiments, it is assumed that the chip includes four parallel computing modules (module 1-module 4), each module has a computing core, these computing modules are parallel resources, each computing module has an independent dynamic random access memory controller and its own bound dynamic random access memory, all computing modules are interconnected through an on-chip network and share the same virtual address space, each computing module only needs to go through the local path between itself and the dynamic random access memory controller when accessing its own dynamic random access memory; computing modules need to go through a multi-hop path of the on-chip network when accessing the dynamic random access memory of other modules.

[0108] Suppose the chip needs to perform internal communication processes involving numerous intermediate buffers. To ensure overall performance, each computing module needs to access a buffer region with identical structure during the execution of communication-related tasks. Assume that each computing module's accessed buffer should reside in its own local dynamic random access memory (DRAM), and multiple modules can use this buffer in parallel. The chip's underlying runtime virtual machine provides a symmetric local memory allocation mechanism for multi-module synchronization, allocating DRAM storage space of the same size to multiple computing modules.

[0109] The local memory allocation mechanism uniformly addresses the storage space within the global virtual address space. Its address structure includes a computing module identifier and an address offset within the computing module. The computing module identifier is used as the high-order address, and the address offset within the computing module is used as the low-order address. The computing module identifier indicates the computing module to which the target storage space belongs, and the address offset within the computing module represents the local address location within that computing module. An example is provided using computing modules 1 and 2 in a chip. (Reference) Figure 4B , Figure 4B This is a schematic diagram of the virtual address provided in the embodiments of this application. The buffer of the calculation module 1 includes address offsets from [0] to [7]. Any address in the buffer of the calculation module 1 is represented as: [Calculation module identifier = 1, buffer address 1 + address offset i], where the value of address offset i is an integer from 0 to 7. Similarly, any address in the buffer of the calculation module 2 is represented as: [Calculation module identifier = 2, buffer address 2 + address offset i], where the value of address offset i is an integer from 0 to 7. It can be seen that when the address offset within the calculation module is used as the low-order address, the local storage space of different calculation modules has a consistent low-order address layout.

[0110] Continue to refer to Figure 4BIf computing module 1 needs to access a certain location in the local storage space of the dynamic random access memory of computing module 2, then modifying the computing module identifier in the address [computing module identifier=1, buffer address 1+address offset i] to "computing module identifier=2" to form the address [computing module identifier=2, buffer address 2+address offset i] will allow access to a certain location in the local storage space of the dynamic random access memory of computing module 2.

[0111] In some embodiments, assuming the first communication task and the second communication task are executed sequentially, a buffer storage space of local dynamic random access memory (DRAM) for a first data amount is allocated for each computing module according to the buffer size required by the first communication task. In response to the second communication task requiring a buffer storage space for a second data amount that is larger than the first data amount, a buffer storage space of local DRAM for a second data amount is allocated for each computing module before executing the second communication task. The runtime virtual machine first dealslocates the original local buffer for the first data amount and then searches for a contiguous physical page of the second data amount in the local physical main memory (per-Tile DRAM) directly connected to the computing module (Tile). Since the physical capacity of the local DRAM corresponding to each module is huge (typically at the GB level), the runtime virtual machine can ensure that the new space for the second data amount is still confined within the local physical boundary of the current module. Although the size of the buffer changes, the global virtual address mapping mechanism provided in this embodiment remains effective: the high-order address (computing module identifier) ​​remains unchanged because the second data amount is still in the local main memory of the same module, and the computing module identifier in the virtual address is still anchored to the hardware ID of the module. The range of low-order addresses is expanded, extending the resolution range of address offsets within the calculation module from the bit width corresponding to the first data volume to the bit width corresponding to the second data volume. For the hardware address resolution unit, this modification only changes the entries in the page table mapping; the local routing nature of addressing remains unchanged.

[0112] In related technologies, if the first data volume is modified to a second data volume, due to the rigid rule of the round-robin memory allocation mechanism that forces the switching of computing modules between each storage address, the increased data volume of the second data volume compared to the first data volume must be allocated to the physical dynamic random access memory of other modules. This means that the modified buffer is physically discontinuous, and when the computing module accesses the subsequently increased data volume, the on-chip network load will instantly surge from 0%, and the communication latency will also increase dramatically. However, in the memory allocation method provided in this application embodiment, the physical location of the increased data volume remains local. When the computing module accesses all the data of the second data volume, its hardware read / write requests are still intercepted on the local bus and never enter the on-chip network.

[0113] Using the above allocation method, the multiple local storage spaces obtained on multiple modules have a consistent low-order address layout, differing only in the high-order module identifier. Any module can directly access the local storage space located in its own dynamic random access memory through its own local address resolution rules; at the same time, by modifying the module identifier information in the address, the corresponding storage instances on different modules can be distinguished at the address level.

[0114] The memory allocation methods used in related technologies do not constrain the module affiliation of buffers. The physical storage pages corresponding to intermediate buffers are distributed across the dynamic random access memory (DRAM) bound to multiple computing modules. This causes each computing module to frequently access the DRAM storage resources of remote modules through the on-chip network when performing communication-related operations. However, the memory allocation method provided in this application adopts a symmetric module-local storage allocation mechanism, ensuring that each computing module obtains an intermediate buffer entirely located in its local DRAM. Because each buffer maintains a consistent low-order address encoding method and distinguishes high-order addresses through a computing module identifier (id), each computing module only accesses its own local storage space during communication, avoiding access to remote DRAM across computing modules.

[0115] In this embodiment of the application, by providing a local (Tile-local) intermediate buffer for aggregated communication in a multi-module chip, the following technical effects can be achieved: (1) Significantly reduced access latency: The intermediate buffer is located entirely in the dynamic random access memory of this module, without needing to access the storage resources of other modules, resulting in smaller and more stable data read and write latency.

[0116] (2) Improve the continuity and locality of data access: All physical space of the buffer is exclusively occupied by this module, and the access path is fixed and consistent, avoiding data jumps and discontinuities caused by physical dispersion.

[0117] (3) Reduce the occupation and conflict of on-chip network: There is no need to perform data transfer across modules, which greatly reduces the use of on-chip network, making the overall communication smoother and less conflicting.

[0118] (4) Enhance the smoothness of communication pipeline: Because the intermediate buffer has lower latency and more stable access, communication pipeline is less likely to be interrupted during transmission and can more effectively mask synchronization delay.

[0119] (5) Improve the overall bandwidth utilization of aggregated communication: After data access is maintained by local modules (Tile-local), the bandwidth utilization is higher and is less likely to decrease due to cross-module contention for resources, thereby improving communication efficiency.

[0120] The following description continues to illustrate the exemplary structure of the memory allocation device 455 provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 2A As shown, the software modules stored in the memory allocation device 455 of the memory 450 may include: A buffer allocation module 4551 is used to set up local first memory for multiple computing modules, wherein each computing module includes a first memory and a second memory; allocate a first buffer to the first memory corresponding to each of the multiple computing modules, with each first memory corresponding to each first buffer in a one-to-one correspondence; an address allocation module 4552 is used to allocate multiple first virtual addresses for each first buffer, wherein the multiple first virtual addresses corresponding to one computing module are consecutive, and the first virtual address includes a computing module identifier and an address offset within the first buffer; a data access module 4553 is used to perform data access operations based on the first virtual address of the first memory of the computing module and the second memory.

[0121] In some embodiments, the memory allocation device is applied to a chip, the chip including an on-chip network and a plurality of computing modules; each computing module is interconnected through the on-chip network and shares the same virtual address space.

[0122] In some embodiments, the data access module 4553 is configured to acquire a write virtual address of a first computing module, a read virtual address of a second computing module, and the requested first target data carried in a first data access request, wherein the write virtual address is a first virtual address used to write the first target data, and the read virtual address is a first virtual address used to read the first target data; if the identifier of the first computing module in the write virtual address is different from the identifier of the second computing module in the read virtual address, the first target data is read from the read virtual address; the first target data is transmitted to the first computing module through the on-chip network, and the first target data is stored in the write virtual address of the first computing module.

[0123] In some embodiments, the buffer allocation module 4551 is configured to determine a first data volume of the first buffer according to a first communication task; for each computing module, determine a contiguous physical storage space of the first data volume from a free physical page of the first memory of the computing module; and use the contiguous physical storage space of the first data volume as the first buffer of the first memory.

[0124] In some embodiments, the buffer allocation module 4551 is configured to, after using the contiguous physical storage space of the first data amount as the first buffer of the first memory, determine a second data amount in response to receiving a second communication task; cancel the first buffer in response to the second data amount being greater than the first data amount and the first buffer containing no data; determine a contiguous physical storage space of the second data amount from the free physical pages of the first memory of each computing module; and use the contiguous physical storage space of the second data amount as the updated first buffer.

[0125] In some embodiments, the first virtual address consists of a high-order field and a low-order field; the address allocation module 4552 is configured to perform the following processing for each of the first buffers: The computation module identifier of the computation module where the first buffer is located is used as the high-order field of each first virtual address; multiple address offsets are determined according to the upper limit of the data capacity of the first buffer, and the multiple address offsets are used as the low-order fields of the first virtual addresses respectively; each high-order address is combined with the low-order address to obtain the multiple first virtual addresses, wherein the low-order fields of the multiple first virtual addresses of the multiple computation modules are encoded in the same way.

[0126] In some embodiments, the data access module 4553 is configured to perform the following processing for each of the computing modules: The system acquires the target virtual address carried in the second data access request and the requested second target data, wherein the target virtual address includes a target computing module identifier and a target address offset; if the target computing module identifier is the same as the computing module identifier in the first virtual address of the computing module identifier, the system determines the target physical address in the first memory based on the target address offset; a second transmission path is established between the physical space pointed to by the target physical address and the second memory, and the second target data is transmitted between the first memory and the second memory based on the second transmission path.

[0127] In some embodiments, the first memory is a dynamic random access memory (DRAM) and the second memory is a static random access memory (SRAM).

[0128] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the memory allocation method described above in this application.

[0129] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the memory allocation method provided in this application, for example... Figure 3A The memory allocation method is shown.

[0130] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0131] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0132] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0133] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0134] In summary, by setting a first buffer in the local first memory of each computing module through the embodiments of this application, the physical carrier of data is completely confined within the computing module when the computing module performs communication or intermediate data caching. This avoids the need for remote memory access across the network-on-chip (NoC) to obtain data, reduces the physical routing latency of a single read / write operation, saves data packet traffic injected into the network-on-chip due to cross-module access, and effectively reduces the probability of network node congestion. By constructing a virtual address space by combining the computing module identifier with the internal address offset, the physical home node corresponding to the address can be accurately determined by extracting the computing module identifier of the address during address resolution, simplifying the address translation and addressing logic. At the same time, allocating continuous virtual addresses to a single computing module ensures the data space locality of the computing module during data throughput, effectively improving the bandwidth utilization of the storage bus for continuous batch read / write. Based on the above buffer and the corresponding virtual address, data access operations are performed between the first memory and the second memory, making the physical path of the access operation fixed and unique. Compared to the memory access latency jitter caused by multi-hop routing in related technologies, the data interaction between the local first memory and the second memory can effectively prevent the data processing pipeline inside the computing module from being interrupted or stalled due to waiting for remote data synchronization, thereby improving the data transmission efficiency in a multi-computing module parallel environment.

[0135] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A memory allocation method, characterized in that, The method includes: A local first memory is set up for multiple computing modules, wherein each computing module includes a first memory and a second memory; A first buffer is allocated to the first memory corresponding to each of the plurality of computing modules, and each first memory corresponds one-to-one with each first buffer. Multiple first virtual addresses are allocated for each first buffer, wherein the multiple first virtual addresses corresponding to one computing module are consecutive, and the first virtual address includes a computing module identifier and an address offset within the first buffer; Data access operations are performed based on the first virtual address of the first memory of the computing module and the second memory.

2. The method according to claim 1, characterized in that, The method is applied to a chip, which includes an on-chip network and multiple computing modules; each computing module is interconnected through the on-chip network and shares the same virtual address space.

3. The method according to claim 2, characterized in that, The method further includes: The system obtains the write virtual address of the first computing module, the read virtual address of the second computing module, and the requested first target data carried in the first data access request, wherein the write virtual address is a first virtual address used to write the first target data, and the read virtual address is a first virtual address used to read the first target data; If the identifier of the first computing module written to the virtual address is different from the identifier of the second computing module read from the virtual address, the first target data is read from the virtual address. The first target data is transmitted to the first computing module via the on-chip network, and the first target data is stored in the write virtual address of the first computing module.

4. The method according to claim 1, characterized in that, The allocation of a first buffer to the first memory corresponding to each of the plurality of computing modules includes: Based on the first communication task, determine the first data volume of the first buffer; For each computing module, a contiguous physical storage space for a first amount of data is determined from the free physical pages of the first memory of the computing module; The contiguous physical storage space of the first data volume is used as the first buffer of the first memory.

5. The method according to claim 4, characterized in that, After using the contiguous physical storage space of the first data volume as the first buffer of the first memory, the method further includes: In response to receiving a second communication task, a second data volume is determined based on the second communication task; In response to the second data volume being greater than the first data volume, and the first buffer containing no data, the first buffer is revoked; For each computing module, a contiguous physical storage space for the second data amount is determined from the free physical pages of the first memory of the computing module; Use the contiguous physical storage space of the second data volume as the updated first buffer.

6. The method according to claim 1, characterized in that, The first virtual address consists of a high-order field and a low-order field; The allocation of multiple first virtual addresses for each of the first buffers includes: For each of the first buffers, the following processing is performed: The computation module identifier of the computation module where the first buffer is located is used as the high-order field of each first virtual address; Multiple address offsets are determined based on the upper limit of the data capacity of the first buffer, and the multiple address offsets are respectively used as the low-order field of the first virtual address; Each of the high-order addresses is combined with the low-order addresses to obtain the plurality of first virtual addresses, wherein the low-order fields of the plurality of first virtual addresses of the plurality of computing modules are encoded in the same way.

7. The method according to claim 1, characterized in that, The data access operation based on the first virtual address of the first memory and the second memory of the computing module includes: For each of the aforementioned computing modules, the following processing is performed: Obtain the target virtual address carried in the second data access request and the requested second target data, wherein the target virtual address includes the target computing module identifier and the target address offset; If the target computing module identifier is the same as the computing module identifier in the first virtual address of the computing module identifier, the target physical address in the first memory is determined based on the target address offset; A second transmission path is established between the physical space pointed to by the target physical address and the second memory, and the second target data is transmitted between the first memory and the second memory based on the second transmission path.

8. The method according to claim 7, characterized in that, The first memory is a dynamic random access memory, and the second memory is a static random access memory.

9. A memory allocation device, the device comprising: A buffer allocation module is used to set up local first memory for multiple computing modules, wherein each computing module includes a first memory and a second memory; and to allocate a first buffer to the first memory corresponding to each of the multiple computing modules, wherein each first memory corresponds one-to-one with each first buffer. The address allocation module is used to allocate multiple first virtual addresses for each first buffer, wherein the multiple first virtual addresses corresponding to one calculation module are consecutive, and the first virtual address includes a calculation module identifier and an address offset within the first buffer. The data access module is used to perform data access operations based on the first virtual address of the first memory of the computing module and the second memory.

10. A chip, characterized in that, The chip includes multiple computing modules and an on-chip network. Each computing module includes a first memory and a second memory. The first memory is a dynamic random access memory, and the second memory is a static random access memory. The chip is used to implement the memory allocation method according to any one of claims 1 to 8.

11. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the memory allocation method according to any one of claims 1 to 8.

12. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the memory allocation method according to any one of claims 1 to 8.

Images