Method and system for optimizing access to constant memory

Abstract

The disclosed systems, structures, and methods are directed to optimizing memory access to constants in heterogeneous parallel computers, including OpenCL-enabled systems. This is implemented in an optimizing compiler that converts program-scoped constants and constants scoped to the outermost level of a kernel to implicit constant pointer parameters. The optimizing compiler also attempts to determine access patterns for constants at compile time and places the constants in various memory types available in the compute device architecture based on these access patterns.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. nonprovisional application No. 17 / 000,781, filed August 24, 2020, entitled “METHOD AND SYSTEM FOR OPTIMIZING ACCESS TOCONSTANT MEMORY”, the contents of which are incorporated herein by reference in any jurisdiction that allows such consolidation. Technical Field

[0003] This invention generally relates to the field of memory access optimization in heterogeneous parallel architectures, and more specifically, to compile-time methods and systems for optimizing access to constant memory in heterogeneous parallel architectures. Background Technology

[0004] For example, current computing architectures used in artificial intelligence, data science, and other applications typically utilize one or more central processing units (CPUs), often with multiple parallel cores, as well as special-purpose processing units such as graphics processing units (GPUs), digital signal processors (DSPs), tensor processing units (TPUs), field-programmable gate arrays (FPGAs), and other types of processors, which may include dozens or even hundreds of dedicated processors, all of which can run in parallel. This type of parallel architecture, which may contain many different types of processors, is called a heterogeneous parallel architecture.

[0005] Developing software to run on heterogeneous parallel architectures has always been a challenge. The various processors used can be based on different computational models and may have different types of memory, among other things. Typically, vendors producing special-purpose processors for heterogeneous parallel systems also provide specialized software development tools, such as compilers, libraries, dedicated assemblers, and other tools for their hardware. Therefore, it is often necessary to rewrite critical blocks of code to migrate from one heterogeneous parallel computing architecture to another.

[0006] OpenCL is an attempt to address these challenges by providing a public and open framework for writing programs that execute across these heterogeneous parallel computing platforms. OpenCL provides a computation and memory model, offering standardized interfaces for parallel computing, including task- and data-based parallelism.

[0007] One type of memory used in the OpenCL memory model is called "constant memory." Constant memory stores constants used in OpenCL programs, and parallel kernels using these constants cannot write to them. Constants stored in constant memory are typically arrays of values, accessed using access indices that access the various elements of the array. Constant memory in OpenCL is often part of global memory, which is accessible to all parallel kernels of the program and is generally relatively slow. In some architectures, dedicated processing units may include special on-chip constant memory and / or constant caches to speed up access to constants. However, these features are not available on all architectures. While most special-purpose architectures include various types of memory that may offer faster access than global memory, these other types of fast memory are generally not used for storing constants. Summary of the Invention

[0008] Advantageously, the present invention provides an optimizer for compilers to allow constants to be stored in various types of memory in heterogeneous parallel architectures, including fast and medium-speed memory that are not typically used for storing constants. Furthermore, the disclosed techniques allow constants to be stored and accessed in special-purpose processing units that do not include on-chip constant memory or constant caches. This can significantly improve the memory performance of heterogeneous parallel computers when accessing constants. While examples of the invention are given for OpenCL, it should be understood that the disclosed techniques can also be used to improve the speed of memory access to constants in other heterogeneous parallel architectures.

[0009] According to one aspect of the invention, the technology is implemented in an apparatus including a processor, memory coupled to the processor, and a compiler residing in the memory and executed by the processor. The compiler includes a constant memory optimizer, comprising: a scope determination module for determining the scope of the constant in the code of a heterogeneous parallel computer; the constant memory optimizer further includes: a code translation module for, if the scope of the constant is the program scope of the kernel in the code referencing the constant or the outermost scope, translating the code to pass the constant as an implicit pointer parameter to the kernel.

[0010] In some implementations, the constant memory optimizer further includes: an access mode determination module for determining the access mode of the constant in the kernel; and a memory type determination module for selecting the memory type to be used for the constant based on the access mode. In some implementations, the code transformation module is further used to transform the code to use the selected memory type for the constant.

[0011] According to other aspects of the present invention, a method for optimizing access to constants in a heterogeneous parallel computer is provided. The method includes: determining the scope of the constant in the code of the heterogeneous parallel computer using a scope determination module; and, if the scope of the constant is the program scope or the outermost scope of the kernel using a code transformation module, transforming the code to pass the constant as an implicit pointer parameter to the kernel in the code referencing the constant using the code transformation module. The method further includes: determining the access mode of the constant in the kernel using an access mode determination module, and selecting a memory type to be used for the constant based on the access mode using a memory type determination module. Transforming the code further includes transforming the code to use the selected memory type for the constant.

[0012] In some implementations of the method, in a first use case where the scope of the constant is the program scope or the outermost scope of the kernel, determining the access mode of the constant includes determining the size of the constant. These selections of the memory type to be used for the constant may include: selecting fast memory for the constant in a small-size sub-use case where the size of the constant is small enough to fit in fast memory; selecting medium-speed memory for the constant in a known access range sub-use case where the size of the constant is too large to fit in fast memory and the access mode includes a known access index range at compile time; and selecting a normal memory type for the constant in a default sub-use case where neither the small-size sub-use case nor the known access range sub-use case applies.

[0013] In some implementations of the method, in a second use case where the constant is a constant pointer parameter, two sub-use cases can be used to select the memory type to be used for the constant. These two sub-use cases include: selecting fast memory or medium-speed memory in the constant pointer known access sub-case, wherein fast memory is selected for the known access index range of the constant if the access pattern includes the known access index range at compile time and the size of the known access index range is small enough to fit in the fast memory, and medium-speed memory is selected for the known access index range of the constant if the access pattern includes the known access index range at compile time and the size of the known access index range is too large to fit in the fast memory; and selecting a normal memory type for the constant value in the constant pointer unknown access sub-case if the access pattern does not include the known access index range at compile time.

[0014] According to another aspect of the invention, the disclosed technique can be implemented as instructions encoded on a computer-readable medium. When these instructions are executed by a processor, they cause the processor to perform operations, including determining the scope of constants in heterogeneous parallel computer code and determining the size and access mode of constants in the kernel of the code. The instructions also cause the processor to select the memory type to be used for the constants based on the access mode.

[0015] When the scope of the constant is the kernel's program scope or the outermost scope, the instruction causes the processor to select a memory type based on several sub-use cases. If the constant is small enough to fit in fast memory, the processor selects fast memory for the constant in the "small size" sub-use case. If the constant is too large to fit in fast memory, and the access mode includes a known access index range at compile time, the processor selects medium-speed memory for the constant in the "program scope known access range" sub-use case. Finally, if neither the small size sub-use case nor the program scope known access range sub-use case applies, the processor selects the normal memory type for the constant in the "default" sub-use case.

[0016] In the second case, where the constant is a constant pointer parameter, the size of the constant is unknown. Therefore, the instruction causes the processor to select the memory type of the constant based on different sub-use cases. In the "constant pointer known access" sub-use case, the instruction causes the processor to select the known access index range for the constant if the access pattern includes a known access index range at compile time, and the size of the known access index range is small enough to fit in the fast memory. If the access pattern includes the known access index range at compile time, and the size of the known access index range is too large to fit in fast memory, medium-speed memory is selected for the known access index range of the constant. In the "constant pointer unknown access" sub-use case, the instruction causes the processor to select the normal memory type for the constant value if the access pattern does not include a known access index range at compile time.

[0017] Ultimately, the instruction causes the processor to translate the code to pass the constant as an implicit pointer parameter to the kernel in the code referencing the constant, and to use the selected memory type of the constant, if the scope of the constant is the program scope of the kernel using the code translation module or the outermost scope.

[0018] It should be understood that in any of these aspects of the invention, the heterogeneous parallel computer may support OpenCL, and the conversion code may include code that converts to an OpenCL intermediate representation, such as SPIR-V (see below) and / or LLVM IR (see below). Furthermore, in any of these aspects, "fast" memory may include registers in the computing device, while "medium-speed" memory may include on-chip memory in the computing device, such as shared or local memory.

[0019] Brief description of the attached figures

[0020] The features and advantages of the invention will be readily understood by reading the following detailed description taken in conjunction with the accompanying drawings, in which:

[0021] Figure 1 This is a block diagram of a computer system including a compiler that implements the constant memory optimization of the present invention;

[0022] Figure 2 (Prior Art) is a block diagram of a heterogeneous parallel computing system conforming to the OpenCL model;

[0023] Figure 3 (Prior Art) shows the memory model of a heterogeneous parallel computing system compliant with OpenCL;

[0024] Figure 4 (Prior Art) illustrates a memory hierarchy for accelerating access to constant memory in an architecture that includes on-chip constant memory and constant cache.

[0025] Figure 5 A compiler including the constant memory optimizer of the present invention is shown;

[0026] Figure 6 A flowchart illustrating the operation of the constant memory optimizer of the present invention is shown;

[0027] Figure 7A and Figure 7B Example code written in the OpenCL C programming language is shown to illustrate the scope of constants that may be subject to the constant memory optimization of this invention;

[0028] Figure 8A and Figure 8B This demonstrates how the constant memory optimizer of this invention can select which type of constant memory based on the constant access pattern.

[0029] It should be noted that in all the accompanying drawings and corresponding descriptions, the same features are identified by the same reference numerals. Furthermore, it should be understood that the drawings and the following description are for illustrative purposes only, and this disclosure is not intended to limit the scope of the claims. Detailed Implementation

[0030] Various representative embodiments of the disclosed technology will now be described more fully with reference to the accompanying drawings. However, this technology can be embodied in many different forms and should not be construed as limited to the representative embodiments set forth herein. In the drawings, the dimensions and relative dimensions of layers and regions may be exaggerated for clarity. Throughout the specification, similar numerals refer to similar elements.

[0031] It should be understood that although the terms first, second, third, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used to distinguish one element from another. Therefore, without departing from the teachings of the invention, the first element discussed below may be referred to as the second element. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0032] It should be understood that when one element is referred to as "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there may be an intermediate element. Conversely, when one element is referred to as "directly connected" or "directly coupled" to another element, there is no intermediate element. Other terms used to describe the relationship between elements (e.g., "between" and "directly between," "adjacent" and "directly adjacent," etc.) should be interpreted in a similar manner. Furthermore, it will be understood that elements can be "coupled" or "connected" mechanically, electrically, communicatively, wirelessly, optically, etc., depending on the type and nature of the elements being coupled or connected.

[0033] The terminology used herein is for describing specific, representative embodiments only and is not intended to limit the scope of the invention. Unless the context clearly indicates otherwise, the singular forms “a” and “described” as used herein are also intended to include the plural forms. It should be further understood that the term “comprising” as used herein indicates the presence of the stated feature, integer, step, operation, element, and / or component, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0034] The functionality of the various elements shown in the figure (including any functional blocks labeled "processor") can be provided using dedicated hardware and hardware capable of executing instructions in association with appropriate software instructions. When provided by a processor, these functions can be provided by a single dedicated processor, a single shared processor, or multiple separate processors, some of which may be shared. In some implementations of the present invention, the processor can be a general-purpose processor, such as a central processing unit (CPU), or a purpose-specific processor, such as a digital signal processor (DSP). Furthermore, the explicit use of the term "processor" should not be construed as referring specifically to hardware capable of executing software, and may implicitly include, but is not limited to, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), read-only memory (ROM), random access memory (RAM), and non-volatile memory for storing software. Other hardware, conventional and / or custom hardware, may also be included.

[0035] A software module, or simply a module or unit of software, may herein be represented as a flowchart element or any combination of other elements indicating the execution of process steps and / or textual descriptions. Such a module may be executed by hardware, whether explicitly or implicitly indicated. Furthermore, it should be understood that a module may include, for example, but not limited to, computer program logic, computer program instructions, software, stacks, firmware, hardware circuitry, or combinations thereof, providing the required capabilities. It will also be understood that a “module” typically defines a logical grouping or organization of associated software code or other elements as described above, related to the defined functionality. Therefore, those skilled in the art will understand that in some implementations, specific code or elements described as part of a “module” may be placed in other modules, depending on the logical organization of the software code or other elements, and such modifications are within the scope of the disclosure as defined in the claims.

[0036] It's important to note that, as used in this article, the term "optimization" means improvement. It doesn't convey that the technology has produced an objectively "best" solution, but rather an improved one. In the context of memory access, it typically means that the efficiency or speed of memory access can be improved.

[0037] As used herein, the term "determine" generally means to perform a direct or indirect calculation, computation, decision, search, measurement, or detection. In some cases, such determination may be approximate. Therefore, a determined value indicates that the value or an approximate value is determined, directly or indirectly, through calculation, computation, decision, discovery, measurement, detection, etc. If an item is "scheduled," it is determined at any time prior to the moment it is indicated as "scheduled."

[0038] This technology can be implemented as a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium (or medium) storing computer-readable program instructions that, when executed by a processor, cause the processor to perform various aspects of the disclosed technology. A computer-readable storage medium may be, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), flash memory, optical disks, memory sticks, floppy disks, mechanical or visual encoding media (e.g., punched cards or barcodes), and / or any combination thereof. As used herein, computer-readable storage media should be construed as non-transient computer-readable media. A computer-readable storage medium itself should not be construed as a transient signal, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through optical fibers), or electrical signals transmitted through wires.

[0039] It should be understood that computer-readable program instructions can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network such as the Internet, local area network, wide area network, and / or wireless network. A network interface in each computing / processing device can receive the computer-readable program instructions from the network and forward them to store the computer-readable program instructions in a computer-readable storage medium within the corresponding computing or processing device.

[0040] Computer-readable program instructions for performing the operations of this invention may be assembler instructions, machine instructions, firmware instructions, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network.

[0041] All descriptions and specific examples of the principles, aspects, and implementations of the present invention herein are intended to include their structural and functional equivalents, whether they are currently known or will be developed in the future. Therefore, for example, those skilled in the art will understand that any block diagram herein represents a conceptual view of an illustrative circuit embodying the principles of the present invention. Similarly, it will be understood that any flowchart, flow diagram, state transition diagram, pseudocode, etc., represents various processes that can be represented substantially by computer-readable program instructions. These computer-readable program instructions can be provided to a processor or other programmable data processing apparatus to produce a machine, such that the instructions are executed by the processor of the computer or other programmable instruction execution apparatus to create mechanisms for implementing the functions / actions specified in the flowcharts and / or block diagrams or blocks. These computer-readable program instructions can also be stored in a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, such that the computer-readable storage medium having the instructions stored therein comprises an article of manufacture of instructions implementing various aspects of the functions / actions specified in the flowcharts, flow diagrams, state transition diagrams, pseudocode, etc.

[0042] Computer-readable program instructions can also be loaded onto a computer, other programmable data processing apparatus or other equipment to cause a series of operational steps to be performed on the computer, other programmable apparatus or other equipment to produce a computer-implemented process, such that the instructions executed on the computer, other programmable apparatus or other equipment implement the functions / actions specified in flowcharts, flow diagrams, state transition diagrams, pseudocode, etc.

[0043] In some alternative implementations, the functions observed in flowcharts, flow diagrams, state transition diagrams, pseudocode, etc., may appear outside the order observed in the diagram. For example, in fact, two boxes shown consecutively in a flowchart may be executed almost simultaneously, or sometimes, the boxes may be executed in reverse order, depending on the functions involved. It should also be noted that each function shown in the diagram, and combinations of these functions, can be implemented by a dedicated hardware-based system that performs the specified function or action, or by a combination of dedicated hardware and computer instructions.

[0044] With these fundamentals in mind, we will now consider some non-limiting examples to illustrate various implementations of various aspects of the invention.

[0045] Figure 1Computer system 100 including compiler 116 is shown, as described in more detail below. As will be understood by those skilled in the art, a compiler, such as compiler 116, is generally a computer program that translates computer code written in one programming language (called "source code," written in the "source language") into computer code in another programming language ("target code," or "object code"), referred to as the "target language"). Typically, the source language will be a relatively "high-level" language (i.e., a language that can be read by humans with knowledge of programming languages), such as "C," "C++," "Java," "Rust," "Go," or many other languages. The target language is typically binary "machine language," which can be executed by a computer implementing a specific hardware architecture. Alternatively, the target language can be an intermediate form that, while closer to machine language than the source language, is still not specific to a single hardware architecture. In some cases, the target language can even be a different high-level language.

[0046] Computer system 100 can be a multi-user server or computer, a single-user computer, a laptop, tablet, smartphone, embedded control system, or any other computer system currently known or subsequently developed. Figure 1 As shown, the computer system 100 includes one or more processors 102, memory 110, memory interface 120, display interface 130, and network interface 140. These system components are interconnected via bus 150.

[0047] Memory 110 may contain data 112, an operating system 114, and a compiler 116. Data 112 may be any data used as input or output for any program in computer system 100. Operating system 114 is an operating system such as Microsoft Windows or Linux. Compiler 116 includes a parser 117 that generates an intermediate representation from a human-readable source language, an optimizer 118 that optimizes the code in the intermediate representation, and a code generator 119 that typically generates assembly or machine code executable by the target computer. It should be noted that optimizer 118 may include code that optimizes memory accesses to constant values ​​in a heterogeneous parallel architecture, as described in more detail below. Those skilled in the art will understand that although compiler 116 is shown to execute on computer system 100, compiler 116 may execute on many computer systems, for example, connected via a network. Furthermore, parser 117, optimizer 118, and code generator 119 may reside on different computer systems. Furthermore, it will be understood that although compiler 116 is shown to be executed on computer system 100, the code generated by compiler 116 does not need to be targeted at computer system 100, but can be code for different computer system architectures.

[0048] Memory interface 120 is used to connect a storage device, such as storage device 125, to computer system 100. One type of storage device 125 is a solid-state drive, which can use integrated circuit components to persistently store data. Different types of storage devices 125 are hard disk drives, such as electromechanical devices that use magnetic storage to store and retrieve digital data. Similarly, storage device 125 can be an optical drive, a card reader that receives removable memory cards (such as SD cards), or a flash memory device that can be connected to computer system 100 via, for example, a universal serial bus (USB).

[0049] In some implementations, computer system 100 may use well-known virtual memory technology, which allows programs of computer system 100 to behave as if they can access a large contiguous address space, rather than accessing multiple smaller storage spaces, such as memory 110 and storage device 125. Therefore, although data 112, operating system 114, and compiler 116 are shown residing in memory 110, those skilled in the art will recognize that these items are not necessarily all contained in memory 110 simultaneously.

[0050] Processor 102 may include one or more microprocessors and / or other integrated circuits. Processor 102 executes instructions stored in memory 110. When computer system 100 starts, processor 102 may initially execute startup routines and / or program instructions that make up operating system 114. Processor 102 may also execute instructions that make up compiler 116.

[0051] Display interface 130 is used to connect one or more displays 135 to computer system 100. For example, these displays 135 may include terminals, monitors, keyboards, pointing devices, touchscreens, and / or other human-computer interface devices, providing users with the ability to interact with computer system 100. However, it should be noted that although display interface 130 is provided to support communication with one or more displays 135, computer system 100 does not necessarily need the displays 135, as all necessary interactions with the user can be performed through network interface 140.

[0052] Network interface 140 is used to connect computer system 100 to other computer systems or networking devices (not shown) via network 160. Network interface 140 may include a combination of hardware and software that allows communication over network 160. The software in network interface 140 may include software that communicates over network 160 using one or more network protocols. For example, the network protocol may include TCP / IP (Transmission Control Protocol / Internet Protocol). In some implementations, network interface 140 may be an Ethernet adapter.

[0053] It should be understood that computer system 100 is merely an example and is executed on computer systems or other computing devices with different configurations according to the compilers and optimizers of the disclosed technology.

[0054] Figure 2 shows a block diagram of an exemplary heterogeneous parallel computing system 200 to which the compiler and optimizer of the present invention are intended. It should be understood that the compiler and optimizer may operate on the same computer system or architecture for which they generate code, or they may run on different systems and / or architectures. In either case, the generated code is intended to execute on the heterogeneous parallel computing system 200, as shown in Figure 2.

[0055] The heterogeneous parallel computing system 200 shown in Figure 2 conforms to the OpenCL system model. Throughout this disclosure, OpenCL-related terminology, system, and memory models will be used. However, it should be understood that the principles described herein can be applied to any heterogeneous parallel computing system, and are not limited to OpenCL. It will also be understood that the heterogeneous parallel computing system 200 shown in Figure 2 represents a logical model for heterogeneous parallel computing, and the way this logical model is mapped to the physical implementation of a heterogeneous parallel computer may vary.

[0056] The heterogeneous parallel computing system 200 includes a host 202 that communicates with one or more computing devices 204. Each computing device 204 includes one or more computing units 206, and each computing unit 206 includes one or more processing elements 208. According to the OpenCL model, the computing system 200 may include any number of computing devices 204, computing units 206, and processing elements 208.

[0057] Host 202 is a computer system that executes a host program that defines the context for one or more kernels and manages their execution. These kernels are functions that execute on one or more computing devices 204. Host 202 queues commands to computing devices 204, which may include, for example, executing kernels or reading and writing memory objects.

[0058] Computing device 204 is a device connected to host 202 that is capable of performing computational functions in parallel with and in parallel with host 202. Examples of computing devices 204 include GPUs, DSPs, multi-core CPUs, TPUs, and other similar devices. Each computing device 204 can execute one or more cores provided by host 202.

[0059] Within computing device 204, parallel instances of the kernel define work items that are executed on processing element 208 and are organized into workgroups, each of which executes on computing unit 206. Processing element 208 within a single computing unit 206 executes a single instruction stream (i.e., instructions associated with the kernel), either as a single instruction multiple data (SIMD) unit, where processing element 208 operates with a locked step size but on different data elements, or as a single program multiple data (SPMD) unit, where each processing element 208 maintains its own program counter.

[0060] Figure 3 shows a block diagram of a memory model 300 for a heterogeneous parallel computing system. Like the computing system 200 described above with reference to Figure 2, memory model 300 conforms to the OpenCL model. It should be understood that the principles disclosed herein can be used with other memory models, and are not limited to OpenCL. It will also be understood that memory model 300 represents a logical memory model, and variations will exist in how this logical model is mapped to the physical memory architecture.

[0061] Memory model 300 includes global memory 310, which is allocated from a global memory pool (not shown) and can be mapped from memory regions of the host device (not shown in Figure 3). Global memory 310 can be dynamically allocated by the host (i.e., allocated at runtime) and can be read from or written to by the host. The kernel typically cannot allocate space in global memory 310 (except for constant memory - see below), but can read from or write to global memory 310.

[0062] Constant memory 312 is a region allocated within global memory 310, accessible as read-only variables within the kernel. During kernel execution, all kernel working items can access these read-only variables. Space in constant memory 312 can be dynamically allocated by the host or statically allocated by the kernel (i.e., at compile time). Unlike the kernel, the host can read and write constant memory 312.

[0063] In some systems, data accessed from global memory 310 and constant memory 312 can be cached in the global / constant memory data cache 314. This helps the kernel executing on computing device 204 to access portions of global memory 310 and constant memory 312 more quickly.

[0064] Local memory 316 is a workgroup-local memory region that can be accessed by all work items within the workgroup for reading and writing. In some systems (as shown in Figure 3), local memory 316 may be implemented as a dedicated memory region on computing device 204 or within computing unit 206. In other systems, local memory 316 may be mapped to a portion of global memory 310. Local memory 316 can be statically allocated within the kernel, or the host may issue a command at runtime that causes local memory 316 to be dynamically allocated. Although the host may cause the allocation of local memory 316, the host cannot read from or write to local memory 316.

[0065] Private memory 318 is a memory region private to a specific work item (as described above, executed on a processing element (PE)). Variables in private memory 318 are not visible to other work items. Since a work item is a parallel instance of the kernel, in a sense, the kernel (i.e., a specific running parallel instance of the kernel) can read from and write to private memory 318, and can statically allocate space in private memory 318. The host has no right to read from, write to, or allocate private memory 318. Typically, private memory 318 can be implemented as a dedicated memory region within computing unit 206 or processing element 208.

[0066] It should be understood that the primary focus of this invention is compile-time optimization of access to constant memory 312. Since constant memory in OpenCL is part of global memory, which is typically the slowest memory type accessed during work item execution, this optimization can significantly improve speed.

[0067] In practice, many GPU and other OpenCL computing device vendors have achieved faster access to constant memory by providing limited on-chip constant memory and constant cache on their devices. Figure 4 shows an example of this memory hierarchy.

[0068] As can be seen, the memory hierarchy 400 includes off-chip memory 402, on-chip memory 404, and registers 406 that can be directly and quickly accessed by the processor. Off-chip memory 402 includes off-chip global memory 410 and off-chip constant memory 412. Off-chip memory 402 may be memory mapped from host memory or other memory not on the same physical integrated circuit as the computing device. On-chip memory 404 includes on-chip constant memory 420 and constant cache 422.

[0069] At runtime, constants are moved from host memory or off-chip constant memory 412 to on-chip constant memory 420. The compiler typically schedules the kernel to access constants from on-chip constant memory 420. When constants are accessed from on-chip constant memory 420, they can be cached in constant cache 422. This means that if the same access index of a constant array is accessed repeatedly in the kernel, the access will be fast because the value at that constant access index is likely cached in constant cache 422. Accesses can also be relatively fast if the access index itself is a constant or a compile-time known value, because these constants can be placed in on-chip constant memory 420, and the compiler can schedule them to be accessed from on-chip constant memory 420. However, if the access index of a constant is dynamic (i.e., it changes at runtime), the access time can be slow because the constant value may need to be accessed from off-chip constant memory 412.

[0070] Therefore, loading from constant memory is inefficient when the access index is unknown at compile time, especially when thread variable indexes exist. Furthermore, the scheme outlined in Figure 4 aims to use the same memory type for all constants, regardless of whether other memory types are available. This limits the ability to optimize access to constant memory across various architectures, and is particularly problematic when used on architectures without on-chip constant memory or constant caches but potentially supporting other types of fast on-chip memory.

[0071] According to the principles of this invention, the compiler can make "intelligent" decisions about the type of memory to be used for constants, utilizing different types of memory that may be available in a particular computing device architecture. For example, since constants are typically relatively small, fast on-chip memory, such as register sets, which can only store small amounts of data, can be used to store some constants. If the compiler can determine the access pattern of constants at compile time, then various memory placement schemes can be used even when accessing constants using dynamic access indexes. The read-only nature of constant memory further enhances the opportunity to optimize access to constant memory in heterogeneous parallel computing systems. Since constant memory cannot be written to, there should be no race conditions (i.e., inconsistencies or anomalous behavior due to event timing, such as multiple processors writing different values ​​to the same memory) or other problems that could lead to unpredictable results in parallel code and potentially complicate optimization. The worst-case scenario is simply that access to constants will be as slow as without optimization.

[0072] Unlike the GPU memory hierarchy discussed above with reference to Figure 4, the disclosed technique does not use a consistent memory location or hierarchy for constants. Instead, the compiler determines the data location of constants, which can be, for example, in various types of on-chip fast-access memory. At runtime, constants are loaded from host or off-chip constant memory into the compiler-specified memory. The compiler also arranges the code so that instances of the kernel load constants from the chosen memory type, which may involve using load instructions of appropriate types, depending on the memory type chosen by the compiler. Because the compiler has the flexibility to control the location of constant data, it can arrange constants to improve the speed of accessing them and can also adapt to mapping the logical memory model shown in Figure 3 to underlying memory systems with very different architectures, and, for example, where on-chip constant memory or constant caches may be lacking.

[0073] According to the present invention, the optimizer has two technical features that enable these benefits. First, constant values ​​in the program's outer scope or the kernel's outermost scope are converted into implicit parameters that point to constants in the kernel, from which the constants can be seen. This changes these constant values ​​in the code, which are typically placed in a predetermined data segment (usually constant memory), into pointers to constants, which can point to any memory type. The second technical feature for implementing the disclosed optimization is analyzing the access patterns of constants to allow constants accessed via dynamic access indexes to be placed in a memory type that is faster than constant memory, provided that the access patterns can be determined at compile time.

[0074] Figure 5 A block diagram of a compiler provided for one implementation of this technology. Compiler 500 includes an analyzer module 502, an optimizer module 520, and a code generator module 530.

[0075] The analyzer module 502 includes a lexical analyzer module 504, a syntax analyzer module 506, a semantic analyzer module 508, and an intermediate code generator module 510. The analyzer module 502 translates source code in a high-level language, such as "C" or "C++", into an intermediate representation. While the details of the analyzer module 502 and its component submodules will vary depending on the high-level language being compiled and the intermediate representation being used, the operation and implementation of the analyzer module 502 follow well-known principles of compiler design and can be implemented by those skilled in the art based on these principles. It should be understood that, for OpenCL, the analyzer module 502 typically translates from a modified version of the "C" programming language known as the "OpenCL C programming language" (although other high-level languages ​​may also be supported in OpenCL) to an OpenCL intermediate representation, such as the "Standard Portable Intermediate Representation" (SPIR, the current version of which is called SPIR-V 1.5) and / or LLVM IR (an intermediate representation defined as part of the LLVM compiler infrastructure, an open-source project managed by the LLVM Foundation).

[0076] Once the source code has been converted into an intermediate representation, the intermediate representation undergoes various optimizations in optimizer module 520. According to the implementation of the invention, the optimizer module includes a constant memory optimizer 522 that optimizes access to constant memory. It should be understood that optimizer module 520 may include many other optimizations. However, since a non-exclusive focus of the invention is constant memory optimization, this is the optimization that will be described in the most detail herein. It will also be understood that constant memory optimizer 522 can use information about the specific heterogeneous parallel computer architecture and memory architecture being targeted to optimize the intermediate representation, such that the optimizations can affect not only the intermediate representation but also later processes in code generator module 530 that convert the intermediate representation into assembly and / or machine code.

[0077] The code generator module 530 translates the optimized intermediate representation code into output code in assembly language or machine code, etc. The output code may be ready for execution, or it may require further steps, such as linking or even further compilation, to execute on the target computer. The output code may be specific to the target architecture and / or memory model. In some implementations, the constant memory optimizer 522 may provide the code generator module 530 with information beyond the optimized intermediate representation to instruct the code generator module 530 to use appropriate assembly or machine code instructions to access the selected memory type, and / or flag the assembly to inform the OpenCL driver about the constant's memory location decisions.

[0078] Details regarding the generation of assembly or machine code from optimized intermediate representation code will vary depending on the intermediate representation and the target architecture. Methods for generating assembly or machine code from intermediate representations are generally well-known to those skilled in the art. It will also be understood that in some implementations, the code generator module 530 may generate code from another intermediate representation, or even from a different high-level programming language, instead of generating assembly or machine code.

[0079] It should be understood that in some implementations, Figure 5 The compiler structure shown can vary. For example, in some implementations, the optimizations can be performed on the source code rather than on the intermediate representation. In these implementations, there may be optimizations, possibly including parts of a constant memory optimizer, that would occur within the optimization submodule (not shown) of analyzer module 502.

[0080] Now for reference Figure 6 The operation of the constant memory optimizer 522 is described in more detail. The constant memory optimizer includes a scope determination module 610, an access mode determination module 620, a memory type determination module 630, and a code translation module 640.

[0081] The scope determination module determines the scope of constant values ​​in a program. Used in this context, the "scope" of a constant refers to its visibility within the program—that is, which parts of the program can see or use the constant value. For constants that are program-scoped constants or kernel-outermost-scoped constants, the optimized properties and the type of memory used to store the constant value may differ from the constant pointer parameter passed (i.e., sent or transferred) to the kernel.

[0082] exist Figure 7A The example code block 700, written in the OpenCL C programming language, illustrates program-scope constants and constants located in the outermost kernel scope. In code 700, line 702 shows the declaration of a program-scope constant array “a” containing eight integers. This constant is a program-scope constant because it is visible (and available) throughout the program. Line 704 declares a kernel named “test”, and line 706 shows the declaration of a constant array “b” with eight items in the outermost scope of the kernel “test”. This means that all working items executing the kernel “test” can access the entire constant array “b”.

[0083] exist Figure 7BThe example code block 750, written in the OpenCL C programming language, illustrates a constant pointer parameter. Line 752 shows the declaration of a kernel named "test". The kernel has two parameters. The first, called "out", is a pointer to an integer in global memory. The second parameter, called "in", is a constant pointer parameter—in this case, a pointer to a constant integer. It's important to note that pointers can be accessed using access indices, just like arrays; therefore, the constant pointer parameter "in" effectively points to a constant array. However, we cannot determine the size of this array from the pointer itself.

[0084] Back Figure 6 In some implementations, the scope determination module 610 examines the declarations of constants to determine whether they are: (a) program-scoped constants or constants in the outermost kernel scope; or (b) constant pointer parameters. Given information typically available in the intermediate representation, this determination can be performed by scanning the intermediate representation operations that declare constants and kernel parameter types.

[0085] In the first case 612, if the constant is a program-scoped constant or a constant in the outermost kernel scope, the access mode determination module 620 will be input with this information, which will allow the size of the constant to be determined. This will also affect the memory type determination module 630 (e.g., because the size of the constant can be used to determine the memory type) and the code translation module 640 (because the code should be translated to pass a pointer to the constant in the selected memory type as an implicit parameter to the kernel). It should be understood that in this context, the “size” of a constant or array refers to its size in memory, i.e., the amount of memory used to store the constant or array. Similarly, the size of the memory does not refer to the physical size of the memory, but rather to how much data it can store.

[0086] In the second case 614, if the constant is a constant pointer parameter, the access mode determination module 620 will be input with this information, meaning the size of the constant will be unknown. This will also affect the memory type determination module 630 and the code translation module 640, as described in more detail below.

[0087] If the constant is not any of the program-scoped constants, kernel-outer-scoped constants, or constant pointer parameters, then in Figure 6 In the implementation shown, the constant memory optimizer 522 will not process further memory access optimizations for the constant. However, it is understood that this does not preclude other optimizers (not shown) from performing optimizations on these constants.

[0088] Next, in the access pattern determination module 620, the optimizer 522 will attempt to find the range of access indices on constants used in the kernel. This can be done at compile time by looking up constant access indices when constants are used in the kernel, or by looking up constant access indices in the kernel, for example, those associated with a loop counter, in which case the loop counter can be used to determine which access indices will be used to access constants. More generally, the access pattern determination module 620 may look for the use of variables (or linear functions of variables) as access indices on constants, where the values ​​of the variables follow a known finite sequence. In intermediate representations suitable for parallel computation, such as SPIR-V and / or LLVM IR, since there are often loop operations, there may be operations or hints that provide information about decomposing loop iterations into work items. Such information may expose the variables used as access indices and their ranges.

[0089] As an example of an access index with a scope that should be determined at compile time, in Figure 7A In loop 708, there is a loop counter called "idx", which is used as an access index for the constant arrays "a" and "b". As found in the loop constraints, "idx" ranges from 0 to 2, which will also be the range of access indices for "a" and "b" in this kernel. Similarly, Figure 7B Loop 754 in the code has a loop counter "idx" that ranges from 2 to 7. Since "idx" is also used as the access index for the constant pointer parameter "in", the access index range for "in" will be from 2 to 7.

[0090] It should be understood that in some cases, the access mode determination module 620 may fail to determine the access mode or access index range of a constant. For example, when the access index is a variable resulting from complex calculations or otherwise does not follow an easily identifiable sequence, the access mode determination module 620 may fail to determine the access mode or access index range at compile time.

[0091] In one implementation, if the constant is a program-scoped constant or a constant in the outermost kernel scope, the access mode determination module 620 determines the size of the constant and analyzes the access index used to access the constant (see above) to attempt to determine the access index range of the constant at compile time. If the access index range can be determined, the size and access index range are forwarded to the memory type determination module 630. If the access index range cannot be determined, the size information and an indicator indicating that the access index range cannot be determined are forwarded to the memory type determination module.

[0092] If the constant is a constant pointer parameter, the access mode determination module 620 will not be able to determine the total size of the constant at compile time. Therefore, the access mode determination module 620 will analyze the access index used to access the constant (see above) to attempt to determine the access index range of the constant at compile time. If the access index range can be determined, the access index range is forwarded to the memory type determination module 630. If the access index range cannot be determined, the indicator for which the access index range cannot be determined is forwarded to the memory type determination module.

[0093] It should be noted that in the above implementation, the access mode determination module 620 determines the access index range of the constant. In some implementations, the access mode determination module 620 can determine other access mode information, such as the sequence of access indices used to access the constant. Of course, this change will also affect the memory type determination module 630 and the code translation module 640.

[0094] In the memory type determination module 630, the optimizer 522 determines the type that will be used to store constants. To describe these memory types, general descriptors such as "fast memory," "medium-speed memory," and "normal memory" are used. These terms are used to broadly describe memory types commonly available in a memory system architecture. However, the exact type may vary depending on the underlying architecture. Because the optimizer 522 can generate code specific to the underlying memory system architecture, it can select the memory type to use in that particular architecture.

[0095] Typically, fast memory is on-chip memory that can be accessed very quickly by the processor (i.e., GPU, DSP, or other computing device), but is usually relatively small in size. For example, fast memory can be registers within a GPU. Registers are typically high-speed, multi-port static RAM that the processor can access directly and extremely quickly. While these registers are relatively few compared to other memory types, a typical GPU may still have one or more register sets, each potentially containing thousands of registers. Constants can be stored in registers, depending on their size and the number of available registers. The compiler needs to track register allocations to ensure that any constants placed in registers are small enough to fit within the available registers, while still leaving enough registers for the processor in the GPU to perform other operations that require registers. The size considered "small enough" will vary depending on the number of registers supported in a particular architecture. Therefore, a constant that is "small enough" to be placed in a register in an architecture supporting 16K registers may be much larger than a constant that is "small enough" to be placed in a register in an architecture that only supports 1K registers. Although registers are described here as fast memory, it is understood that other types of fast memory may exist on computing devices, which are characterized by high-speed access (often the fastest type of memory available on a chip in a given architecture) and small size (at least relative to other types of memory).

[0096] Medium-speed memory refers to memory that is slower to access than "fast memory" but faster than "normal" memory (such as global memory) for storing constants. Medium-speed memory is typically on-chip memory in GPUs, DSPs, or other computing devices. For example, on a GPU, medium-speed memory could be on-chip shared or local memory or other on-chip memory. This type of memory tends to be much larger than fast memory—often orders of magnitude—but can still be limited compared to off-chip memory. Like fast memory, medium-speed memory is a finite resource, so constants stored in medium-speed memory may still need to have a known finite size. The size limit of constants that can be stored in medium-speed memory in a given architecture will vary depending on the size of the medium-speed memory. It will also be understood that some architectures can support multiple types of on-chip memory operating at "medium" speeds. In some implementations, any or all of these types of memory can be used as medium-speed memory.

[0097] Here, "normal memory" refers to the type of memory typically used for constants. This is usually off-chip memory, which is relatively slow to access. However, depending on the target memory system architecture, other types of memory can also be used. In some implementations, this can be host-based memory that has been mapped to allow access to computing devices such as GPUs. Normal memory is typically very large—currently often measured in GBs—but its access time is relatively slow.

[0098] After explaining the memory types that can be assigned to constants, now refer to Figure 8A and Figure 8B Explain the operation of memory type determination module 630. Figure 8A In the first use case, the scope of the constant is either the kernel's program scope or the outermost scope, and the size of the constant will be known. Figure 8A Table 800 of the sub-cases shown can be used to determine the memory type applied to the storage constant.

[0099] In the "small size" sub-use case 802, if the constant is small enough to fit in fast memory (as mentioned above, this will depend on the size of the fast memory being determined), the memory type determination module 630 will select fast memory, such as a register, to store the constant. Because the overall size of the constant is relatively small, this can be accomplished in some implementations regardless of the access index range or access mode.

[0100] In the "Program Scope Known Access Range" sub-use case 804, if the constant is large (i.e., the small-size sub-use case 802 cannot place the entire constant in fast memory) and the access index range is known at compile time (i.e., the access mode determination module 620 can determine the access index range), then the memory type determination module 630 will select medium-speed memory, such as on-chip shared or local memory, for the constant. In this sub-use case 804, it can be understood that since the access index range of the constant is known, only the portion of the constant within the access index range actually needs to be transferred to and stored in medium-speed memory. Therefore, selecting medium-speed memory for the constant means that those portions of the constant within the known access index range will be stored in medium-speed memory at runtime. This provides the advantage that less data needs to be moved from host or global memory to medium-speed memory at runtime, and the portion of the constant within the known access index range will use less space in the limited medium-speed memory than the entire constant. Furthermore, there may be cases where the entire constant is too large to fit in medium-speed memory, but the portion of the constant within the access index range can be placed there. It should be noted that in some implementations, if the access index range is small enough, fast memory can be selected for the portion of the constants within the access index range.

[0101] Finally, in the "default" sub-use case 806, if the constant is insufficient to fit in fast memory and the access index range is unknown at compile time (i.e., the access mode determination module 620 cannot determine the access index range), then the memory type determination module 630 selects normal memory for the constant. In many cases, this will be global memory or constant memory (a portion of global memory), which may be relatively slow to access. However, it should be noted that this is generally not worse than not using optimizer 522.

[0102] Figure 8B The second use case, where the constant is a constant pointer parameter pointing to the kernel, illustrates memory type selection. Table 850 shows the sub-use cases that can be applied. Since the size of the constant is unknown if it is a constant pointer parameter, the size of the constant cannot be used to determine the memory type in this case.

[0103] In sub-use case 852, "Constant pointer known access," if the access index range is known at compile time (i.e., the access mode determination module 620 is able to determine the access index range), the memory type determination module 630 selects either fast or medium-speed memory. If the portion of the constant within the access index range is small enough to fit in fast memory, the memory type determination module 630 will select fast memory for the constant. As described above, this means that the portion of the constant within the access index range will be stored in fast memory at runtime.

[0104] If the portion of the constant within the access index range is insufficient to fit in fast memory, the memory type determination module 630 will select medium-speed memory for the constant. This means that the portion of the constant within the access index range will be stored in medium-speed memory at runtime. It should be understood that in some cases, the portion of the constant within the access index range may be too large for medium-speed memory. If this occurs, the constant will simply be stored in the normal memory used to store the constant.

[0105] In the constant pointer unknown access sub-use case 854, if the access index range of the constant pointer parameter is unknown at compile time (i.e., the access mode determination module 620 cannot determine the access index range), the memory type determination module 630 selects normal memory for the constant. In many cases, this will be global memory or constant memory (a portion of global memory), and access may be relatively slow. As mentioned above, this is generally not worse than not using optimizer 522.

[0106] Refer again Figure 6 Once a memory type has been chosen for a constant, the code should be translated to reflect this decision. This is handled by the code translation module 640.

[0107] If the constant is a program-scoped constant or a constant in the outermost kernel scope, placing the constant in memory outside of the normal memory where it is stored may require greater flexibility. According to one aspect of the technique disclosed herein, this flexibility can be achieved by treating these constant values ​​as implicit parameters to the kernel, where these parameters are visible. This means that for program-scoped constants, the constant can be passed as an implicit parameter to every kernel. For constants in the outermost kernel scope, the constant should be passed as an implicit parameter to the kernel in its outermost scope. Advantageously, this changes the constant values ​​in the code, which are typically placed in a predetermined data segment (usually constant memory), into pointers to the constants. Pointers to constants are not significantly different from any other pointers and can point to any memory type. In the case where the constant is a constant pointer parameter, there is no need to treat the constant as an implicit parameter, as it is already an explicit kernel parameter and already a pointer to a constant.

[0108] The code transformation module 640 uses the selected memory type to change the memory type of the constant and propagates this change to locations in the code where the constant is used. In some implementations, this might primarily involve changing the memory type pointed to by the pointer to the constant and arranging the code so that the constant will be moved to that memory type at runtime. In some implementations, the code transformation module 640 may apply other transformations to the intermediate representation, such as applying an offset to an access index where a portion of the constant within the access index range is stored in the selected memory type.

[0109] Furthermore, the code conversion module 640 can provide the code generator module 530 with information about memory type selection, instructing the code generator module 530 to use appropriate assembly or machine code instructions to access the selected memory type. In the case of the OpenCL implementation, the information provided to the code generator module 530 can be used to mark assembly code to inform the OpenCL driver of the memory location determination of constants.

[0110] It should be understood that the constant memory optimization operations and functionalities described in this paper, although illustrated with example code written in OpenCL and OpenCL C, can be used with other heterogeneous parallel computing systems and architectures, as well as other programming languages. In general, the techniques disclosed in this paper can be used with almost any programming language handling constant data, and in some cases, can also be applied to static data (i.e., data that exists throughout the program's execution but can be modified at runtime). Because static data may be modified at runtime, additional analysis may be required to check whether static memory has been modified or whether it is "load-only" memory (i.e., it is loaded or initialized at runtime but not changed thereafter). If not, the compiler may need to analyze memory dependencies to ensure data correctness.

[0111] It will also be understood that, although the embodiments presented herein have been described with reference to specific features and structures, various modifications and combinations may be made without departing from the invention. Therefore, the specification and drawings should be simply regarded as an illustration of the discussed implementations or embodiments and principles as defined in the appended claims, and are intended to cover any and all modifications, variations, combinations, or equivalents that fall within the scope of this invention.

Claims

1. An apparatus, characterized in that, include: processor; Memory coupled to the processor; A compiler residing in the memory and executed by the processor, the compiler including a constant memory optimizer, the constant memory optimizer comprising: The scope determination module is used to determine the scope of constants in the code of a heterogeneous parallel computer. A code conversion module is used to convert the code so that the constant can be passed to the kernel as an implicit pointer parameter when the scope of the constant is the kernel's program scope or the outermost scope in the code that references the constant. An access mode determination module is used to determine the size of the constant in the kernel and / or the access index range of the constant; A memory type determination module is used to select the memory type to be used for the constant based on the access mode. The optimizer defines multiple use cases, including: In the first use case, the scope of the constant is the program scope or the outermost scope of the kernel. The first use case includes multiple sub-use cases where the memory type determination module selects the memory type to be used for the constant based on the size of the constant and the access index range of the constant. The second use case, wherein the constant is a constant pointer parameter, includes multiple sub-use cases in which the memory type determination module selects the memory type to be used for the constant based on the access index range of the constant.

2. The apparatus according to claim 1, characterized in that, The code conversion module is also used to convert the code to use the selected memory type for the constant.

3. The apparatus according to claim 1, characterized in that, The first use case includes multiple sub-use cases, specifically including: Small size sub-use case, wherein the memory type determination module is used to select the fast memory for the constant if the size of the constant is small enough to be placed in fast memory; The program scope is a known access range sub-use case, wherein the memory type determination module is used to select medium-speed memory for the constant when the size of the constant is too large to be placed in the fast memory and the access mode includes a known access index range at compile time; The default sub-use case, wherein the memory type determination module is used to select the normal memory type for constants when neither the small-size sub-use case nor the sub-use case with the known access range of the program scope is applicable.

4. The apparatus according to claim 3, characterized in that, The fast memory includes registers.

5. The apparatus according to claim 3, characterized in that, The medium-speed memory includes on-chip memory in computing devices.

6. The apparatus according to claim 1, characterized in that, The second use case includes multiple sub-use cases, specifically including: A known access sub-use case for a constant pointer, wherein the memory type determination module is configured to select fast memory for the known access index range of the constant when the access pattern includes a known access index range at compile time and the size of the known access index range is small enough to be placed in fast memory; wherein the memory type determination module is configured to select medium-speed memory for the known access index range of the constant when the access pattern includes the known access index range at compile time and the size of the known access index range is too large to be placed in fast memory. The constant pointer unknown access sub-use case, wherein the memory type determination module is used to select the normal memory type for the constant value when the access mode does not include a known access index range at compile time.

7. The apparatus according to any one of claims 1 to 6, characterized in that, The heterogeneous parallel computer supports OpenCL.

8. The apparatus according to claim 7, characterized in that, The code conversion module is used to convert code represented in OpenCL intermediate format.

9. The apparatus according to any one of claims 1 to 6 or 8, characterized in that, The heterogeneous parallel computer lacks on-chip constant memory.

10. A method for optimizing access to constants in a heterogeneous parallel computer, characterized in that, include: The scope determination module is used to determine the scope of the constant in the code of the heterogeneous parallel computer. In the case where the scope of the constant is the kernel's program scope or the outermost scope, the code transformation module is used to transform the code so that the constant is passed as an implicit pointer parameter to the kernel in the code that references the constant; The access mode determination module is used to determine the size of the constant in the kernel and / or the access index range of the constant. Based on the size of the constant and / or the access index range of the constant, the memory type determination module selects the memory type to be used for the constant. The code conversion also includes converting the code to use the selected memory type for the constant; In the first use case where the scope of the constant is the program scope of the kernel or the outermost scope, selecting the memory type to be used for the constant includes selecting from multiple sub-use cases for memory speed based on the size of the constant. In the second use case where the constant is a constant pointer parameter, selecting the memory type to be used for the constant includes selecting from multiple sub-use cases targeting memory speed based on the access index range of the constant.

11. The method according to claim 10, characterized in that, In the first use case where the scope of the constant is the kernel's program scope or the outermost scope, selecting the memory type to be used for the constant specifically includes: In the case where the size of the constant is small enough to fit in fast memory, fast memory is selected for the constant in the small-size sub-use case; If the size of the constant is too large to fit in the fast memory, and if the access mode includes a known access index range at compile time, then in the known access range subuse case of the program scope, medium-speed memory is selected for the constant. If neither the small-size sub-use case nor the sub-use case with the known access range of the program scope is applicable, the normal memory type for constants is selected in the default sub-use case.

12. The method according to claim 11, characterized in that, Selecting the fast memory includes selecting a register to store the constant.

13. The method according to claim 11, characterized in that, Selecting the medium-speed memory includes selecting on-chip memory in the computing device to store the constant.

14. The method according to claim 11, characterized in that, In the second use case where the constant is a constant pointer parameter, selecting the memory type to be used for the constant specifically includes: In the case of a known access subcase of a constant pointer, fast memory or medium memory is selected, wherein fast memory is selected for the known access index range of the constant if the access mode includes the known access index range at compile time and the size of the known access index range is small enough to be placed in fast memory, and medium memory is selected for the known access index range of the constant if the access mode includes the known access index range at compile time and the size of the known access index range is too large to be placed in fast memory; In cases where the access mode does not include a known access index range at compile time, the normal memory type for the constant value is selected in the constant pointer unknown access subcase.

15. The method according to any one of claims 10 to 14, characterized in that, The heterogeneous parallel computer supports OpenCL.

16. The method according to claim 15, characterized in that, The conversion code also includes code that converts the representation in OpenCL intermediate form.

17. A non-transitory computer-readable medium, characterized in that, This includes instructions that, when executed by the processor, cause the processor to perform the following operations: Determine the scope of constants in heterogeneous parallel computer code; Determine the size of the constant in the kernel within the code and / or the access index range of the constant; The memory type to be used for the constant is selected based on the size and / or the access index range of the constant in the following manner: In the first use case where the scope of the constant is the kernel's program scope or the outermost scope; In the case where the size of the constant is small enough to fit in fast memory, fast memory is selected for the constant in the small-size sub-use case; If the size of the constant is too large to fit in the fast memory, and the access mode includes a known access index range at compile time, then in the known access range subuse case of the program scope, medium-speed memory is selected for the constant. When neither the small-sized sub-use case nor the sub-use case with the known access range of the program scope is applicable, normal memory for constants is selected in the default sub-use case; In the second use case where the constant is a constant pointer parameter: In the case of a known access subcase of a constant pointer, fast memory or medium memory is selected, wherein fast memory is selected for the known access index range of the constant if the access mode includes the known access index range at compile time and the size of the known access index range is small enough to be placed in fast memory, and medium memory is selected for the known access index range of the constant if the access mode includes the known access index range at compile time and the size of the known access index range is too large to be placed in fast memory; In cases where the access mode does not include a known access index range at compile time, normal memory for the constant value is selected in the constant pointer unknown access subcase. In cases where the scope of the constant is the kernel's program scope or the outermost scope using the code transformation module, the code is transformed to pass the constant as an implicit pointer parameter to the kernel in the code that references the constant and to use the selected memory type of the constant.

Images