A method, apparatus, medium of loading a compute core on a processing unit

Abstract

The present disclosure provides a method, device and medium for loading a computing core on a processing unit. The method for loading a computing core on a processing unit is applied to an intelligent computing device comprising a plurality of processing units, and the method comprises: running a resident runtime program on each processing unit; in response to a computing core execution request, the resident runtime program performs the following steps: saving the runtime context of the current processing unit; obtaining description information of a target computing core; constructing an execution environment for the target computing core according to the description information, which comprises: allocating storage space for a thread data element and completing the dynamic binding of its running address, and setting the computing core entry parameter according to a parameter data element; and jumping to the entry address of the code element to start the execution of the target computing core. The embodiments of the present disclosure aim to reduce the loading overhead, support the sharing and differentiated execution of computing cores among multiple processing units, and simplify the programming model.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, in particular to a method, device and medium for loading computing core on processing unit. BACKGROUND

[0002] In the field of artificial intelligence and high-performance computing, in order to improve data processing capability, hardware systems are usually integrated with multiple parallel operation units (or processing units, computing cores). Application programs, especially AI programs, are usually organized into multiple parallel executable computing cores and issued to these operation units. A typical application mode is that the same computing core needs to be issued to multiple different processing units to process different data partitions, thereby realizing data parallelism.

[0003] In a traditional graphics processing unit (GPU) or general-purpose graphics processing unit (GPGPU) platform, the loading of computing cores is usually completed by a hardware-logic-implemented loader. Such computing cores are usually designed as simple instruction sequences, and the hardware loader is efficient but fixed in function and lacks flexibility, which limits the complexity of the computing core and the programming model.

[0004] Emerging AI processors begin to use more general processor architectures (such as RISC-V) as their operation units, which can execute more complex instruction sequences and control logic. However, such processing units are usually optimized for data operation, and the storage resources allocated to program loading and management are very limited. It is challenging to implement flexible computing core loading under the condition of limited resources: first, the limited storage space is difficult to support complex dynamic linking and loading algorithms; second, it is necessary to support efficient deployment and running of the same program image on multiple processing units.

[0005] The existing technical solutions have the following disadvantages: 1) static compilation: the generated program is bound to an absolute address, which cannot be changed at runtime, and is not suitable for scenarios where multiple cores share an image. 2) Address-independent code (PIC): with a global offset table (GOT), although address independence can be achieved, the GOT needs to be repaired by a runtime loader, which introduces additional memory access overhead, and the processing of computing cores containing data segments is cumbersome and has large loading overhead. 3) Memory management unit (MMU): the address view can be changed through page table remapping, but this solution relies on MMU hardware support, and the page table switching overhead during process / context switching is large, which is not suitable for use in scenarios where AI computing cores are frequently loaded in a lightweight and low-latency manner.

[0006] Therefore, there is a need for a lightweight and flexible computing core loading method suitable for emerging AI processor architectures, to efficiently support the deployment and execution of complex computing cores on multiple processing units under the condition of limited storage resources.

[0007] It should be noted that the above introduction to the technical background is only for the convenience of clearly and completely describing the technical scheme of the present application and facilitating the understanding of those skilled in the art. The above technical scheme cannot be considered as known to those skilled in the art merely because it is described in the background section of the present application. SUMMARY

[0008] To solve at least one of the above problems, and one or more of other potential problems, the present disclosure proposes a method for loading a computing kernel on a processing unit, a computing device and a storage medium, to solve the problem of how to efficiently and flexibly load and run complex computing kernels on a resource-limited multi-processing unit AI acceleration platform. Some embodiments of the present disclosure aim to reduce the loading overhead, support the sharing and differentiated execution of computing kernels among multiple processing units, and simplify the programming model.

[0009] In a first aspect of the present disclosure, a method for loading a computing kernel on a processing unit is provided, wherein the method is applied to an intelligent computing device comprising a plurality of processing units, and the method comprises: running a resident runtime program on each processing unit; in response to a computing kernel execution request, the resident runtime program performs the following steps: S1: saving the runtime context of the current processing unit; S2: obtaining the description information of the target computing kernel, the description information at least comprising: address-independent code element information, thread data element information, and parameter data element information; S3: constructing an execution environment for the target computing kernel according to the description information, the construction of the execution environment including: allocating storage space for the thread data element and completing its runtime address dynamic binding, and setting the computing kernel entry parameter according to the parameter data element; S4: jumping to the entry address of the code element to start the execution of the target computing kernel; wherein the code element contains address-independent executable code and read-only data associated with a program counter (PC) obtained by compilation and linking; the thread data element contains global variable data that needs to be dynamically bound to the runtime address at the time of loading.

[0010] Further, in some embodiments, the running of the resident runtime program on each processing unit comprises: all processing units sharing the same read-only code image of the resident runtime program; the resident runtime program dynamically determines the value of a first base address register according to the identification information of the processing unit on which it is running at runtime, and binds the access of its non-readonly data to the memory region with the first base address register value as the base address.

[0011] Further, in some embodiments, in step S3, the allocating storage space for the thread data element and completing dynamic binding of its run address comprises: allocating a piece of space from a private memory area of the current processing unit for storing the thread data element; loading a start address of the space into a second base address register; wherein the instructions of the target computing core address its global variables by accessing the second base address register.

[0012] Further, in some embodiments, the parameter data element comprises common parameters, and the thread data element comprises private parameters; the common parameters are used to be passed to all processing units executing the same target computing core, and the private parameters are used to be passed to a specific processing unit; in step S3, the setting computing core entry parameters according to the parameter data element comprises: selecting corresponding private parameters according to the identification information of the current processing unit to set.

[0013] Further, in some embodiments, after the execution of the target computing core is completed, the method further comprises: the resident runtime program restores the runtime context saved in step S1, and returns to a ready state to wait for a next computing core execution request.

[0014] In a second aspect of the present disclosure, a processing unit for an intelligent computing device is also provided, comprising: a processing core; a memory; wherein the processing core is configured to execute instructions stored in the memory to implement the function of the resident runtime program in the method of loading a computing core on a processing unit as described above.

[0015] In a third aspect of the present disclosure, an intelligent computing device is also provided, comprising: a plurality of processing units as described above; an external management unit configured to issue a computing core execution request and description information of the computing core to the plurality of processing units.

[0016] In a fourth aspect of the present disclosure, a computing core compiling method is also provided for generating a computing core that can be loaded and executed by the method as described above, comprising: compiling and linking source code to generate address-independent code elements, the code elements containing executable code and read-only data; separating global variable data in the source code to form a thread data element, and generating relative addressing instructions dependent on a base address register to access these global variables; generating a parameter data element description structure for describing parameter information required by a computing core entry function.

[0017] In a fifth aspect of the present disclosure, a computer readable storage medium having a computer program stored thereon is also provided, wherein the computer program is executed by a processing unit to implement the method as described above.

[0018] In a sixth aspect of the present disclosure, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method as described above when executing the computer program.

[0019] The present disclosure has the following beneficial effects over the prior art:

[0020] In some embodiments, the software-implemented resident runtime program can handle more complex compute kernel formats (such as program sets containing multi-level subroutine calls, global / local variables) and support flexible runtime parameter binding and exception handling, compared with a hardware loader. Further, a compilation and runtime model of “PC binding code element + GPR binding data element” is proposed, which, in combination with the shared resident program technology, realizes address independence, does not require an MMU or a complex dynamic linker, and greatly saves storage resources and runtime overhead. Furthermore, through the defined thread data element and parameter format, different global data can be efficiently bound for the same compute kernel instance running on different processing units, perfectly supporting the data parallel computing paradigm. Further, the resident program can share the same image on multiple processing units and automatically locate private data according to the running location, simplifying the loading process of the system management unit and reducing the overall storage requirement. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings, in which:

[0022] Figure 1 A schematic diagram of a compute kernel definition according to an embodiment of the present disclosure is shown;

[0023] Figure 2 A schematic diagram of a target compute kernel description information extraction process according to an embodiment of the present disclosure is shown;

[0024] Figure 3 A schematic diagram of a resident runtime program memory layout according to an embodiment of the present disclosure is shown;

[0025] Figure 4 A schematic diagram of a compute kernel loading and runtime memory layout according to an embodiment of the present disclosure is shown;

[0026] Figure 5 A flowchart of a method of loading a compute kernel on a processing unit according to an embodiment of the present disclosure is shown;

[0027] Figure 6 A schematic diagram of the overall architecture of a system and resident program loading according to an embodiment of the present disclosure is shown;

[0028] Figure 7 A diagram showing the compiled organization of a compute kernel (code element, thread data element, parameter data element);

[0029] Figure 8 A diagram showing the flow of a resident runtime program responding to a request, building a compute kernel execution environment;

[0030] Figure 9 A diagram showing the memory access view of a compute kernel runtime, showing access to thread data elements through a base address register;

[0031] Figure 10 A diagram showing the memory mapping of multiple processing units sharing resident program code but having independent data spaces; and

[0032] In the various drawings, like or corresponding reference numerals designate identical or corresponding parts throughout the several views. DETAILED DESCRIPTION

[0033] Embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. While several embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and fully convey the scope of the present disclosure to those skilled in the art. It should be understood that the drawings and embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Also, in order to clearly and concisely describe the present disclosure, descriptions of well-known functions and structures incorporated herein can be omitted.

[0034] In the description of embodiments of the present disclosure, the term "including" and its derivatives, shall be understood as comprising, i.e., open-ended, and do not exclude the presence of additional elements. The term "based on" shall be understood as "based, at least in part, on." The term "one embodiment" or "an embodiment" shall be understood as "at least one embodiment." The term "first," "second," and the like, can refer to different or identical objects. Other definitions can be explicitly provided herein. It is also to be understood that the terminology "and / or" as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0035] It is to be understood that, although the terms first, second, third, etc. can be used herein to describe various information, the information should not be limited to these terms. These terms are only used to differentiate one piece of information from another. For example, a first state could also be referred to as a second state, and similarly, a second state could also be referred to as a first state, without departing from the scope of the present disclosure. Depending on the context, the word "if' as used herein can be interpreted as meaning "when" or "in response to determining."

[0036] It should be understood that in various embodiments of the present disclosure, the processing unit refers to a computing core capable of independently executing instructions in an AI processor, accelerator or SoC, such as a RISC-V core, DSP, MCU, etc. The computing core refers to a basic task unit that can be issued to the processing unit for execution in parallel computing scenarios such as AI. In the present disclosure, the computing core is defined as a program set containing multiple sub-programs, with only one entry program, supporting global variables, local variables and internal sub-program calls. The resident embedded runtime program refers to a lightweight software program that is preloaded and continuously running on each processing unit, responsible for managing the loading, execution, context switching and basic services of the computing core. The code element refers to a single data block bound to the program counter (PC) formed by statically compiling and linking all code segments and read-only data segments in the computing core (or resident program). Thread data element refers to a set of global variable data in the computing core that needs to be bound differently between different processing unit instances, which is decoupled from the specific base address register (GPR) at compile time. Parameter data element refers to the set of call parameters required by the entry program of the computing core.

[0037] It should also be understood that in embodiments of the present disclosure, the computing core is defined as a program set that can contain multiple programs, with only one execution entry program. The entry program of the computing core is a program that can pass in parameters, and local variables can be defined within the program. Local variables can only be used in the current program. Any program can call other sub-programs within the same program set. Global variables can be defined outside the program and can be shared and used by multiple programs in the program set. Compared to traditional computing cores, the computing core defined in embodiments of the present disclosure is more flexible. The definition diagram of the computing core can be seen in Figure 1 .

[0038] Further, since AI processors need to face data operations, the storage resources on the operation unit are mainly used for business data calculation, so only limited storage resources are allocated for computing core loading and other non-data operation purposes. The conditions for implementing computing core loading on processors with limited storage resources are more stringent: on the one hand, low storage resources prevent the processor from implementing complex dynamic loading and scheduling algorithms; on the other hand, in AI operation acceleration platforms, a system generally contains multiple operation processors, and these operation processors often need to use the same program image. For example, in some embodiments of the present disclosure, the computing core needs to run on multiple processors in the system to implement data processing parallelism.

[0039] Further, regarding the traditional static compilation, the running address of the program needs to be determined in the linking stage, which leads to the program compiled by static compilation being address-dependent and unable to change the loading and execution address at runtime. Thus, a solution is to use the address-independent compilation feature provided by the compiler to generate address-independent code, data, and a global offset table (GOT, Global Offset Table), and the access of the code and data needs to be implemented based on the global offset table at runtime. This scheme needs to be matched with a program loader, which modifies the global offset table according to the configuration information at runtime, thereby realizing the binding with the running address. It can be seen that this scheme is relatively cumbersome to implement and is not suitable for AI and other operation acceleration platforms. In AI and other acceleration platforms, the calculation core needs to be frequently loaded, and there are certain requirements and limitations on the loading overhead. The introduction of the global offset table increases the access memory, and for the calculation core carrying the data segment, the differentiated loading of the data segment also needs to be implemented.

[0040] Further, regarding the processor with a memory management unit (MMU, Memory Management Unit), the running address can be changed by replacing the page table mapping through the MMU, but this scheme can only be applied to the processor with the MMU, and one or more page table switches need to be accompanied when the program is switched.

[0041] Based on the above problems, in some embodiments of the present disclosure, a method for loading a calculation core on a processing unit is provided, and the method is described with reference to Figure 2 At the time of static compilation, the program and read-only data of the calculation core are associated with the PC (Program Counter, PC for short) to realize address decoupling. At the time of linking, the code segment and the read-only data segment are packaged into an integral segment, which is referred to as a code element. The local variable in the calculation core program is a temporary variable, which is referred to as a temporary data element, and the temporary data element does not need to be loaded. The data segment where the global variable in the calculation core is located needs to be address-decoupled at the time of compilation. For the data segment storing the global variable, it is referred to as a thread data element, which needs to be dynamically address-bound according to the processing unit on which the calculation core runs at the time of loading the calculation core, so as to realize differentiated loading. The parameters of the entry program of the calculation core need to be specified at the time of loading, which is referred to as a parameter data element, and the same parameter data element is used for the same calculation core.

[0042] Thus, in the embodiments of the present disclosure, the loading of the embedded runtime program that resides on the processing unit (first step, loading the resident program) is proposed. For example, in some embodiments, a resident embedded runtime program, referred to as a resident program, needs to be run on the processing unit, the resident program needs to be run on each processing unit, the resident program is loaded onto the target processing unit by an external management unit, after the loading is completed, the resident program starts in the target processing unit and performs a self-loading process. The self-loading process includes but is not limited to the following operations: initializing global variables, allocating and initializing stack space, setting the initial environment of program running, etc. In particular, the same resident program is run on all processing units, and the conventional processing method is to allocate a private space on each processing unit to store the resident program and the data generated during the running of the resident program, and each processing unit only runs in its own private space. This method will cause the storage resource demand in the entire system to increase with the increase of the number of processing units, and the external management unit needs to load the resident program onto the target processing unit in sequence. It should be noted that some embodiments of the present disclosure also provide a method for loading and running the resident program on the processing unit, see Figure 3 , all processing units can share the same resident program, and the storage of the data generated during the running of the resident program will automatically allocate addresses according to the running processing unit and store it on the external global space, without occupying additional storage resources. In particular, the read-only data in the resident program is bound to the PC (Program Counter) to realize the address-independent access of the read-only data, and the non-read-only data in the resident program is bound to the selected base address register GPR (General Purpose Register), and the value of the base address register is determined by the resident program during running according to the identification information of the processing unit currently running in.

[0043] Further, in some embodiments, in parallel with the loading of the embedded runtime program that resides on the processing unit, the embedded runtime program that resides on the processing unit also responds to the execution request of the computing core (program backup before running, recovery after running). Accordingly, in some embodiments, see Figure 4 and Figure 5The present disclosure also provides a method for loading a computing core on a processing unit. At static compilation time, the program and read-only data of the computing core are associated with a PC (Program Counter). At linking time, the code segment and read-only data segment are packaged into an integral segment, referred to as a code element. Local variables in the computing core program are temporary variables, referred to as temporary data elements, which do not need to be loaded. The data segment where the global variables in the computing core are located needs to be address decoupled at compilation time. For the data segment where the global variables are stored, referred to as a thread data element, dynamic address binding needs to be performed according to the processing unit where the computing core runs at the time of loading the computing core, so as to achieve differential loading. The parameters of the entry program of the computing core need to be bound at the time of loading, referred to as a parameter data element, and the same parameter data element is used for the same computing core. It should be noted that the computing core parameter information needs to be used at the time of loading the computing core, and the computing core parameters include the parameter data element and the thread data element. When the resident program on the processing unit detects a computing core task request to be executed, the current runtime context is first saved, including the values of the program counter, the stack pointer register, the general register, the base address register, the state register and other key registers. Subsequently, the runtime program reads the entry program address of the computing core and the computing core parameter information from the preset shared memory or communication interface. The resident program allocates storage space for the computing core according to the parameter information of the computing core. For the parameter data element in the computing core, the resident program constructs the computing core running stack space in the storage space allocated for the computing core, and sequentially extracts and fills the parameter data element. Only the data length semantics is concerned during extraction and filling, so as to simplify the processing logic. For the thread data element, the resident program needs to extract the data to be used on the current processing unit, and fill the data to the storage space allocated for the computing core, while completing the dynamic address binding. Finally, jump to the entry program address of the computing core, and start executing the computing core. During the running of the computing core, the address bound by the resident program can be used to access the global variables.

[0044] Figure 6A diagram showing the loading and initialization process of the resident embedded runtime program. In this diagram embodiment, the process method includes: step S101: the same copy of the resident program is loaded into a shared storage area accessible to multiple processing units in the system by an external management unit (such as a host CPU, system controller). Step S102: the external management unit starts the target processing unit in sequence or concurrently, and each processing unit starts executing the startup code of the resident program from a predetermined address in the shared storage area. Step S103: the resident program on each processing unit performs a self-loading process. The key of this process is address adaptive initialization. Further, in some embodiments, the resident program is compiled so that its read-only part (code element) is bound to the PC; and its writable data part (non-read-only data, such as global variables, stack) is bound to a predetermined base address register (GPR, for example, x3). During initialization, the program dynamically calculates the offset base address Base_Addr of the core private data in the global data area according to the unique ID (for example, core number CoreID) of the current processing unit through a predefined formula or lookup table, and writes it to the predetermined GPR. Step S104: all subsequent accesses of the resident program to its global data are implemented by adding a fixed offset to the GPR (for example, lw a0, offset(x3)). In this way, all processing units execute the same instruction stream, but through different GPR values, naturally access their own private data area, achieving "one image, multiple running".

[0045] Figure 7 A diagram showing the execution process of the self-loading process of the embedded runtime program running on the operation unit. In this diagram embodiment, the execution process method includes: step S201, the "embedded runtime program" starts; step S202, the "embedded runtime program" obtains the starting address of the allocated data segment storage area and the size of a single data segment; step S203, the "embedded runtime program" calculates the offset address of the allocated data segment storage area by the ID of the current running unit and the size of a single data segment, denoted as region A; step S204, the "embedded runtime program" performs a scatter loading process to copy the data segment image stored in the program segment to region A; step S205, the "embedded runtime program" sets the starting address of region A to the thread register; step S206, the "embedded runtime program" self-loading process ends.

[0046] Figure 8This illustration shows a schematic diagram of parameter construction for an embedded runtime program running on a computing unit, where parameters for the computing core are built. In this illustrated embodiment, it demonstrates how the embedded runtime program constructs and organizes the computing core's runtime parameters before executing a computing core task, particularly the construction process of parameter data elements and thread data elements. In some embodiments, it shows how the embedded runtime program prepares the runtime environment for the computing core after receiving a computing core execution request, with a focus on the structure and construction method of the parameters. Specifically, in Figure 8 In this embodiment, from a memory layout perspective, it includes: 1) Control information and configuration parameters, located at the top (high address) of the memory layout, containing basic control information required for the operation of the computing core, such as: the base address of the computing core stack space, the stack space size, the number of integer / floating-point parameters, and other configuration information (such as thread priority, entry address, etc.); this part is a fixed structure, provided to the runtime program by an external management unit or the main control system. 2) Common parameter area (thread-shared parameters), located below the control information, divided by parameter type: Integer parameter area: stores integer parameters, arranged in order (e.g., parameter 1, 2…N); Floating-point parameter area: stores floating-point parameters, arranged in order (e.g., parameter 1, 2…N); these parameters are shared by the same computing core running on all processing units, and are selected and constructed by the runtime program on each unit. 3) Private parameter area (unique to each processing unit), located below or parallel to the public parameter area, organized by processing unit number (thread 1, thread 2... thread N): For example, each processing unit has its own set of private parameters; including integer, floating-point, and other types, and may also contain unit-specific configuration or debugging information; the runtime program will select the corresponding parameters from the private parameter area according to the currently running processing unit ID to achieve "differentiated loading".

[0047] Furthermore, in some embodiments, a data structure for computing kernel parameter information is provided. This structure is a contract between the resident program and the upper-level compiler / scheduler, including: a control information area: containing metadata such as stack base address, stack size, and number of integer / floating-point parameters; a public parameter area: storing entry parameters that are the same for all computing kernel instances running on all processing units, such as constants and shared configurations; and a private parameter area: this is an array or list, with each element corresponding to a processing unit ID. Each element contains thread data elements specific to that processing unit instance, i.e., the initial values ​​of global variables that need to be differentiated.

[0048] Figure 9 and Figure 10 A schematic diagram illustrating the memory and access patterns of a computing core runtime environment according to embodiments of this disclosure is provided. It demonstrates the organization of the memory layout, data access paths, and the mapping relationship between registers and memory during actual execution of the computing core. Specifically, Figure 9The core structure (from high address to low address) presents a schematic diagram of the memory space layout of the computing core runtime, focusing on the organization structure of the address space. It includes: 1) high address area; 2) thread stack register pointing area -> thread private parameter area, including: control parameter-1, control parameter-2,..., control parameter-N, and thread public parameter-float type parameter; 3) main stack register pointing area -> thread public parameter-integer type parameter, which is the storage area of the integer public parameter; 4) thread stack, used to store function call stack, local variable, etc.; 5) thread heap, used for dynamic memory allocation; 5) low address area. Further, in Figure 9 the double stack mechanism is embodied: there are "main stack register" and "thread stack register", which point to different types of data areas respectively. And the address space is divided clearly: from high to low, it is: private parameter and control parameter, public parameter (float, integer), stack space, heap space; the association of registers and memory: through the register pointing to a specific memory area, fast access and data positioning are realized.

[0049] Further, Figure 10 The core structure (also from high address to low address) more specifically shows how the computing core runtime data is loaded from memory to register, embodying the process of parameter passing and register binding. It includes: 1) high address; 2) thread private parameter area, control parameter-1, control parameter-2,..., control parameter-N; 3) thread public parameter area, float type parameter (parameter 1, 2…10, N), integer type parameter (parameter 1, 2…10, N); 4) thread stack; 5) thread heap; 6) low address. In particular, "stored in float register", which means that the float type public parameter is loaded into the float register at runtime; "stored in integer register", which means that the integer type public parameter is loaded into the integer register at runtime; "main stack register", which points to the integer parameter area, used to access the integer public parameter; "thread stack register", which points to the private parameter area, used to access the thread private parameter and control parameter. Further, Figure 9 and Figure 10The memory and register organization model of the computing core runtime is jointly described, wherein: 1) the memory layout is layered, the upper layer (high address): control parameters + private parameters, the middle layer: public parameters (floating point + integer), and the lower layer (low address): stack + heap; 2) the register binding mechanism, thread stack register → binding private parameter area, main stack register → binding integer public parameter area, floating point / integer register → used for storing corresponding type parameters loaded from the memory, respectively; 3) the access flow, when the computing core is started: a) the resident program initializes the memory layout according to the computing core parameter information, b) the public parameters are loaded into the corresponding registers, c) the private parameters are located according to the processing unit ID, and d) the global variables and parameters are accessed through the bound registers. Further, in response to a computing core execution request, Figure 9 and Figure 10 The memory distribution of the computing core runtime is shown, which embodies the storage and access mode of the parameter data element (public parameter) and the thread data element (private parameter), and then shows how to quickly access the global variables through the registers after dynamic address binding. From the perspective of the parameter information format, the control information and the configuration parameters correspond to the control parameters-1…N, the public parameter area points to the integer / floating point parameter list, and the private parameter area points to the thread private parameter. Therefore, in the embodiments of the above Figure 9 and Figure 10 The lightweight register binding is implemented: without complex MMU or GOT, the address decoupling is realized through the base address register; double stack support: supporting parameter isolation and sharing in the multi-thread / multi-core environment; differentiated loading visualization: clearly showing how the same computing core accesses different private parameters on different processing units; efficient data access path: through register direct mapping, the memory access delay is reduced.

[0050] Further, in some embodiments, a loading and initialization method of an embedded runtime program is provided, which is suitable for an embedded system with programmable computing core (such as DSP, MCU, coprocessor, etc.). The method comprises the following steps: step S301, generating an image file, specifically comprising: first, compiling and linking the embedded runtime program by a compiler to generate an image file containing executable code segment and data segment. The image file supports the execution architecture format of the target operation unit, such as ELF or custom binary format. Step S302, loading the image and starting the target operation unit, specifically comprising: writing the above-mentioned image file into the storage area accessible by the target operation unit, such as on-chip RAM, external SRAM, NOR / NAND Flash, etc. by the external control core (such as host CPU, internal host of SoC, debugger or emulator, etc.). Then, the external control core releases the reset state of the target operation unit through the control signal or register configuration, so that it starts to execute the embedded runtime program contained in the image. Step S303, data segment allocation and initialization, specifically comprising: after the target operation unit is powered on and started, the self-loading code preset in the runtime program is executed. The code is responsible for allocating the actual address space of the data segment at runtime, and copying the initialization data from the "data initialization area" in the image to the allocated RAM area, thereby completing the initialization of the data segment. Step S304, saving the data segment base address, specifically comprising: after the data segment initialization is completed, the target operation unit writes the allocated data segment base address into a dedicated base address register for subsequent program use. Step S305, base address-based access, specifically comprising: the subsequent instructions in the embedded runtime program achieve relative addressing or offset addressing of the data segment by reading the value of the base address register, thereby accessing the initialized global variables, static data, etc.

[0051] Further, in some embodiments, a parameter information format for embedded runtime program building computing kernels is provided. The parameter information format can be recognized and parsed by the runtime program, and accordingly the runtime environment and initialization parameters are configured for the target computing kernel. The format includes the following: 1) control information and configuration parameter area, which is used to provide the basic control information required for computing kernel initialization, including but not limited to: a) computing kernel stack space base address: used to set the starting address of the stack space used by the computing kernel; b) computing kernel stack space size: indicates the total number of bytes of the stack space; c) integer parameter number: indicates the number of integer parameters that the computing kernel will accept; d) floating point parameter number: indicates the number of floating point parameters that the computing kernel will accept; e) other optional configuration information: such as thread priority, thread entry address, start flag, etc. 2) common parameter area (thread shared parameters), which is used to store common parameters for all computing kernel threads, i.e. parameters that can be shared by threads running on different operation units, which are divided by parameter type into: a) integer parameter area: used to save parameter values passed through integer registers; b) floating point parameter area: used to save parameter values passed through floating point registers; wherein the above parameters are usually arranged in order, so that the embedded runtime program can parse and fill the corresponding registers or memory area according to the parameter number and type. 3) private parameter area (for each operation unit), specifically including: this part is used to provide a specific private parameter set for each computing kernel thread. When starting the target computing kernel thread, the runtime program selects the corresponding private parameters from this area according to the ID of the operation unit (such as core number, cluster number, processor index, etc.), and builds a personalized execution environment. The type and structure of private parameters can be similar to public parameters, or can be extended to a structure form. For example, the private parameter structure can include the following fields: i) integer parameter array (each core has an independent set); ii) floating point parameter array; iii) specific configuration flag or debugging parameter.

[0052] In some embodiments, the method for loading a computing core on a processing unit is also provided in the embodiments of the present disclosure, wherein the method is executed by a resident embedded program running on the processing unit, and the method comprises: saving a current running context of the resident program in response to an execution request for a target computing core; allocating a running storage space for the target computing core according to computing core parameter information associated with the target computing core, wherein the computing core parameter information at least comprises parameter data elements and thread data elements; determining private global variable data corresponding to the current processing unit based on the thread data elements, and loading the private global variable data into the allocated running storage space while completing dynamic address binding of the private global variable data; constructing a running stack in the allocated running storage space based on the parameter data elements, and filling entry program parameters; jumping to an entry program address of the target computing core to execute the target computing core; and restoring the saved running context of the resident program after the execution of the target computing core is completed.

[0053] Further, in some embodiments, the resident embedded program runs on multiple processing units in a shared manner; read-only data in the resident program is bound to a program counter (PC) to achieve address-independent access; non-read-only data in the resident program is bound to a base address register (GPR); and the resident program dynamically determines the value of the base address register according to identification information of the processing unit where the resident program is located to locate the storage address of the non-read-only data at runtime.

[0054] Further, in some embodiments, the target computing core is a program set generated through static compilation, the program set contains an entry program and at least one subprogram, and the code and read-only data in the program set are bound to a program counter (PC) and packaged as address-independent code elements when being compiled and linked; and global variable data of the program set is decoupled from a selected base address register (GPR) at the time of compilation to form thread data elements.

[0055] Further, in some embodiments, the thread data elements contain differentiated global variable data corresponding to different processing units; and the step of determining private global variable data corresponding to the current processing unit based on the thread data elements specifically comprises: indexing and extracting global variable data exclusive to the current processing unit from the thread data elements according to the identification of the current processing unit.

[0056] Further, in some embodiments, the above-mentioned computing core parameter information is a data structure with a predefined format, and the data structure includes: a control information area for indicating stack space configuration and parameter number of the target computing core; a public parameter area for storing an entry program parameter shared by all processing units; and a private parameter area for storing differentiated global variable data in the thread data element indexed by processing unit identification.

[0057] In summary, the present disclosure provides a complete software and hardware collaborative solution, from the compilation definition of the computing core, the design of the resident program, to the loading and scheduling mechanism at runtime, effectively solving the problem of flexible and efficient loading and execution of the computing core on the resource-constrained AI processor.

[0058] In some embodiments, a processing unit for an intelligent computing device is also provided, including: a processing core; a memory; wherein the processing core is configured to execute instructions stored in the memory to implement the functions of the resident runtime program in the above-mentioned method of loading a computing core on a processing unit.

[0059] In some embodiments, an intelligent computing device is also provided, including: a plurality of processing units as described above; and an external management unit configured to issue a computing core execution request and description information of the computing core to the plurality of processing units.

[0060] In some embodiments, a computing core compilation method is also provided for generating a computing core that can be loaded and executed by the above-mentioned method, including: compiling and linking the source code to generate address-independent code elements, the code elements containing executable code and read-only data; separating the global variable data in the source code to form a thread data element, and generating relative addressing instructions that depend on the base address register to access these global variables; and generating a parameter data element description structure for describing the parameter information required by the computing core entry function.

[0061] In other embodiments, an electronic device is also provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method when executing the computer program.

[0062] In yet other embodiments, a computer-readable storage medium is also provided, having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the above-mentioned method.

[0063] It should be understood that the embedded program in the embodiments of the present disclosure focuses on the loading of the data segment, so that the "compute core" can be executed at any position, and the data segment is decoupled, and the kernel-level data and the thread-level data are distinguished, which has less overhead than the traditional GOT method. In addition, in some embodiments of the present disclosure, the call stack information of the compute core is also constructed, and the parameter transmission is completed through the call stack, so that the compute core does not need to perceive the use mode of the input parameter.

[0064] It should also be understood that the "resident program" in the present disclosure is required to capture the exception of the "compute core", and the "resident program" is required to "kill the compute core" after the exception of the "compute core". The problem that needs to be additionally handled is that the "resident program" and the "compute core" are at the same privilege level, which is different from the conventional user-kernel mode, and it is required to ensure that the "resident program" still has normal running ability after the exception of the "compute core" at the same privilege level.

[0065] The above has described the embodiments of the present disclosure, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles, practical application, or technical improvement in the market of the embodiments, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

[0066] The above is only optional embodiments of the present disclosure, and is not used to limit the present disclosure. For those skilled in the art, the present disclosure can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for loading a computing core onto a processing unit, characterized in that, Applied to an intelligent computing device comprising multiple processing units, the method includes: running a resident runtime program on each processing unit; the resident runtime program responding to a computing core execution request and performing the following steps: S1: Save the runtime context of the current processing unit; S2: Obtain the description information of the target computing core, the description information including at least: address-independent code metadata, thread data metadata, and parameter data metadata; S3: Based on the description information, construct an execution environment for the target computing core. The construction of the execution environment includes: allocating storage space for the thread data element and completing the dynamic binding of its running address, and setting the computing core entry parameters according to the parameter data element. S4: Jump to the entry address of the code element and start the execution of the target computing core; The code element contains executable code and read-only data that are compiled and linked, have addresses independent of the program counter (PC), and are associated with the program counter (PC). The thread data element contains global variable data that needs to be dynamically bound to the runtime address at load time. The parameter data element information includes public parameters, and the thread data element includes private parameters. The public parameters are used to pass to all processing units that execute the same target computing core, and the private parameters are used to pass to specific processing units. The computing core is set as a program assembly containing multiple subroutines, with one and only one entry point, and supports calls to global variables, local variables, and internal subroutines.

2. The method according to claim 1, characterized in that, The process of running a resident runtime program on each processing unit includes: All processing units share the same read-only code image of the resident runtime program; When the resident runtime program is running, it dynamically determines the value of a first base address register based on the identification information of the processing unit it is in, and binds the access of its non-read-only data to the memory region with the value of the first base address register as the base address.

3. The method according to claim 1 or 2, characterized in that, In step S3, allocating storage space for the thread data element and dynamically binding its runtime address includes: Allocate a space in the private memory area of ​​the current processing unit to store the thread data elements; Load the starting address of this space into a second base address register; The instructions of the target computing core address its global variables by accessing the second base address register.

4. The method according to claim 1, characterized in that, In step S3, setting the calculation kernel input parameters according to the parameter data elements includes: Select and set the corresponding private parameters based on the identification information of the current processing unit.

5. The method according to claim 1, characterized in that, After the target computing core has finished executing, the method further includes: The resident runtime program restores the runtime context saved in step S1 and returns to the ready state to wait for the next compute core execution request.

6. A processing unit for an intelligent computing device, characterized in that, include: Processing core; Memory; The processing core is configured to execute instructions stored in the memory to implement the resident runtime program in the method of loading a computing core on a processing unit as described in any one of claims 1 to 5.

7. An intelligent computing device, characterized in that, include: Multiple processing units as described in claim 6; An external management unit is configured to send a computing core execution request and a description of the computing core to the plurality of processing units.

8. A method for compiling a computational kernel, used to generate a computational kernel loaded and executed by the method according to any one of claims 1 to 5, characterized in that, include: The source code is compiled and linked to generate address-independent code elements, which contain executable code and read-only data; The global variable data in the source code is separated into thread data elements, and relative addressing instructions that depend on the base address register are generated to access these global variables. Generate a parameter data element description structure to describe the parameter information required to compute the kernel entry function.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processing unit, it implements the method as described in any one of claims 1 to 5.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in any one of claims 1 to 5.

Images