A memory allocation method and computing device

Abstract

This application provides a memory allocation method and computing device, relating to the field of operator development technology, for determining a memory allocation scheme during operator development so that developers can develop operators based on the memory allocation scheme. The method includes: obtaining an operation sequence of a target operator, the operation sequence including at least one set of operands with temporal relationships, each set of operands including at least two operands; determining memory conflict relationships between operands based on the operation sequence, the memory conflict relationships describing whether different operands can be stored in the same memory block; and determining a target memory allocation scheme based on the memory conflict relationships; the target memory allocation scheme includes a target number of memory blocks obtained by dividing the target memory, and the operands to be stored in each memory block.

Description

Technical Field

[0001] This application relates to the field of operator development technology, and in particular to a memory allocation method and computing device. Background Technology

[0002] During operator computation, the processor transfers source data from main memory to the on-chip cache, invokes vector computation instructions to perform calculations, stores the computation results in the on-chip cache, and finally transfers the final computation result back to main memory, completing the computation of a portion of the data slice. By storing frequently accessed data in the fast-access on-chip cache, access to main memory is reduced, thereby improving computational efficiency.

[0003] For devices where on-chip caches can be partitioned by developers at any granularity, during operator development, developers need to allocate memory for the data generated during operator execution. A memory allocation scheme refers to the method for allocating memory for the data generated during operator execution; the quality of the memory allocation scheme directly affects the efficiency of on-chip cache utilization and the throughput of computing units.

[0004] Analyzing memory reuse strategies for complex operators is a rather tedious and time-consuming task. Moreover, developers may not consider all memory allocation schemes and may easily miss better ones. Summary of the Invention

[0005] This application provides a memory allocation method and computing device for determining a memory allocation scheme during operator development so that developers can develop operators based on the memory allocation scheme.

[0006] To achieve the above technical objectives, this application adopts the following technical solution:

[0007] In a first aspect, embodiments of this application provide a memory allocation method applied to an operator development platform; the method includes:

[0008] Obtain the operation sequence of the target operator. The operation sequence includes at least one set of operands with temporal relationship, and each set of operands includes at least two operands.

[0009] Based on the operation sequence, the memory conflict relationship between operands is determined. The memory conflict relationship is used to describe whether different operands can be stored in the same memory block.

[0010] Based on memory conflict relationships, a target memory allocation scheme is determined. The target memory allocation scheme includes the target number of memory blocks obtained by dividing the target memory, and the number of operations to be stored in each memory block.

[0011] The technical solution provided in this application offers at least the following benefits: By obtaining the operation sequence of the target operator, the memory conflict relationship between operands can be determined, that is, whether there is a conflict of physical memory resources between operands generated during the calculation of the target operator, i.e., whether different operands can be stored in the same memory block. Based on the memory conflict relationship, a target memory allocation scheme can be determined, allowing developers to develop the target operator based on the target memory allocation scheme and operation sequence. The target memory allocation scheme is automatically determined by the operator development platform. The determination of the memory allocation scheme is fast, reducing user operation steps, allowing for the rapid acquisition of multiple memory allocation schemes, and enabling the testing of multiple memory allocation schemes in a short time to find the optimal memory allocation scheme. This helps developers find the optimal memory allocation scheme and assists them in developing operators based on the memory allocation scheme.

[0012] In one possible implementation, memory conflict relationships are represented by a memory conflict graph. The memory conflict graph includes multiple nodes and at least one edge. Different nodes are used to represent different operands in the operation sequence. Any two operands in the same group are connected by an edge. Operands corresponding to two nodes connected by an edge cannot be stored in the same memory block.

[0013] In one possible implementation, memory conflict relationships are represented by a memory conflict graph. The memory conflict graph includes multiple nodes and at least one edge. Different nodes are used to represent different operands in the operation sequence. Nodes corresponding to operands that overlap between any two active regions are connected by an edge. Operands corresponding to two nodes connected by an edge cannot be stored in the same memory block. One end value of the active region is used to represent the timing of the first group of operands in the operation sequence, and the other end value is used to represent the timing of the last group of operands in the operation sequence.

[0014] In one possible implementation, determining a target memory allocation scheme based on memory conflict relationships includes: traversing the operands used in the operation sequence and adding tags to the traversed operands according to the memory conflict relationships to obtain at least one memory allocation scheme; wherein the tags are selected from any one of K, where K is a positive integer greater than or equal to 1; different tags represent different memory blocks; different tags are added to any two operands that cannot be stored in the same memory block; and a target memory allocation scheme is determined based on at least one memory allocation scheme, where the target number is the number of tags corresponding to the target memory allocation scheme.

[0015] In one possible implementation, the method further includes: outputting a target memory allocation scheme, wherein the target memory allocation scheme is a coloring map; in the coloring map, memory blocks with different markings are represented by different colors, and the node corresponding to the operand is assigned a first color, indicating that the operand is stored in the memory block represented by the first color, wherein the first color is any color.

[0016] In one possible implementation, if a first memory block is allocated for the first operand, the method further includes: if the second operand also corresponds to the first memory block, and the timing of the first occurrence of the first operand in the operation sequence is earlier than the timing of the first occurrence of the second operand in the operation sequence, and the timing of the last occurrence of the first operand in the operation sequence is later than the timing of the first occurrence of the second operand in the operation sequence, then a second memory block is allocated for the first operand in memory at a storage level lower than the target memory and adjacent to the target memory. The second memory block is used to store the first operand when the second operand is stored in the first memory block.

[0017] In one possible implementation, determining a target memory allocation scheme based on at least one memory allocation scheme includes: when a memory allocation scheme satisfies the memory allocation condition for the first time, determining the memory allocation scheme that satisfies the memory allocation condition as the target memory allocation scheme; the memory allocation condition is used to characterize at least one of the following: whether overflow operands are allowed, the upper limit of overflow operands, or the range of values ​​for K; wherein, in the memory allocation scheme, an overflow operand refers to an operand that is overwritten by other operands before its last occurrence in the operation sequence.

[0018] In one possible implementation, determining a target memory allocation scheme based on at least one memory allocation scheme includes: for each memory allocation scheme, determining a scheme score for the memory allocation scheme based on scheme evaluation parameters; and determining a target memory allocation scheme based on the scheme score for each memory allocation scheme. The scheme evaluation parameters include: an overflow parameter and / or an idle parameter. The overflow parameter is used to characterize the number of overflow operands in the memory allocation scheme, where an overflow operand is an operand that is overwritten by other operands before its last occurrence in the operation sequence. The idle parameter is used to characterize the utilization rate of memory blocks in the target memory.

[0019] In one possible implementation, the target memory allocation scheme is determined based on the scheme score of each memory allocation scheme, including: when the scheme evaluation parameters only include the overflow parameter, selecting the memory allocation scheme with the minimum overflow cost as the target memory allocation scheme; when the scheme evaluation parameters only include the idle parameter, selecting the memory allocation scheme with the minimum idle cost as the target memory allocation scheme; when the scheme evaluation parameters include both the overflow parameter and the idle parameter, selecting the memory allocation scheme with the minimum total cost as the memory allocation scheme; wherein the overflow cost is determined based on the overflow parameter, the idle cost is determined based on the idle parameter, and the total cost is determined based on both the overflow parameter and the idle parameter.

[0020] In one possible implementation, obtaining the operation sequence of the target operator includes: obtaining multiple sets of operands input by the user and the temporal relationship between the multiple sets of operands; or, obtaining the expression in the target operator input by the user, parsing the expression to obtain multiple sets of operands, and combining the multiple sets of operands sequentially according to the temporal relationship to obtain the operation sequence.

[0021] In one possible implementation, the operation sequence includes multiple operation tuples, each representing a set of operands. The order of the multiple operation tuples matches the temporal order of the multiple sets of operands. Each operation tuple includes multiple operands with an order relationship. The Nth operand in the operation tuple represents the destination operand in the set of operands corresponding to the operation tuple, and the other operands in the operation tuple, excluding the Nth operand, represent the source operands in the set of operands corresponding to the operation tuple.

[0022] In one possible implementation, the capacity of each memory block is matched to the amount of data of the operands stored in the memory block.

[0023] Secondly, embodiments of this application provide a memory allocation apparatus, comprising:

[0024] The acquisition module is used to acquire the operation sequence of the target operator. The operation sequence includes at least one set of operands with temporal relationship, and each set of operands includes at least two operands.

[0025] The processing module is used to determine the memory conflict relationship between operands based on the operation sequence. The memory conflict relationship is used to describe whether different operands can be stored in the same memory block.

[0026] The processing module is also used to determine a target memory allocation scheme based on memory conflict relationships; the target memory allocation scheme includes the target number of multiple memory blocks obtained by dividing the target memory, and the number of operations to be stored in each memory block.

[0027] In one possible implementation, memory conflict relationships are represented by a memory conflict graph. The memory conflict graph includes multiple nodes and at least one edge. Different nodes represent different operands in the operation sequence. Any two operands in the same group are connected by an edge, and the operands corresponding to two nodes connected by an edge cannot be stored in the same memory block. The processing module is specifically used to: construct a node for each group of operands in the operation sequence, with the same operands in different groups corresponding to the same node; construct an edge between any two operands in the same group to obtain the memory conflict graph; wherein, the operands corresponding to two nodes connected by an edge cannot be stored in the same memory block.

[0028] In one possible implementation, memory conflict relationships are represented by a memory conflict graph. The memory conflict graph includes multiple nodes and at least one edge. Different nodes represent different operands in the operation sequence. Nodes corresponding to operands with overlapping active regions are connected by an edge. Operands corresponding to two nodes connected by an edge cannot be stored in the same memory block. One end value of the active region represents the timing of the first occurrence of the operand in the operation sequence, and the other end value represents the timing of the last occurrence of the operand in the operation sequence. Specifically, the processing module is used to: construct a node for each operand in the operation sequence, with identical operands in different groups corresponding to the same node; for each operand in the operation sequence, determine the active region of the operand; one end value of the active region represents the timing of the first occurrence of the operand in the operation sequence, and the other end value represents the timing of the last occurrence of the operand in the operation sequence; construct an edge between nodes corresponding to operands with overlapping active regions to obtain the memory conflict graph; operands corresponding to two nodes connected by an edge cannot be stored in the same memory block.

[0029] In one possible implementation, the processing module is specifically used to: traverse the operands used in the operation sequence, and add a mark to the traversed operands according to the memory conflict relationship to obtain at least one memory allocation scheme; wherein the mark is selected from any one of K, where K is a positive integer greater than or equal to 1; different marks represent different memory blocks; different marks are added to any two operands that cannot be stored in the same memory block; and determine the target memory allocation scheme according to the at least one memory allocation scheme, where the target number is the number of marks corresponding to the target memory allocation scheme.

[0030] In one possible implementation, the memory allocation device further includes an output module; the output module is used to output a target memory allocation scheme, wherein the target memory allocation scheme is a coloring map; in the coloring map, memory blocks with different markings are represented by different colors, and the node corresponding to the operand is assigned a first color, indicating that the operand is stored in the memory block represented by the first color, wherein the first color is any color.

[0031] In one possible implementation, if a first memory block has been allocated for the first operand, the processing module is further configured to: if the second operand also corresponds to the first memory block, and the timing of the first occurrence of the first operand in the operation sequence is earlier than the timing of the first occurrence of the second operand in the operation sequence, and the timing of the last occurrence of the first operand in the operation sequence is later than the timing of the first occurrence of the second operand in the operation sequence, then allocate a second memory block for the first operand in memory whose storage level is lower than the target memory and whose storage level is adjacent to the target memory. The second memory block is used to store the first operand when the second operand is stored in the first memory block.

[0032] In one possible implementation, the processing module is specifically used to: when a memory allocation scheme meets the memory allocation conditions for the first time, determine the memory allocation scheme that meets the memory allocation conditions as the target memory allocation scheme; the memory allocation conditions are used to characterize at least one of the following: whether overflow operands are allowed, the upper limit of overflow operands, or the range of values ​​for K; wherein, in the memory allocation scheme, an overflow operand refers to an operand that is overwritten by other operands before its last occurrence in the operation sequence.

[0033] In one possible implementation, the processing module is specifically used to: for each memory allocation scheme, determine the scheme score of the memory allocation scheme based on the scheme evaluation parameters; and determine the target memory allocation scheme based on the scheme score of each memory allocation scheme. The scheme evaluation parameters include: overflow parameters and / or idle parameters. The overflow parameters are used to characterize the number of overflow operands in the memory allocation scheme. Overflow operands refer to operands that are overwritten by other operands before their last occurrence in the operation sequence. The idle parameters are used to characterize the utilization rate of memory blocks in the target memory.

[0034] In one possible implementation, the processing module is specifically used to: select the memory allocation scheme with the lowest overflow cost as the target memory allocation scheme when the scheme evaluation parameters only include the overflow parameter; select the memory allocation scheme with the lowest idle cost as the target memory allocation scheme when the scheme evaluation parameters only include the idle parameter; and select the memory allocation scheme with the lowest total cost as the memory allocation scheme when the scheme evaluation parameters include both the overflow parameter and the idle parameter. The overflow cost is determined based on the overflow parameter, the idle cost is determined based on the idle parameter, and the total cost is determined based on both the overflow parameter and the idle parameter.

[0035] In one possible implementation, the acquisition module is specifically used to: acquire multiple sets of operands input by the user and the temporal relationship between the multiple sets of operands; or, acquire the expression in the target operator input by the user, parse the expression to obtain multiple sets of operands, and combine the multiple sets of operands sequentially according to the temporal relationship to obtain the operation sequence.

[0036] In one possible implementation, the operation sequence includes multiple operation tuples, each representing a set of operands. The order of the multiple operation tuples matches the temporal order of the multiple sets of operands. Each operation tuple includes multiple operands with an order relationship. The Nth operand in the operation tuple represents the destination operand in the set of operands corresponding to the operation tuple, and the other operands in the operation tuple, excluding the Nth operand, represent the source operands in the set of operands corresponding to the operation tuple.

[0037] In one possible implementation, the capacity of each memory block is matched to the amount of data of the operands stored in the memory block.

[0038] Thirdly, this application provides a computing device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the memory allocation methods provided in the first aspect above.

[0039] Fourthly, this application provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement any of the memory allocation methods provided in the first aspect above.

[0040] Fifthly, this application provides a computer program product including computer instructions that, when executed by a processor, implement any of the memory allocation methods provided in the first aspect above.

[0041] For a detailed description of the second to fifth aspects and their various implementations in this application, please refer to the detailed description in the first aspect and its various implementations; and for a detailed analysis of the beneficial effects of the second to fifth aspects and their various implementations in the first aspect and its various implementations, please refer to the beneficial effect analysis in the first aspect and its various implementations, which will not be repeated here.

[0042] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0043] Figure 1 A schematic diagram of the structure of a processor for operator deployment provided in an embodiment of this application;

[0044] Figure 2 This is a schematic diagram of the structure of an AI Core provided in an embodiment of this application;

[0045] Figure 3 A schematic diagram of a programming paradigm provided in an embodiment of this application;

[0046] Figure 4 This is a schematic diagram of a memory allocation scenario provided in an embodiment of this application;

[0047] Figure 5 A schematic diagram of the architecture of a memory allocation system to which a memory allocation method is applicable, as provided in an embodiment of this application;

[0048] Figure 6 This application provides a schematic diagram of the hardware structure of a computing device according to an embodiment of the present application.

[0049] Figure 7 A flowchart illustrating a memory allocation method provided in an embodiment of this application;

[0050] Figure 8 A scenario illustration of a memory allocation method provided in this application embodiment. Figure 1 ;

[0051] Figure 9 A scenario illustration of a memory allocation method provided in this application embodiment. Figure 2 ;

[0052] Figure 10 A scenario illustration of a memory allocation method provided in this application embodiment. Figure 3 ;

[0053] Figure 11 A scenario illustration of a memory allocation method provided in this application embodiment. Figure 4 ;

[0054] Figure 12 A scenario illustration of a memory allocation method provided in this application embodiment. Figure 5 ;

[0055] Figure 13 A scenario illustration of a memory allocation method provided in this application embodiment. Figure 6 ;

[0056] Figure 14 A scenario illustration of a memory allocation method provided in this application embodiment. Figure 7 ;

[0057] Figure 15 This is a schematic diagram of a memory allocation device provided in an embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0060] Operators: Deep learning algorithms are composed of operators (OPs). In a network model, an operator corresponds to the computational logic within a layer. For example, a convolutional layer is an operator; the weight summation process in a fully-connected layer (FC layer) is also an operator. In mathematics, a mapping O: X → Y from one function space to another is called an operator. More broadly, any operation performed on a function can be considered an operator, such as differential operators and indefinite integral operators.

[0061] The above are the technical terms used in this application, and will not be repeated below.

[0062] In AI computing, operators are the basic computational units that make up neural network models. The development and optimization of operators determine the model's performance, energy efficiency, and deployment flexibility. For example, the AI ​​Core is the computational core of an AI processor, such as... Figure 1 As shown, an AI processor may include an AI Core, a CPU, etc. The operators executed within the AI ​​Core are called TBE (TensorBoost Engine) operators, responsible for performing computationally intensive operator tasks involving matrices, vectors, and scalars. For example, ... Figure 2 As shown, the AI ​​Core can include a computing unit, a storage unit, and a control unit. The computing unit includes an accumulator, a cube unit, a vector unit, and a scalar unit. The storage unit includes a memory transfer engine (MTE), a bus interface unit (BIU), a buffer L1, a buffer L0A, a buffer L0B, a buffer L0C, a unified buffer (UB), a scalar buffer, a general-purpose register (GPR), and a special-purpose register (SPR). The control unit includes a system control module, an instruction cache module, a scalar instruction processing queue (Scalar PSQ), an instruction dispatch module, a cube queue, a vector queue, a memory transfer queue (MTE Queue), and an event synchronization module. The calculation unit is used to implement the calculation process of the operator, the UB in the storage unit is used to store the data generated during the calculation process of the operator, and the control unit provides instruction control for the entire calculation process.

[0063] During operator computation, the processor transfers source data from main memory to the on-chip cache, invokes vector computation instructions to perform the computation, stores the computation result in the on-chip cache, and finally transfers the final computation result back to main memory, completing the computation of a portion of the data slice. By storing frequently accessed data in the fast-access on-chip cache, access to main memory is reduced, thus improving computational efficiency. For example, when performing vector programming, developers can follow the following... Figure 3 The programming paradigm shown is as follows: data is transferred from the main memory (Global Memory, GM) (the main memory and AI Core are deployed together in the processor) to the on-chip cache (Unified Buffer, UB) (also known as Local Memory), then vector calculation instructions are called to perform calculations and the calculation results are stored on the UB. Finally, the results are transferred from the UB to the GM, thus completing the calculation of a part of the data slice.

[0064] For devices where on-chip caches can be partitioned by developers at any granularity, during operator development, developers need to allocate memory for the data generated during operator execution. A memory allocation scheme refers to the method of allocating memory for the data generated during operator execution. The quality of the memory allocation scheme directly affects the efficiency of on-chip cache utilization and the throughput of the computing unit. An example of memory allocation is a binaristic addition operator, considering two input vectors (vector A and vector B) and one output vector, such as... Figure 4 As shown in (a), the on-chip cache can be divided into three blocks to store the two input vectors and the output vector respectively. Alternatively, after obtaining the output vector, the two previously input vectors will not be retrieved again, as shown in (a). Figure 4 The output vector shown in (b) can reuse the memory block of one of the input vectors, so it is obvious that the on-chip cache can be divided into two equal parts. For example, the on-chip cache could be the aforementioned UB. Figure 4 In the diagram (a), UB is divided into 3 parts. Figure 4 (b) in the diagram divides UB into two parts.

[0065] Analyzing memory reuse strategies for complex operators is a tedious and time-consuming task. Furthermore, developers may not consider all memory allocation schemes comprehensively, easily overlooking better alternatives. Therefore, a solution is needed that can automatically analyze the memory allocation strategies of complex operators to help developers find the optimal memory allocation scheme and assist them in developing operators based on that scheme.

[0066] This application provides a memory allocation method. By obtaining the operation sequence of a target operator, which includes multiple computational operations with temporal relationships and each operation contains at least one operand, the method determines the memory conflict relationship between operands. This means determining whether there is a conflict of physical memory resources between operands generated during the target operator's computation, i.e., whether different operands can be stored in the same memory block. Based on the memory conflict relationship, a target memory allocation scheme can be determined, i.e., determining the target number of memory blocks to be divided into and the operands to be stored in each memory block. After the target memory allocation scheme is output, developers can develop the target operator based on the target memory allocation scheme and the operation sequence. The target memory allocation scheme is automatically determined by the operator development platform. The determination of the memory allocation scheme is fast, reducing user steps and allowing for the rapid generation of multiple memory allocation schemes. Multiple memory allocation schemes can also be tested in a short time to find the optimal one, thus helping developers find the optimal memory allocation scheme and assisting them in developing operators based on it.

[0067] The memory allocation method provided in this application can be applied to computing devices.

[0068] In one implementation, an operator development platform is deployed in the computing device. The computing device can obtain the operation sequence corresponding to an operator through the operator development platform, determine the memory conflict relationship corresponding to the operator based on the operation sequence, and thus determine the target memory allocation scheme corresponding to the operator based on the memory conflict relationship. Developers can then program the operator on the operator development platform based on the target memory allocation scheme to complete the operator development.

[0069] To achieve these functions, the computing device may optionally include an operation sequence acquisition module, a memory conflict relationship determination module, and a memory allocation scheme determination module.

[0070] The operation sequence acquisition module is used to obtain the operation sequence of the target operator. This operation sequence can be multiple calculation operations that the developer inputs sequentially according to the temporal relationship, or it can be an expression input by the developer. The operation sequence acquisition module parses the expression to obtain an operation sequence composed of multiple calculation operations with temporal relationship.

[0071] In practical applications, an operation sequence can take the form of a basic block of arithmetic instructions. This basic block is a pseudocode segment that precisely describes the computation steps.

[0072] The memory conflict determination module is used to determine the memory conflict relationship between operands based on the operation sequence, and describes whether different operands in the operation sequence can be stored in the same memory block.

[0073] The memory allocation scheme determination module is used to determine the memory allocation scheme corresponding to the operator based on the memory conflict relationship, so that developers can develop the operator based on the memory allocation scheme.

[0074] Optionally, the memory allocation scheme determination module can be further divided into a memory allocation scheme generation module and a memory allocation scheme evaluation module. The memory allocation scheme generation module generates multiple memory allocation schemes based on memory conflict relationships, while the memory allocation scheme evaluation module evaluates the multiple memory allocation schemes according to the indicators set by the developer and selects the target memory allocation scheme that best meets the developer's indicator requirements.

[0075] In some embodiments, the computing device may further include an operator generation module. The operator generation module is used to generate operator code based on a target memory allocation scheme. In practice, the operator generation module may be a code generator or a component of a code generator.

[0076] Optionally, the memory allocation method provided in this application can also be applied to, for example... Figure 5 The memory allocation system 1 shown includes: terminal device 10 and server 20.

[0077] In this process, a communication connection is established between the terminal device 10 and the server 20. It should be understood that the connection method can be a wireless connection, such as a Bluetooth connection or a wireless fidelity (Wi-Fi) connection; or, the connection method can be a wired connection, such as a fiber optic connection, etc., and there is no limitation on this.

[0078] In some embodiments, server 20 can be used to determine the memory conflict relationship between operands based on the operation sequence of the target operator, and then determine the target memory allocation scheme corresponding to the target operator.

[0079] In some embodiments, the terminal device 10 and the server 20 work together to perform memory allocation. Specifically, the developer inputs the operation sequence of the target operator through the terminal device 10, which then sends the operation sequence to the server 20. The server 20 determines memory conflict relationships based on the operation sequence of the target operator, and then determines the target memory allocation scheme corresponding to the target operator based on the memory conflict relationships. The server 20 then sends the target memory allocation scheme to the terminal device 10, so that the developer can develop operators on the terminal device 10 according to the target memory allocation scheme.

[0080] In some embodiments, the user configures the generation constraints of the memory allocation scheme and the evaluation criteria of the target memory allocation scheme through the terminal device 10. The terminal device 10 sends the generation constraints of the memory allocation scheme and the evaluation criteria of the memory allocation scheme to the server 20. The server 20 generates the memory allocation scheme according to the generation constraints of the memory allocation scheme, and then selects the target memory allocation scheme that best meets the user's needs according to the evaluation criteria of the memory allocation scheme.

[0081] In some embodiments, the terminal device 10 and the server 20 can be as follows: Figure 5 The diagram shows two separate devices, or the terminal device 10 and the server 20 can be integrated together.

[0082] In some embodiments, the terminal device 10 may be a device capable of supporting operator development. For example, the terminal device 10 may be a personal computer (PC), a laptop computer, a mobile device, a tablet computer, a laptop computer, etc. This application does not limit the specific form of the terminal device 10.

[0083] In some embodiments, server 20 may be a single server or a server cluster.

[0084] When the terminal device 10 and the server 20 are separate, the terminal device 10 can be regarded as the aforementioned computing device; when the terminal device 10 and the server 20 are integrated into one device, the device can be regarded as the aforementioned computing device.

[0085] Figure 6 This is a schematic diagram of the hardware structure of a computing device provided in an embodiment of this application. The hardware structure of the computing device includes... Figure 6 The components included in the computing device shown. See also Figure 6 , Figure 6 The computing device 300 shown may include a processor 301, a memory 302, a communication interface 303, and a bus 304. The processor 301, the memory 302, and the communication interface 303 can be connected via the bus 304.

[0086] The processor 301 is the control center of the computing device 300, and can be a general-purpose central processing unit (CPU) or other general-purpose processors. The general-purpose processor can be a microprocessor or any conventional processor. In this embodiment, the memory allocation method can be executed by the processor 301 in the computing device 300. The specific structure of the processor 301 can be as follows: Figure 1 As shown.

[0087] As an example, processor 301 may include one or more CPUs, for example Figure 6 CPU 0 and CPU 1 are shown in the diagram.

[0088] The memory 302 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, or other type of dynamic storage device capable of storing information and instructions. It may also be an electrically erasable programmable read-only memory (EEPROM), a disk storage medium, or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In this embodiment, the memory 302 in the computing device 300 may be used to store information such as data stream transmission status, switch port data, etc.

[0089] In one possible implementation, the memory 302 may exist independently of the processor 301. The memory 302 can be connected to the processor 301 via a bus 304 and is used to store data, instructions, or program code. When the processor 301 calls and executes the instructions or program code stored in the memory 302, it can implement the memory allocation method provided in the embodiments of this application.

[0090] In another possible implementation, the memory 302 can also be integrated with the processor 301.

[0091] The communication interface 303 is used for connecting the computing device 300 to other devices via a communication network, which may be Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The communication interface 303 may include a receiving unit for receiving data and a transmitting unit for transmitting data.

[0092] Bus 304 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0093] It should be pointed out that, Figure 6 The structure shown does not constitute a limitation on the computing device 300, except Figure 6 In addition to the components shown, the computing device 300 may also include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0094] The implementation methods of the embodiments of this application will now be described in detail with reference to the accompanying drawings. The memory allocation method provided in the embodiments of this application will be explained using the above-described computing device as an example.

[0095] like Figure 7 As shown in the figure, this application provides a memory allocation method, which includes the following steps:

[0096] S101, Obtain the operation sequence of the target operator.

[0097] The operation sequence includes at least one set of operands with temporal relationship, and each set of operands includes at least two operands.

[0098] Operands are the entities on which operators act; they are components of an expression. Operands specify the quantities to be calculated in the computation instruction. An expression is a combination of operands and operators.

[0099] In some embodiments, step S101 can be specifically implemented as: obtaining multiple sets of operands input by the user and the temporal relationship between the multiple sets of operands; or, obtaining the expression in the target operator input by the user, parsing the expression to obtain multiple sets of operands, and combining the multiple sets of operands sequentially according to the temporal relationship to obtain the operation sequence.

[0100] For example, the expression corresponding to the target operator is: The user breaks down the expression of the target operator into three steps according to the vector operation instructions in the programming: ①T1=AB; ②T2=A+B; ③ The operands in a step are grouped together, and the steps corresponding to a group of operands can be called a computation operation. The timing relationship between these three computation operations is the timing relationship of multiple groups of operands manually entered by the user, which are ①②③ in sequence.

[0101] Another example is that the user directly inputs the expression corresponding to the target operator. Parsing the expression yields three steps: ④T1=AB, ⑤T2=A+B, ⑥ The temporal relationship between these three computational operations can be determined as ④⑤⑥, or the temporal relationship between these three computational operations can be determined as ⑤④⑥.

[0102] The operation sequence can represent the computation process of the target operator, allowing us to focus on whether the source and destination operands need to be stored independently. This enables us to allocate different memory blocks for operands requiring independent storage during subsequent memory allocation. Users can input pre-defined computation steps or directly input the computation formula, making it more flexible for practical applications.

[0103] Since this application does not focus on the specific operation of the calculation (e.g., Add, Mul, etc.), but only considers the dependency relationship between each operand (whether different operands can reuse the same memory block), useless operation type information can be removed when inputting the calculation operation, and only the input format of the dependency relationship between operands can be retained.

[0104] In one possible implementation, the operation sequence includes multiple operand tuples. Each operand tuple represents a set of operands, and the order of the operand tuples matches the temporal order of the operands. Each operand tuple contains operands with an order relationship. The Nth operand in the operand tuple with temporal order X represents the destination operand in the corresponding set of operands. The other operands in the operand tuple with temporal order X, excluding the Nth operand, represent the source operands in their respective sets of operands. For example, in AB = C, C is the destination operand, and A and B are the source operands. Another example is a × b + c = d, where d is the destination operand, and a, b, and c are the source operands.

[0105] For example, if the first operand in each operand tuple is used to represent the destination operand of the operand tuple, and the other operands in the operand tuple, excluding the first operand, are used to represent the source operands of the operand tuple, such as... Figure 8 As shown, in the operation sequence input interface, users can enter a set of operands in the parentheses "()". The first operand entered by the user in the parentheses "()" is determined as the destination operand, and the other operands entered by the user in the parentheses "()" are determined as the source operands. Users can also click the "+ Add" control to add new operation tuples. The order of the operation tuples represents the timing of multiple sets of operands in the operation sequence. Users can also click the "× Delete" control to delete existing operation tuples.

[0106] Another example is the expression corresponding to the target operator: It is broken down into three steps: ①T1 = AB; ②T2 = A + B; ③ The operation sequence can include the following operands: (T1, A, B), (T2, A, B), and (T3, T1, T2). It can be seen that the operand (T1, A, B) corresponds to the operation T1 = AB, the operand (T2, A, B) corresponds to the operation T2 = A + B, and the operand (T3, T1, T2) corresponds to the operation T1 = AB. Correspondingly, the first operand in the operand tuple is used to represent the destination operand in the computation operation corresponding to the operand tuple, and the other operands in the operand tuple are used to represent the source operands in the computation operation corresponding to the operand tuple.

[0107] Representing the computational operations of target operators using the format of operation tuples eliminates unnecessary operation type information, allowing for a more concise description of the dependencies between operands and saving storage resources. Furthermore, operation tuples offer better extensibility and ease of programming, supporting unary, binary, and even n-ary operations.

[0108] S102. Determine the memory conflict relationship between operands based on the operation sequence.

[0109] Among them, memory conflict relationship is used to describe whether different operands can be stored in the same memory block.

[0110] As described in step S101 above, operands are represented by symbols, with each symbol corresponding to a symbolized memory location. Different symbolized memory locations only indicate different operands, not different physical memory blocks. Memory conflict relationships describe whether different operands can be stored in the same memory block; in other words, they describe whether operands represented by different symbols can be allocated to the same memory block.

[0111] In some embodiments, memory conflict relationships can be represented in different forms, such as graphs, tables, etc.

[0112] In some embodiments, memory conflict relationships are represented by a memory conflict map. Determining the memory conflict map can be achieved in the following two ways:

[0113] Method 1: The memory conflict graph consists of multiple nodes, each representing a different operand in the operation sequence. Nodes corresponding to any two operands in the same group are connected by an edge. Operands corresponding to two nodes connected by an edge cannot be stored in the same memory block. For each group of operands in the operation sequence, a node is constructed for each operand, with identical operands in different groups corresponding to the same node. An edge is constructed between any two nodes corresponding to operands in the same group to obtain the memory conflict graph; where operands corresponding to two nodes connected by an edge cannot be stored in the same memory block.

[0114] For example, if operand A appears in both computation operation T1 = AB and computation operation T2 = A + B, this does not mean that there are two nodes with the symbol A in the memory conflict graph; there is exactly one node with the symbol A. For T1 = AB, T2 = A + B, For each of these three computational operations, any two operands within that operation must share an edge in the memory conflict graph. An edge between two operands indicates a memory resource conflict, meaning they should be stored separately. The expression is as follows: The memory conflict graph between operands includes the following nodes: T1, A, B, T2, T3. There is an edge between node T1 and node A, node T1 and node B, node A and node B, node T2 and node A, node T2 and node B, node T1 and node T2, node T3 and node T1, and node T3 and node T2.

[0115] By constructing nodes for each operand and edges between different operands in the same computational operation, a memory conflict graph can be obtained, which represents the dependencies between operands. This facilitates the processing of the memory relationship graph in subsequent operations to obtain a memory allocation scheme.

[0116] Method 2: The memory conflict graph consists of multiple nodes, each representing a different operand in the operation sequence. Nodes corresponding to operands in any two active regions that overlap are connected by an edge. Operands corresponding to two nodes connected by an edge cannot be stored in the same memory block. One endpoint of the active region represents the timing of the first occurrence of the operand in the operation sequence, and the other endpoint represents the timing of the last occurrence of the operand in the operation sequence. A node is constructed for each operand in the operation sequence, with identical operands in different groups corresponding to the same node. For each operand in the operation sequence, the active region of the operand is determined. One endpoint of the active region represents the timing of the first occurrence of the operand in the operation sequence, and the other endpoint represents the timing of the last occurrence of the operand in the operation sequence. An edge is constructed between nodes corresponding to operands in any two active regions that overlap, resulting in the memory conflict graph. Operands corresponding to two nodes connected by an edge cannot be stored in the same memory block. The active interval is a closed interval. According to the general representation of intervals, the left endpoint of the active interval is used to represent the order of the first occurrence of the operand among multiple groups with temporal relationships; the right endpoint of the active interval is used to represent the order of the last occurrence of the operand among multiple groups with temporal relationships.

[0117] For example, for each operand, such as operand A, find its earliest and latest occurrence positions. For the computation operations, T1 = AB, T2 = A + B, etc. In this example, operand A appears earliest in the first operation and latest in the second operation, so the active range of operand A is 1 to 2, and the active interval of operand A can be denoted as [1, 2]. Similarly, the active interval of operand T1 is [1, 3]. The active intervals of all operands are calculated, and nodes are constructed for all operands in the operation sequence. Then, an edge is constructed between nodes corresponding to operands whose active intervals overlap. For example, the active intervals of operand A and operand T1 overlap, so an edge should be constructed between node A and node T1 in the memory conflict graph. It is important to note that if two operands have active intervals of [1, 2] and [2, 3] respectively, then these two operands' active intervals also overlap, and an edge exists between these two operands in the memory conflict graph; these two operands cannot share the same memory block.

[0118] It's easy to see that for any given computational operation, the active regions of all operands appearing in that operation overlap. Therefore, the memory conflict graph constructed using method two must contain all the edges of the memory conflict graph constructed using method one. The memory conflict graph constructed using this method detects more hidden conflicts. Specifically, if an operand that is assigned a value early in the operation sequence appears in a later computational operation, then the active region of this operand will cover all computational operations between those two operations. This ensures that there will inevitably be a conflict (or overlapping active regions) between the operands involved in the computational operations within this region and operands with a large active region, preventing a variable from being overwritten even if it hasn't been used for a long time after its initial assignment. If developers don't want operands to overwrite other operands that are still needed, the memory conflict graph constructed using method two covers all conflicts. When determining a memory allocation scheme based on the memory conflict graph constructed using method two, the phenomenon of operand memory overwriting can be avoided.

[0119] If memory conflict relationships are represented in tabular form, then one of the row labels or column labels should represent the computation operation, and the other should represent the operand in the computation operation. For example, if the target operator is (A+B)*CB, resulting in an operation sequence including the computation operations: T1 = A+B, T2 = T1*C, T3 = T2-B, the memory conflict relationships in tabular form can be shown in Table 1 below:

[0120] Table 1

[0121] <![CDATA[T1=A+B]]> <![CDATA[T1]]> A B <![CDATA[T2=T1*C]]> <![CDATA[T1]]> <![CDATA[T2]]> C <![CDATA[T3=T2-B]]> <![CDATA[T3]]> <![CDATA[T2]]> B

[0122] In Table 1, row labels represent computational operations, and column labels represent operands in computational operations. Based on Table 1, it can be seen that when two different operands are in the same row, it indicates that there is a memory conflict between the two operands. When two different operands are in the same column, it indicates that there is no memory conflict between the two operands.

[0123] If memory conflict relationships are represented in the form of a graph, then the memory conflict graph corresponding to (A+B)*CB can be represented as follows: Figure 9 As shown, Figure 9 In the diagram, each operand A, B, C, T1, T2, and T3 corresponds to a node. There is an edge between each pair of A and B and T1, between each pair of C and T1 and T2, between each pair of B and T1 and T2, and between each pair of B and T2 and T3.

[0124] S103. Determine the target memory allocation scheme based on memory conflict relationships.

[0125] The target memory allocation scheme includes the target number of memory blocks obtained by dividing the target memory, and the number of operations to be stored in each memory block.

[0126] The target memory is the on-chip cache (such as the aforementioned UB) in the device on which the target operator runs, used to store intermediate data generated during the operator's computation. The target memory can be all or part of the on-chip cache. If the target memory capacity is insufficient, the operator cannot run, so the case of insufficient target memory capacity is not considered in this application.

[0127] In some embodiments, step S103 can be specifically implemented as follows: traversing the operands used in the operation sequence, and adding a mark to the traversed operands according to the memory conflict relationship to obtain at least one memory allocation scheme; wherein, the mark is selected from any one of K, where K is a positive integer greater than or equal to 1; different marks represent different memory blocks; different marks are added to any two operands that cannot be stored in the same memory block; and a target memory allocation scheme is determined according to at least one memory allocation scheme, where the target number is the number of marks corresponding to the target memory allocation scheme.

[0128] For example, the markers can be color markers, image markers, text markers, symbol markers, etc.

[0129] Based on memory conflict relationships, marking the operands encountered during the traversal to obtain at least one memory allocation scheme is the logic for coloring a graph using graph coloring methods. A graph coloring method is a method for assigning colors to the vertices of a graph; the coloring logic of a graph coloring method is that two vertices connected by an edge have different colors.

[0130] For example, the marking is represented by color, with different colors representing different memory blocks. Different colors are assigned to any two operands that cannot be stored in the same memory block. When memory conflict relationships are represented by a memory conflict graph, the memory allocation scheme is represented by a coloring graph. In the coloring graph, the node corresponding to the operand is assigned a first color, indicating that the operand is stored in the memory block represented by the first color. The first color can be any color.

[0131] Taking color as the representation of the markers as an example, this application assumes that when generating the colorization map, it starts with one color and generates all possible color combinations for each node (for K colors and n nodes, theoretically there exist K...). n (Several possible coloring graphs). In this process, when the number of colors is small, the generated coloring graph may contain two nodes connected by an edge that are assigned the same color, which does not conform to the logic of the graph coloring method and results in the output coloring graph failing.

[0132] For example, if K is 1, all nodes in the memory conflict graph are of the same color, which does not conform to the coloring logic of the graph coloring method, and the coloring graph output fails. If K is 2, all nodes in the memory conflict graph have a total of 2 colors. If any 3 nodes are connected by an edge, this also does not conform to the coloring logic of the graph coloring method, and the coloring graph output fails. If K is 3, all nodes in the memory conflict graph have a total of 3 colors, and so on. With different values ​​of K, the nodes in the memory conflict graph are traversed and colored. After coloring, each node corresponds to a color, and the color corresponding to each operand represents the physical memory block to which that operand will be allocated. If two operands correspond to different colors, it indicates that these two operands will be allocated to different physical memory blocks. This application focuses on how to partition target memory. By using graph coloring logic, the memory allocation problem is abstracted into a mathematical problem and solved using knowledge from multiple disciplines such as graph theory and data structures. Under given memory allocation conditions, a brute-force enumeration method is used to output memory allocation schemes that meet the conditions, avoiding the problem of missing better schemes or difficulty in determining the optimality of a scheme during analysis. This makes it easier to select the scheme that best meets the developer's needs in subsequent processes.

[0133] In this example where the calculation operations are T1 = A + B, T2 = T1 * C, and T3 = T2 - B in sequence, for Figure 9 The memory conflict map shown can be colored to obtain various colored maps. Figure 10 and Figure 11 Schematic diagrams of two types of coloring are given.

[0134] In the two methods of generating memory conflict graphs mentioned above, in method one, there is an edge between any two operands in the same group. In this case, there may be a situation where two operands are not in the same group, but they are assigned the same color, indicating that they are stored in the same memory block. However, the timing of the two operands overlaps. That is, after one operand is stored in the memory block, the operand assigned the same color will also be stored in the same memory block, causing the previously stored operand to be overwritten.

[0135] If a first memory block is allocated for the first operand, and the second operand also corresponds to the first memory block, and the timing of the first operand's first appearance in the operation sequence is earlier than the timing of the second operand's first appearance in the operation sequence, and the timing of the first operand's last appearance in the operation sequence is later than the timing of the second operand's first appearance in the operation sequence, then a second memory block is allocated for the first operand in memory at a storage level lower than the target memory and adjacent to the target memory. This second memory block is used to store the first operand when the second operand is stored in the first memory block. The first operand can then be referred to as the overflow operand.

[0136] For example, in the example where the calculation operations are T1 = A + B, T2 = T1 * C, and T3 = T2 - B, if a first memory block is allocated for operand A and also for operand T2, then operand A is first stored in the first memory block. When operand T2 also needs to be stored in the first memory block, operand A is first stored in the second memory block, and operand T2 overwrites operand A in the first memory block. When operand A is used again, operand A is read from the second memory block.

[0137] The operands to be overwritten are stored in memory at a lower level of storage, sacrificing some read speed in order to reduce the number of blocks that the target memory is divided into.

[0138] Method 2 focuses on the order in which each operand appears in the entire operation sequence. When the active regions of two operands from different groups overlap, there is an edge between the nodes corresponding to these two operands. Therefore, there will be no overflow operands when marking the memory conflict graph generated based on Method 2.

[0139] As one possible implementation, graph coloring can generate memory allocation schemes without limitation, thus allowing the setting of memory allocation conditions. When a memory allocation scheme first satisfies the conditions, it is identified as the target memory allocation scheme. The memory allocation conditions characterize at least one of the following: whether overflow operands are allowed, the upper limit of overflow operands, or the range of values ​​for K. In the memory allocation scheme, an overflow operand refers to an operand that is overwritten by other operands before its last occurrence in the operation sequence.

[0140] When there are no overflow operands, all operands in the operation sequence are stored in the target memory, resulting in faster read speeds and eliminating the need to read data generated during computation across storage levels.

[0141] For example, when a memory allocation scheme without overflow operands is first encountered, the generation of memory allocation schemes stops, and this memory allocation scheme without overflow operands is determined as the target memory allocation scheme. This setting allows for the rapid finding of a memory allocation scheme without overflow operands. If the nodes are traversed in ascending order of color quantity, the first memory allocation scheme without overflow operands will use the fewest colors, indicating that the target memory needs to be divided into the fewest blocks, which better meets the developer's needs.

[0142] If a user is willing to sacrifice some read speed to reduce the number of blocks the target memory is divided into in order to improve the read throughput of the computing unit, then the maximum number of overflow operations that can occur in the memory allocation scheme can be set.

[0143] The value range of K includes an upper and lower bound. When the number of label types is less than the lower bound of K, it means that a memory allocation scheme satisfying the graph coloring logic cannot be generated based on these labels. When the number of label types is greater than the upper bound of K, it means that the target memory is divided into too many blocks, affecting the throughput of the computing unit. For example, if the user determines that neither one color nor two colors can generate a coloring graph, the user can set the lower bound of K to 3, meaning that a minimum of three colors must be used to generate the coloring graph. As another example, the user can set the upper bound of K to the total number of operands in the operation sequence, meaning that at most one memory block will be allocated for each operand in the operation sequence. When the number of colors used exceeds the number of nodes in the memory conflict graph, the generation of a memory allocation scheme stops.

[0144] By setting memory allocation conditions, a suitable memory allocation scheme can be found as quickly as possible, preventing the generation of memory allocation schemes without limit. This provides developers with guidance on memory allocation when dealing with complex operators.

[0145] In some embodiments, the generated memory allocation scheme can be displayed in the form of tables, graphs, or other formats. Displaying the memory allocation scheme in tabular or graphical form makes it easier for developers to read and evaluate the scheme.

[0146] For example, taking a table format as an example of how memory allocation schemes are displayed, one of the row labels and column labels in the table represents a memory block, and the other represents a computational operation. Figure 12 As shown in the table, row labels represent a computational operation in the operation sequence, and column labels represent all the physical memory blocks used. Elements in the table indexed by a specific row label are symbolic memory blocks. When a symbolic memory block appears in the cell indexed by a specific computational operation and a specific physical memory block, it indicates that the physical memory block stores the operand represented by that symbol during the execution of that computational operation. For example, when indexed based on T1 = AB and the physical memory block represented in red, the cell containing A indicates that operand A is stored in the physical memory block represented in red during the computational operation T1 = AB. Operand B is stored in the physical memory block represented in orange, operand T1 is stored in the physical memory block represented in green, and operand T2 is stored in the physical memory block represented in blue.

[0147] In such Figure 12 In the memory allocation scheme shown, memory blocks that are not used in the calculation operation are not marked in the corresponding cells. For example, in the row corresponding to the calculation operation T1 = AB, only symbolic memory A, B and T1 are used. Then, it is only necessary to mark these three symbolic memory blocks in the corresponding physical memory blocks. The empty cells can be used to characterize the idle parameters of the memory allocation scheme and to evaluate the memory allocation scheme.

[0148] For example, the expression (A+B)×CB can be broken down into three calculation operations: T1=A+B, T2=T1×C, and T3=T2-B. If overflow operands are not allowed in the memory allocation scheme, the demonstrated memory allocation scheme can be as follows: Figure 13 As shown in (a) Figure 13 (a) and Figure 11 (Correspondingly), operand T1 is stored in the red physical memory block, operand A is stored in the orange physical memory block, and operand B is stored in the yellow physical memory block. After the first calculation operation ends, operand A will no longer be used, so operand T2 in the second calculation operation can be stored in the orange physical memory block. Operand C is stored in the green physical memory block. After the second calculation operation ends, operand T1 will no longer be used, so operand T3 in the third calculation operation can be stored in the red physical memory block (operation C will also no longer be used, so operand T3 can also be stored in the green physical memory block). If overflow operands are allowed in the memory allocation scheme, the displayed memory allocation scheme can be as follows: Figure 13 (b)( Figure 13 (a) and Figure 10 As shown in the diagram, operand B is only used in the first and third computation operations. Therefore, in the second computation operation, operand C can be stored in the physical memory block represented by yellow, and operand B is overwritten by operand C (operand B is stored in memory at a lower storage level). At this time, operand B is an overflow operand. When it is used again in the third computation operation, operand B is restored to the physical memory block represented by yellow.

[0149] Saving or restoring overflow operands involves data movement, which is a time-consuming operation. In computationally intensive operators (rather than movement-intensive operators), the time spent on data movement may be masked by the computational pipeline. Therefore, the number of overflow operands in a memory allocation scheme can be used to evaluate the scheme. When generating memory allocation schemes, it is possible to generate schemes with and without overflow operands, providing users with multiple options.

[0150] In some embodiments, determining a target memory allocation scheme based on at least one memory allocation scheme can be specifically implemented as follows: for each memory allocation scheme, determining a scheme score for the memory allocation scheme based on scheme evaluation parameters; determining a target memory allocation scheme based on the scheme score for each memory allocation scheme; wherein, the scheme evaluation parameters include: overflow parameters and / or idle parameters; the overflow parameters are used to characterize the number of overflow operands in the memory allocation scheme, and overflow operands refer to operands that are overwritten by other operands before their last occurrence in the operation sequence; the idle parameters are used to characterize the utilization rate of memory blocks in the target memory.

[0151] When memory allocation schemes are presented in tabular form, for example Figure 12 The memory allocation scheme shown can be characterized by the number of blank cells in the table, that is, the idle parameter can be the number of blank cells in the table of the memory allocation scheme.

[0152] For example, when the scheme evaluation parameters only include the overflow parameter, the memory allocation scheme with the minimum overflow cost is selected as the target memory allocation scheme, and the scheme score is represented by the overflow cost; when the scheme evaluation parameters only include the idle parameter, the memory allocation scheme with the minimum idle cost is selected as the target memory allocation scheme, and the scheme score is represented by the idle cost; when the scheme evaluation parameters include both the overflow parameter and the idle parameter, the memory allocation scheme with the minimum total cost is selected as the memory allocation scheme, and the scheme score is represented by the total cost; wherein, the overflow cost is determined based on the overflow parameter, the idle cost is determined based on the idle parameter, and the total cost is determined based on both the overflow parameter and the idle parameter.

[0153] The overflow cost can be obtained by performing a mathematical operation between the overflow parameter and a first preset value. For example, the overflow cost is the product of the overflow parameter and 5.

[0154] The idle cost can be obtained by performing a mathematical operation on the idle parameter and a second preset value. For example, the idle cost is the product of the idle parameter and 1.

[0155] The total cost can be obtained by performing mathematical operations on the overflow cost and the first preset weight, and the idle cost and the second preset weight. For example, the total cost is the sum of the overflow cost and the idle cost. As another example, if the first preset weight is 60% and the second preset weight is 40%, the total cost is the overflow cost × 60% + the idle cost × 40%.

[0156] For example, if there are x overflow operands in the memory allocation scheme, the overflow cost is 5x; if there are y blank cells, the idle cost is y; and the total cost is 5x + y.

[0157] For example, for Figure 13The memory allocation scheme shown in (b) has an overflow cost of 1×5=5, an idle cost of 0, and a total cost of 5.

[0158] Overflow cost represents the cost of redundant data movement caused by insufficient physical memory blocks, forcing some symbolic memory to be moved to a lower-level memory (e.g., HBM) when physical memory blocks are insufficient. Idle cost represents the underutilization of memory caused by excessive physical memory blocks, resulting in some memory blocks being idle most of the time. By defining overflow and / or idle costs, an evaluation criterion for measuring the quality of memory allocation schemes is provided, enabling quantitative evaluation of memory allocation schemes and further combining them with search algorithms to obtain the optimal memory allocation scheme.

[0159] In some embodiments, when at least one memory allocation scheme is generated, each memory allocation scheme is evaluated to obtain a scheme score for each memory allocation scheme, and the M memory allocation schemes with the highest scheme scores are displayed, allowing the user to select one as the target memory allocation scheme according to their own needs.

[0160] by Figure 13 The two memory allocation schemes shown are examples. Figure 13 The memory allocation scheme shown in (b) has an overflow cost of 6, an idle cost of 0, and a total cost of 6. Figure 13 The memory allocation scheme shown in (a) has an overflow cost of 0, an idle cost of 3, and a total cost of 3. To select the memory allocation scheme with the minimum total cost, then... Figure 13 The two memory allocation schemes shown should be selected. Figure 13 The memory allocation scheme shown in (a) is the target memory allocation scheme.

[0161] In some embodiments, the capacity of each memory block is matched to the amount of data of the operands stored in the memory block. For example, if operand A occupies 16Kb of storage resources and operand B occupies 12Kb of storage resources, then a memory block with a capacity of 16Kb is allocated for operand A and a memory block with a capacity of 12Kb is allocated for operand B.

[0162] In some embodiments, for multiple operands allocated the same memory block, a memory block with a capacity equal to the storage resources occupied by the operand that occupies the most storage resources is allocated to these operands. For example, if operand A and operand T3 are allocated to the same memory block, and operand T3 occupies more storage resources than operand A, then the capacity of the memory block allocated to these two operands is equal to the storage resources occupied by operand T3.

[0163] In some embodiments, a memory allocation algorithm may be deployed in the computing device. By inputting operation sequences into the memory allocation algorithm, at least one memory allocation scheme is output based on metrics set by the developer (e.g., whether overflow operands are allowed, the upper limit of overflow parameters, the range of values ​​for K, etc.). The memory allocation scheme may be obtained by the developer training a preset algorithm using a training sample set. The training sample set includes multiple sample operation sequences. For different metrics, the preset algorithm is trained using the training sample set to obtain the trained memory allocation algorithm.

[0164] In some embodiments, the computing device outputs a target memory allocation scheme so that developers can develop target operators based on the target memory allocation scheme and operation sequence. The target memory allocation scheme can have various output formats, such as tabular, graphical, or textual formats.

[0165] For example, such as Figure 14 As shown, the target memory allocation scheme for the operator corresponding to the expression (A+B)×CB is displayed in a table format on the display interface.

[0166] Figure 7 The technical solution presented offers at least the following benefits: By obtaining the operation sequence of the target operator, which includes multiple computational operations with temporal relationships and each operation contains at least one operand, the memory conflict relationship between operands can be determined based on the operation sequence. This means determining whether the operands involved in the target operator can be stored in the same memory block. Furthermore, based on the memory conflict relationship, a target memory allocation scheme can be determined, i.e., determining the target number of memory blocks to be divided into and the operands to be stored in each memory block. After the target memory allocation scheme is output, developers can develop the target operator based on the target memory allocation scheme and the operation sequence. The target memory allocation scheme is automatically determined by the operator development platform. This fast determination reduces user steps, saves manpower costs, and allows for the rapid acquisition of multiple memory allocation schemes. Multiple memory allocation schemes can also be tested quickly to find the optimal one, thus helping developers find the optimal memory allocation scheme and assisting them in operator development based on it.

[0167] If the memory allocation method provided in this application is by Figure 5If the terminal device and the server work together, then step S101 is executed by the terminal device. The terminal device obtains the operation sequence of the target operator and sends the operation sequence to the server. The server executes steps S102 and S103 to obtain the target memory allocation scheme of the target operator, and then sends the target memory allocation scheme to the terminal device. The terminal device outputs the target memory allocation scheme so that the developer can develop the target operator based on the target memory allocation scheme and the operation sequence.

[0168] In some embodiments, the aforementioned process of determining the target memory allocation scheme can also be combined with the operator programming process. After the user inputs the operation sequence of the target operator, the target memory allocation scheme is automatically determined, and the target operator is programmed based on the target memory allocation scheme to obtain the programmed target operator. When the programmed target operator is run, it will divide memory blocks according to the target memory allocation scheme to store the operands involved in the calculation process. This can greatly reduce the cost of operator development and design for developers and promote operator development.

[0169] The foregoing primarily describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the aforementioned functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0170] like Figure 15 As shown in the illustration, this application also provides a memory allocation apparatus for executing the memory allocation method shown in the above method embodiments. The memory allocation apparatus 400 includes: an acquisition module 401, a processing module 402, and an output module 403.

[0171] The acquisition module 401 is used to acquire the operation sequence of the target operator. The operation sequence includes at least one set of operands with a temporal relationship, and each set of operands includes at least two operands. The processing module 402 is used to determine the memory conflict relationship between operands based on the operation sequence. The memory conflict relationship is used to describe whether different operands can be stored in the same memory block. The processing module 402 is also used to determine the target memory allocation scheme based on the memory conflict relationship. The target memory allocation scheme includes the target number of multiple memory blocks obtained by dividing the target memory, and the operands used to store each memory block.

[0172] In one possible implementation, memory conflict relationships are represented by a memory conflict graph. The memory conflict graph includes multiple nodes, with different nodes representing different operands in the operation sequence. Any two operands in the same group are connected by an edge, and the operands corresponding to the two connected nodes cannot be stored in the same memory block. The processing module 402 is specifically used to: construct a node for each operand in each group of operands in the operation sequence, with the same operands in different groups corresponding to the same node; construct an edge between any two operands in the same group to obtain the memory conflict graph; wherein, the operands corresponding to the two connected nodes cannot be stored in the same memory block.

[0173] In one possible implementation, memory conflict relationships are represented by a memory conflict graph. The memory conflict graph includes multiple nodes, with different nodes representing different operands in the operation sequence. Nodes corresponding to operands in any two active regions that overlap are connected by an edge. Operands corresponding to two nodes connected by an edge cannot be stored in the same memory block. One end value of the active region represents the timing of the first occurrence of the operand in the operation sequence, and the other end value represents the timing of the last occurrence of the operand in the operation sequence. Specifically, the processing module 402 is used to: construct a node for each operand in the operation sequence, with identical operands in different groups corresponding to the same node; for each operand in the operation sequence, determine the active region of the operand; one end value of the active region represents the timing of the first occurrence of the operand in the operation sequence, and the other end value represents the timing of the last occurrence of the operand in the operation sequence; construct an edge between nodes corresponding to operands in any two active regions that overlap, thus obtaining the memory conflict graph; operands corresponding to two nodes connected by an edge cannot be stored in the same memory block.

[0174] In one possible implementation, the processing module 402 is specifically used to: traverse the operands used in the operation sequence, and add a mark to the traversed operands according to the memory conflict relationship to obtain at least one memory allocation scheme; wherein, the mark is selected from any one of K, where K is a positive integer greater than or equal to 1; different marks represent different memory blocks; different marks are added to any two operands that cannot be stored in the same memory block; and determine the target memory allocation scheme according to the at least one memory allocation scheme, where the target number is the number of marks corresponding to the target memory allocation scheme.

[0175] In one possible implementation, the markings are represented by colors, with different colors representing different memory blocks. Operands that cannot be stored in the same memory block are assigned different colors. The output module 403 is used to output the target memory allocation scheme, enabling developers to develop target operators based on the target memory allocation scheme and operation sequence. The output form of the target memory allocation scheme can be any of the following: tabular, graphical, or textual. If the output form of the target memory allocation scheme is a graphical representation, then the target memory allocation scheme is a coloring graph. In the coloring graph, the node corresponding to the operand is assigned a first color, indicating that the operand is stored in the memory block represented by the first color. The first color can be any color.

[0176] In one possible implementation, if a first memory block has been allocated for the first operand, the processing module 402 is further configured to: if the second operand also corresponds to the first memory block, and the timing of the first occurrence of the first operand in the operation sequence is earlier than the timing of the first occurrence of the second operand in the operation sequence, and the timing of the last occurrence of the first operand in the operation sequence is later than the timing of the first occurrence of the second operand in the operation sequence, then allocate a second memory block for the first operand in memory whose storage level is lower than the target memory and whose storage level is adjacent to the target memory. The second memory block is used to store the first operand when the second operand is stored in the first memory block.

[0177] In one possible implementation, the processing module 402 is specifically used to: when a memory allocation scheme meets the memory allocation conditions for the first time, determine the memory allocation scheme that meets the memory allocation conditions as the target memory allocation scheme; the memory allocation conditions are used to characterize at least one of the following: whether overflow operands are allowed, the upper limit of overflow operands, or the range of values ​​for K; wherein, in the memory allocation scheme, an overflow operand refers to an operand that is overwritten by other operands before its last occurrence in the operation sequence.

[0178] In one possible implementation, the processing module 402 is specifically used to: for each memory allocation scheme, determine the scheme score of the memory allocation scheme based on the scheme evaluation parameters; and determine the target memory allocation scheme based on the scheme score of each memory allocation scheme. The scheme evaluation parameters include: overflow parameters and / or idle parameters. The overflow parameters are used to characterize the number of overflow operands in the memory allocation scheme. Overflow operands refer to operands that are overwritten by other operands before their last occurrence in the operation sequence. The idle parameters are used to characterize the utilization rate of memory blocks in the target memory.

[0179] In one possible implementation, the processing module 402 is specifically used to: select the memory allocation scheme with the lowest overflow cost as the target memory allocation scheme when the scheme evaluation parameters only include the overflow parameter; select the memory allocation scheme with the lowest idle cost as the target memory allocation scheme when the scheme evaluation parameters only include the idle parameter; and select the memory allocation scheme with the lowest total cost as the memory allocation scheme when the scheme evaluation parameters include both the overflow parameter and the idle parameter; wherein the overflow cost is determined based on the overflow parameter, the idle cost is determined based on the idle parameter, and the total cost is determined based on both the overflow parameter and the idle parameter.

[0180] In one possible implementation, the acquisition module 401 is specifically used to: acquire multiple sets of operands input by the user and the temporal relationship between the multiple sets of operands; or, acquire the expression in the target operator input by the user, parse the expression to obtain multiple sets of operands, and combine the multiple sets of operands in sequence according to the temporal relationship to obtain the operation sequence.

[0181] In one possible implementation, the operation sequence includes multiple operation tuples, each representing a set of operands. The order of the multiple operation tuples matches the temporal order of the multiple sets of operands. Each operation tuple includes multiple operands with an order relationship. The Nth operand in the operation tuple represents the destination operand in the set of operands corresponding to the operation tuple, and the other operands in the operation tuple, excluding the Nth operand, represent the source operands in the set of operands corresponding to the operation tuple.

[0182] In one possible implementation, the capacity of each memory block is matched to the amount of data of the operands stored in the memory block.

[0183] It should be noted that, Figure 15 The module division shown is illustrative and represents only one logical functional division; in actual implementation, other division methods are possible. For example, two or more functions can be integrated into a single processing module. These integrated modules can be implemented either in hardware or as software functional modules.

[0184] Another embodiment of this application provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the memory allocation method shown in the above embodiments.

[0185] In actual implementation, the acquisition module 401, processing module 402, and output module 403 can be implemented by the processor of the computer device calling computer program code in memory. The specific execution process can be found in the memory allocation method section above, and will not be repeated here.

[0186] Another embodiment of this application provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the memory allocation method shown in the above embodiments.

[0187] In another embodiment of this application, a computer program product is also provided, which includes computer instructions that, when executed by a processor, implement the steps of the memory allocation method shown in the above embodiments.

[0188] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), etc.

[0189] The above description is merely a specific embodiment of this application. Any variations or substitutions conceived by those skilled in the art based on the specific embodiments provided in this application should be covered within the protection scope of this application.

Claims

1. A memory allocation method, characterized in that, Applied to a computing device; the method includes: Obtain the operation sequence of the target operator, wherein the operation sequence includes at least one set of operands with temporal relationship, and each set of operands includes at least two operands; Based on the operation sequence, the memory conflict relationship between the operands is determined, and the memory conflict relationship is used to describe whether different operands can be stored in the same memory block; Based on the memory conflict relationship, a target memory allocation scheme is determined; the target memory allocation scheme includes the target number of multiple memory blocks obtained by dividing the target memory, and the number of operations to be stored in each memory block; If a first memory block is allocated for a first operand, and a second operand also corresponds to the first memory block, and the timing of the first operand's first appearance in the operation sequence is earlier than the timing of the second operand's first appearance in the operation sequence, and the timing of the first operand's last appearance in the operation sequence is later than the timing of the second operand's first appearance in the operation sequence, then a second memory block is allocated for the first operand in a memory location with a storage level lower than the target memory and a storage level adjacent to the target memory. The second memory block is used to store the first operand when the second operand is stored in the first memory block.

2. The method according to claim 1, characterized in that, The memory conflict relationship is represented by a memory conflict graph; the memory conflict graph includes multiple nodes and at least one edge, different nodes are used to represent different operands in the operation sequence, any two operands in the same group are connected by an edge, and the operands of two nodes connected by the edge cannot be stored in the same memory block.

3. The method according to claim 1, characterized in that, The memory conflict relationships are represented by a memory conflict graph; the memory conflict graph includes multiple nodes and at least one edge, different nodes are used to represent different operands in the operation sequence, nodes corresponding to operands that overlap in any two active intervals are connected by an edge, and operands corresponding to two nodes connected by the edge indicate that they cannot be stored in the same memory block; wherein, one end value of the active interval is used to represent the timing of the first group of operands in the operation sequence, and the other end value is used to represent the timing of the last group of operands in the operation sequence.

4. The method according to claim 1, characterized in that, The step of determining the target memory allocation scheme based on the memory conflict relationship includes: The operands used in the operation sequence are traversed, and a mark is added to the traversed operands according to the memory conflict relationship to obtain at least one memory allocation scheme; wherein, the mark is selected from any one of K, where K is a positive integer greater than or equal to 1; different marks represent different memory blocks; different marks are added to any two operands that cannot be stored in the same memory block; Based on the at least one memory allocation scheme, a target memory allocation scheme is determined, wherein the target quantity is the number of tags corresponding to the target memory allocation scheme.

5. The method according to claim 4, characterized in that, The method further includes: Output the target memory allocation scheme, wherein the target memory allocation scheme is a coloring map; in the coloring map, memory blocks with different markings are represented by different colors, and the node corresponding to the operand is assigned a first color, indicating that the operand is stored in the memory block represented by the first color, where the first color is any color.

6. The method according to claim 4 or 5, characterized in that, Determining the target memory allocation scheme based on the at least one memory allocation scheme includes: When the memory allocation scheme meets the memory allocation conditions for the first time, the memory allocation scheme that meets the memory allocation conditions is determined as the target memory allocation scheme; The memory allocation condition is used to characterize at least one of the following: whether overflow operands are allowed, the upper limit of overflow operands, or the range of values ​​for K; wherein, in the memory allocation scheme, the overflow operand refers to an operand that is overwritten by other operands before its last occurrence in the operation sequence.

7. The method according to claim 4 or 5, characterized in that, Determining the target memory allocation scheme based on the at least one memory allocation scheme includes: For each memory allocation scheme, a scheme score is determined based on the scheme evaluation parameters; The target memory allocation scheme is determined based on the scheme score of each memory allocation scheme; The scheme evaluation parameters include: overflow parameter and / or idle parameter; the overflow parameter is used to characterize the number of overflow operands in the memory allocation scheme, and the overflow operand refers to the operand that is covered by other operands before its last occurrence in the operation sequence; The idle parameter is used to characterize the utilization rate of memory blocks in the target memory.

8. The method according to claim 1, characterized in that, The operation sequence for obtaining the target operator includes: Obtain multiple sets of operands input by the user and the timing relationship between the multiple sets of operands; or, The expression in the target operator input by the user is obtained, the expression is parsed to obtain the multiple sets of operands, and the multiple sets of operands are combined sequentially according to the temporal relationship to obtain the operation sequence.

9. The method according to any one of claims 1-5, characterized in that, The operation sequence includes multiple operands, each operand representing a set of operands. The order of the operands matches the temporal order of the operands. Each operand includes multiple operands with an order relationship. The Nth operand in the operand represents the destination operand in the set of operands corresponding to the operand. The other operands in the operand, excluding the Nth operand, represent the source operands in the set of operands corresponding to the operand.

10. The method according to any one of claims 1-5, characterized in that, The capacity of each memory block is matched to the amount of operand data stored in the memory block.

11. A computing device, comprising a memory and a processor, characterized in that, The memory is used to store computer instructions, and the processor is used to run the computer instructions to cause the computing device to perform the method according to any one of claims 1-10.

Images