Method for transforming abstract representation of trained neural network into program code of target language
By decoupling memory usage planning and neural network code generation, the optimal memory planning proposal is selected, which solves the problem of high memory consumption on embedded devices, and achieves efficient neural network installation and computing efficiency improvement.
Patent Information
- Application Number
- CN202380059132.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-08-12
- Filing Date
- 2023-07-13
- Publication Date
- 2025-03-25
AI Technical Summary
How to efficiently plan memory usage to reduce memory consumption and improve computing efficiency when installing trained neural networks on embedded devices, especially in environments with limited RAM.
By reading into the abstract representation of the neural network, the intermediate representation is calculated, and multiple memory planning proposals are determined, and the optimal planning proposal is selected based on the quality metric, thereby generating program code in the target language, decoupling memory usage planning and code generation.
It realizes efficient installation of trained neural networks on target devices, reduces memory consumption, improves computing efficiency, and is suitable for embedded devices with limited RAM.
Smart Images
Figure CN119698615A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for transforming an abstract representation of a trained neural network into a program code in a target language, which program code can be transformed into an executable program code using a compiler for the target language. The invention also relates to a computer program for implementing the above method, a machine-readable data carrier and / or a download product having such a computer program, and one or more computers and / or computing instances comprising the above computer program. Background Art
[0002] A neural network (NN) is a graph consisting of individual layers connected to each other via input-output relationships. The simplest type is a sequential network, in which operators are arranged in a chain and the current operator receives data from its predecessor and transfers further processed data to its successor. The corresponding data is stored in the so-called intermediate buffer. However, many NNs have a much more complex structure with branched paths that converge again later. In addition to the intermediate buffer for intermediate storage of data exchanged between operators, there can also be buffers within the operator for storing intermediate results there. These buffers are called internal buffers. The size of the intermediate buffer is predetermined by the network architecture. The internal buffer depends on the specific implementation of each layer. It is thus possible to save computing time at the expense of additional memory consumption when using the internal buffer.
[0003] When implementing NN on computer hardware, memory must be provided for all buffers. In the training framework, this task can be solved dynamically. If there is a need, new memory is allocated and the new buffer is saved there. When installing on an application device, especially in the embedded area, if possible, pre-allocated memory is used. This is possible because, with a few exceptions - such as NMS layers or LSTMs that handle variable sequence lengths - the graph structure and the sizes of all buffers are fully specified in the case of NN. Here, the address planning of the buffers can also be called memory planning.
[0004] If the NN is implemented in C or a comparable language, a pre-calculated amount of memory is allocated and each buffer is assigned an address within that memory. Generally speaking, this approach results in lower memory consumption than with other approaches.
[0005] When training on a GPU with many GB of RAM (Random Access Memory), minimizing the memory requirements is a task that cannot be ignored. This becomes important especially when the trained NN should be installed on an embedded device with its very limited RAM.
[0006] The general memory planning problem is NP-hard. That is, it is impossible to find an efficient algorithm for this in the sense of "polynomially growing runtime". Instead, approximations or exact solutions for smaller problems or solutions that make full use of the typical properties of NNs must be used. Summary of the invention
[0007] Within the scope of the present invention, a method for converting an abstract representation of a trained neural network into a program code in a target language has been developed. Here, the above program code can be converted into an executable program code using a compiler for the target language. Here, the method comprises at least the steps described below. In a first method step, an abstract representation of a trained neural network is read in, wherein the abstract representation comprises at least an architecture and parameters obtained from training, which characterize the behavior of the neural network. In a subsequent method step, an intermediate representation of the neural network is calculated from the abstract representation. Here, the intermediate representation describes a calculation graph for the output of the neural network. Then, in another method step, a plurality of planning proposals for planning memory usage during the execution of the calculation graph are determined. In addition, a quality metric is determined for each planning proposal based on at least one predetermined criterion. In a subsequent method step, a planning proposal is then selected from a plurality of determined planning proposals based on the determined quality metric. Then, the sought program code is generated in the target language from the intermediate representation and the selected planning proposal.
[0008] The advantage of the method proposed above and below is that the planning of memory use is decoupled from the generation of code. That is, in the process of generating program code in the target language, an intermediate representation is first generated, and then the planning of efficient and optimal memory use is performed by means of the intermediate representation, so as to resume code generation later-using the selected plan for memory use. Different algorithms can be used for planning memory use here, and the plans developed when using these algorithms can be compared with each other later. This makes it possible to select a memory plan that is "customized" optimally for the device on which the program code is to be installed. When RAM is a valuable and limited resource, this brings significant advantages together. This is especially true in the case of a tiny microcontroller, on which a (suitable) neural network should be installed and subsequently run by means of the method proposed here. Here, a corresponding microcontroller with greatly limited storage space can be used in vehicles, household appliances-especially in washing machines or dishwashers-and can also be used in power tools.
[0009] Generating program code can be time-intensive; for example, compiling a neural network in C code can take 20 minutes or significantly more, while planning the memory usage can ideally be carried out in a few seconds. At the same time, particularly advantageous memory usage is important for - subsequently - using the neural network as optimally as possible on a terminal device / terminal devices with limited memory capacity. By "checking" multiple memory plans, the most optimal (full) utilization of the storage space on the terminal device / terminal devices can be ensured. At the same time, by dividing the method into program code generation and creating multiple plans for memory usage and selecting the memory plan that is most suitable for the respective situation, resource time is "saved" or gained within the scope of the installation and even during the later reasoning of the corresponding neural network.
[0010] The method presented here is intended to install a trained network on a target device for production use. In this application, a compilation that is as high quality as possible is of great value. The slightly longer computing time that must be spent during the installation process for calculating and comparing different plans for memory usage—for example, in the range of several minutes up to approximately one hour in extreme cases—can be easily tolerated for the high quality of the compilation that is to be achieved with this additional effort.
[0011] According to one exemplary embodiment, the planning proposal is determined using a plurality of different predefined algorithms for memory planning. Additionally or alternatively, the planning proposal can be determined using a parameterization scheme with a plurality of different values for the free parameters.
[0012] In this way, the proposed method is flexible, in particular with regard to the type of algorithms used within the scope of determining a suitable memory planning. Instead of using one or more complete algorithms, a large number of algorithms can be used within the method proposed here. Among the large number of algorithms, there can be a simple but possibly very fast algorithm, which in some cases directly finds the optimal solution, but in other cases - in some cases due to the complexity of the corresponding memory planning problem on a given terminal device - fails. In addition, among the large number of algorithms, there may be algorithms whose solutions exist in respectively different approximation methods - up to very complex but more time-intensive approximation methods. The method described above and below makes it possible to execute a variety of solutions for memory planning and to compare the different results determined (in particular also in parallel).
[0013] In the methods described above and below, it is not important to use an algorithm that finds the optimal solution in some cases but fails in many other cases. If the algorithm should not be able to determine the solution - for example not within a predetermined time or not even substantially because the heuristic on which the algorithm is based does not work for the given problem in the specific case - the memory plan of another algorithm can be used.
[0014] On the other hand, it is also conceivable that a planning proposal already determined within the method already has a quality metric which at least exceeds a predeterminable quality limit - or does not exceed this predeterminable quality limit depending on the definition of the metric - and can therefore already be considered sufficiently "good". In this case, it can be provided that the determination of further planning proposals for memory planning is interrupted or aborted and the method continues to generate the sought program code in the target language from the intermediate representation and the already determined planning proposal, the quality metric of which exceeds (or does not exceed) the quality limit. In this way, the overall installation time can be reduced while at the same time ensuring a high quality of the compilation to be generated.
[0015] According to another embodiment, at least one planning proposal includes dynamically allocating storage space for flexible components of intermediate results that occur while executing the computation graph and later releasing the storage space.
[0016] In this regard, different algorithms that provide corresponding planning proposals can pursue different goals. The main goal here is always to use as little total memory capacity as possible for all buffers. However, for example, an algorithm can deliberately ignore smaller buffers in the planning in order to be able to solve the remaining planning problem better. In addition, additional algorithms can pursue different approximate solutions, etc.
[0017] In general, planning proposals for the subsequent memory usage each assign corresponding addresses within a predefined allocated memory (of the terminal at which the inference is to take place) to buffers provided in the structure of the neural network.
[0018] According to a further embodiment, the quality metric measures, in addition to the maximum total requirement for memory space, the computing speed advantage due to an unplanned internal buffer (Intrabuffer), which is therefore held in registers of the CPU.
[0019] According to one embodiment, the computation graph of the neural network includes transferring intermediate results from a layer or operator of the neural network to a later layer or later operator of the neural network via an intermediate buffer. Additionally or alternatively, the computation graph of the neural network may include intermediate storage of intermediate results within a layer of the neural network in an internal buffer for later use in the same layer.
[0020] In a neural network, intermediate results are transferred from one layer to a later subsequent layer. These intermediate results require an intermediate buffer, which must ensure that the corresponding data is stored over a period of time until the calculation in the corresponding later layer is completed. If the corresponding calculation in the later layer is completed, the intermediate result no longer needs to be "canceled", and the storage space previously assigned to the intermediate result can be released and rewritten. In addition, in some cases, the results in a layer must be stored intermediately for further calculations in the layer. For this purpose, there is an internal buffer, which stores the corresponding data intermediately until the data is no longer needed and the corresponding storage space can be released again even in this case. In particular, the size of the memory required for the internal buffer and, to a lesser extent, the intermediate buffer is related to the architecture of the terminal device on which the neural network should be executed during the reasoning process. Therefore, the method proposed here allows flexible action for installing a neural network that can be executed as optimally as possible afterwards.
[0021] The memory plan should specify the addresses of the buffers so that the buffers are not overwritten as long as the corresponding data is still needed. For internal buffers, this means that calculations are allowed during the associated operator without overwriting the internal buffer. Intermediate buffers must be protected from overwriting from the moment the calculation of an operator that produces an intermediate buffer as output begins, i.e. until the last operator that uses the intermediate buffer as input has completed its calculation.
[0022] According to an exemplary embodiment, one or more algorithms in the algorithm for performing memory planning in one of the above-mentioned embodiments include at least the following steps: In one step, a global memory area with a predetermined size is provided. Here, the size corresponds to at least MaxLB, wherein MaxLB corresponds to the maximum combined memory requirement of all intermediate results, and the simultaneous maintenance of the intermediate results is still necessary at the time point of executing the operator. In a subsequent step, the minimum free memory area within the global memory area is selected respectively, and the minimum free memory area can accommodate each intermediate result and is available within the necessary maintenance time of the intermediate result. In addition, the minimum free memory area is assigned to the intermediate result within the necessary maintenance time. This type of storage space assignment can also be called "trivial memory planning".
[0023] The above-described approach to trivial memory planning may hereby be able to calculate optimal solutions for small NNs and NNs with favorable structures.
[0024] According to a further exemplary embodiment, one of the above-mentioned algorithms from a plurality of predefined algorithms comprises optimizing N starting addresses for intermediate results by means of integer linear programming. i , i=1, ..., N. Here, the i-th intermediate result has a memory requirement of s i , and according to the memory requirements s i Occupies the starting address o in memory i and the final address e i In addition, the pairs (u, v) of the intermediate results u and v are stored in the conflict set C, the simultaneous maintenance of which is necessary. Here, the memory areas occupied by these intermediate results u and v cannot overlap. i , i = 1, ..., N is then intended to minimize the occurrence of the highest final address e i .
[0025] As an alternative to the above-described exemplary embodiment, in particular a global memory area with a predetermined size can be provided, which corresponds at least to MaxLB. In this case, MaxLB corresponds to the combined memory requirement of all intermediate results which are still necessary at the time of executing the operator. Furthermore, the start address can then be optimized according to the following equation: i : -e i =o i +s i , -0≤o i ≤MaxLB-s i , -o u ≥e v or v ≥e u ,, if (u, v)∈C.
[0026] Every solution to this system of equations is an optimal memory efficient plan.
[0027] According to another embodiment, if the above optimization does not lead to a solution, those above embodiments in which MaxLB is considered as a predetermined size of the memory area are changed: in this case, the size of the considered global memory area can be increased to a value between MaxLB and 2*MaxLB.
[0028] According to another embodiment, one of the algorithms from a plurality of predetermined algorithms for performing memory planning includes at least the steps listed below. In a first step, the intermediate result that is estimated to require the largest storage space is determined. Additionally or alternatively, those layers of the neural network that are estimated to require the largest storage space for the intermediate results during their calculation can be determined. In a subsequent step, an initial memory plan is first created for the aforementioned intermediate results - without considering the remaining intermediate results. That is, the aforementioned intermediate results and / or layers of the neural network determined in the first step that are estimated to require the largest storage space for the intermediate results during their calculation are placed in the global memory before assigning storage space to the remaining intermediate results. As a result, the "largest" intermediate results or the intermediate results required in the layers with the largest memory requirements are placed in the memory first or a memory area is assigned to the intermediate results first. Only then are the remaining intermediate results placed in another step using another algorithm for performing memory planning.
[0029] The typical memory structure of existing NN architectures can thus be considered in a way that is particularly beneficial for solving memory planning problems.
[0030] For example, those algorithms described further above that perform the steps for "trivial memory planning" may be selected for creating the aforementioned initial storage plan and / or for placing remaining intermediate results.
[0031] According to another exemplary embodiment, the method proposed here further comprises the following steps. In one method step, intermediate results are selected that should not be included in the memory planning but should be dynamically stored on the stack. In subsequent method steps, a plurality of predefined algorithms for memory planning are then executed only under consideration of the intermediate results that are not to be stored on the stack.
[0032] The problem of memory planning can thus be configured flexibly. Thus, intermediate results can optionally be included in the memory planning or alternatively be allocated on the stack. The memory requirement can thus be increased in some cases, but the calculation time can be improved if necessary, because the CPU array can be kept locally in registers.
[0033] According to a further embodiment, for debugging purposes, the planning proposal is determined under the additional boundary condition that a memory area once assigned for one intermediate result is not reused for another intermediate result.
[0034] In this case, after the NN is traversed, all intermediate results of the individual layers and other auxiliary variables are available for analysis. That is to say, in the above-mentioned embodiment, all intermediate results of the individual layers and the values of the internal buffer are obtained and can be analyzed after the execution of the NN. After the analysis is completed, it is then possible to "switch" to a memory planning that saves memory. However, it is especially important to ensure that the results of the NN remain the same in a precisely accurate manner.
[0035] According to another embodiment, the proposed method further comprises the following steps. In one method step, the program code in the target language is converted into an executable program code. In a subsequent method step, the executable program code is executed on at least one computer and / or on at least one computing instance, i.e., in such a way that at least one input of the neural network is converted into at least one output of the neural network.
[0036] Furthermore, the invention relates to a computer program with machine-readable instructions which, when executed on one or more computers and / or computing instances, cause the one or more computers and / or computing instances to perform the method according to the invention as previously described and described below. Likewise, the invention comprises a machine-readable data carrier and / or a download product on which the above-mentioned computer program is stored, as well as a computer and / or a computing instance equipped with the aforementioned computer program and / or the aforementioned machine-readable data carrier.
[0037] Further measures to improve the invention are described in more detail below with reference to the figures together with the description of preferred exemplary embodiments of the invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 An embodiment of the method proposed herein is shown; Figure 2A -C shows the Figure 1 An embodiment of the sub-steps of the method. DETAILED DESCRIPTION
[0039] Figure 1 An exemplary embodiment of a method 1000 for converting an abstract representation 1 of a previously trained neural network into a program code 6 in a target language is shown. The latter program code 6 can be converted into an executable program code 7 using a compiler for the target language. Within the scope of method 1000, an abstract representation 1 of an already trained neural network is read in in step 100. The abstract representation 1 includes at least an architecture 11 and parameters 12 obtained from the training of the neural network, ρ i, which characterizes the behavior of the neural network. In step 200, an intermediate representation 2 of the neural network is then calculated from the abstract representation 1. The intermediate representation 1 describes at least one calculation graph 21 for the output of the neural network. Then, in step 300, a plurality of planning proposals 31, 32, 33 are determined for planning the memory usage during the execution of the calculation graph 21. According to at least one predetermined criterion, a quality measure Q1, Q2, Q3 is determined for each planning proposal 31, 32, 33 in step 400. For example, the quality measures Q1; Q2; Q3 can determine the maximum total memory requirement during the execution of the calculation graph 21. Then, in step 500, a planning proposal 31; 32; 33 is selected based on the determined quality measures Q1, Q2 and Q3. In method step 600, the sought program code 6 is generated in the target language from the intermediate representation 21 and the selected planning proposal Q2.
[0040] The planning proposals 31, 32, 33 determined in step 300 can be determined using a plurality of different predefined algorithms A1, A2, A3 for memory planning and / or using parameterization schemes with a plurality of different values for free parameters. At least one of the planning proposals 31, 32, 33 can include a storage space S for intermediate results that occur when executing the calculation graph 21. i Dynamic allocation and later release of that storage space.
[0041] Here, in Figure 1 The computation graph 21 of the neural network outlined in 20 comprises the transfer of intermediate results from a layer O1 or an operator O1 of the neural network to a subsequent layer O2 of the neural network or a subsequent operator O2 via an intermediate buffer 210. Alternatively or additionally, the computation graph 21 comprises the intermediate storage of intermediate results within the layer O2 of the neural network in an internal buffer 211 for later use in the same layer O2.
[0042] exist Figure 1 The following is also outlined in the following: In a substep 301 of method step 300, intermediate results are first determined which are not to be included in the memory allocation but are to be stored dynamically on the stack. In a substep 302, a plurality of predefined algorithms A1, A2, A3 for memory allocation are then executed taking into account only the intermediate results which are not to be stored on the stack.
[0043] according to Figure 1 In step 700, the program code in the target language is further converted into an executable program code 7. Subsequently, the executable program code 7 can be executed on at least one computer and / or on at least one computing instance in step 800, so that at least one input of the neural network is converted into at least one output of the neural network.
[0044] Figure 2A The method 1000 described above is related to steps that can be performed in conjunction with at least one of the algorithms A1, A2, A3 in the method 1000 described above. Therefore, one of the algorithms A1 from the plurality of predefined algorithms A1, A2, A3 for performing memory planning can include at least Figure 2A . In step A100, a global memory area 8 with a predetermined size 80 is provided, which corresponds to at least MaxLB. As has been further explained above, MaxLB corresponds here to the combined memory requirement of all intermediate results, the simultaneous maintenance of which is still necessary to the maximum extent at the point in time when the operator is executed. In step A200, the smallest free memory area 81 within the global memory area 80 is then selected respectively, which smallest free memory area 81 can accommodate each intermediate result and is available within the necessary maintenance time of the intermediate result. In step A300, the smallest free memory area 81 is assigned to the intermediate result 82 within the necessary maintenance time. If the trivial memory planning algorithm should fail, other heuristics are selected.
[0045] Figure 2B These are steps in algorithm A2 from among a plurality of predefined algorithms in method 1000. The algorithm in question includes optimizing N starting addresses for intermediate results by means of integer linear programming. i , i=1, ..., N. Here, the i-th intermediate result has a memory requirement of s i In the memory, the i-th intermediate result is stored according to the memory requirement s i Occupies the starting address o i With the final address e i In addition, the pairs (u, v) of intermediate results u and v are stored in the conflict set C, the simultaneous maintenance of which is necessary. Obviously, the memory areas occupied by these intermediate results u and v are not allowed to overlap. The optimization then aims to minimize the occurrence of the highest final address e i .exist Figure 1 In step 100 of the method 1000 shown in FIG. 1 , within determining a planning proposal, according to Figure 2B , a global memory area 8 with a predetermined size 80 is now provided as step B100 in the algorithm, which corresponds to at least MaxLB. MaxLB is in turn given by the combined memory requirements of all intermediate results, which are still necessary to maintain at the same time at the time of executing the operator. In step B200, the start address o is calculated by the relevant algorithm. i , so that the following equation is satisfied: -e=o i +s i, -0≤o i ≤MaxLB-s i , -o u ≥e v , or v ≥e u ,, if (u, v)∈C.
[0046] If the algorithm does not find a solution, the size of the global memory region may be continually increased to a value between MaxLB and 2*MaxLB and the optimization repeated with that value.
[0047] exist Figure 2C , steps within algorithm A3 from among a plurality of predefined algorithms A1, A2, A3 for memory planning are shown, which steps can be performed in method step 300 of method 1000 within a certain scope of memory planning. In this case, in step C100, algorithm A3 first determines those intermediate results which must be kept in the memory at the same time and which require the largest memory space in total. Alternatively or additionally, in this step C100, those layers O1, O2 of the neural network which require the largest memory space for the intermediate results during their calculation are determined. In this case, these are obviously the "largest" intermediate results or intermediate results in the layers with the largest memory requirements. Subsequently, in step C200, an initial memory plan is created for these intermediate results - without taking into account the remaining intermediate results. In this case, for example, algorithm A1 or A2 can be used, the steps of which for memory planning are in Figure 2A In step C300, another algorithm for memory planning is then used to place the remaining intermediate results that have not been included in the memory planning in step C200. To place the remaining intermediate results, the algorithm A1 or A2 can be used again, the step for memory planning of which is in step C300. Figure 2A or 2B (at least partially).
[0048] For debugging purposes, the planning proposal can also be determined under the additional boundary condition that a memory area once assigned to one intermediate result cannot be used again for another intermediate result.
Claims
1. A method (1000) for transforming an abstract representation (1) of a trained neural network into a program code (6) in a target language, the program code (6) being convertible into an executable program code (7) using a compiler for the target language, the method (1000) comprising the following steps: - reading in an abstract representation (1) of an already trained neural network, wherein said abstract representation (1) comprises at least an architecture (11) and parameters (12) obtained from training and characterizing the behavior of said neural network, (100), - computing an intermediate representation (2) of the neural network from the abstract representation (1), wherein the intermediate representation (1) describes a computation graph (21) for an output of the neural network, - determining a plurality of planning proposals (31, 32, 33) for planning the memory usage during the execution of the calculation graph (21), (300), wherein the planning proposals respectively assign corresponding addresses within a predefined allocated memory of the terminal to a buffer provided in the structure of the neural network, - determining a quality metric (Q1, Q2, Q3) for each planning proposal (31, 32, 33) according to at least one predetermined criterion, wherein the quality metric measures the maximum total memory requirement during the execution of the calculation graph, wherein the lowest possible total memory capacity should be used for all buffers, (400), - selecting a planning proposal (31, 32, 33) based on the determined quality measures (Q1, Q2, Q3), (500), and - Generating the sought program code (6), (600) in the target language from the intermediate representation (21) and the selected planning proposal (Q2).
2. A method (1000) according to claim 1, wherein the planning proposal (31, 32, 33) is determined using a plurality of different predefined algorithms (A1, A2, A3) for memory planning and / or using a parameterization scheme with a plurality of different values for free parameters.
3. A method (1000) according to any one of claims 1 to 2, wherein at least one planning proposal (31; 32; 33) includes dynamically allocating storage space (Si) for intermediate results that occur when executing the computation graph (21) and later releasing the storage space.
4. A method (1000) according to any of the preceding claims, wherein the quality metric (Q1; Q2; Q3) measures the maximum total requirement for storage space and, if necessary, the program running time based on the number of buffers dynamically placed during the execution of the computation graph (21).
5. The method (1000) according to any one of the preceding claims, wherein the computational graph (21) of the neural network comprises - transferring an intermediate result from a layer (O1) or operator (O1) of the neural network to a subsequent layer (O2) or subsequent operator (O2) of the neural network via an intermediate buffer (210), and / or - intermediate storage of intermediate results within a layer (O2) of the neural network in an internal buffer (211) for later use in the same layer (O2).
6. The method (1000) according to claim 2 and optionally in addition to any one or more of claims 3 to 5, wherein one of the algorithms (A1) from a plurality of predefined algorithms (A1, A2, A3) for performing memory planning comprises the following steps: - providing a global memory area (8) with a predetermined size (80), the predetermined size corresponding to at least MaxLB, wherein MaxLB corresponds to the combined memory requirement of all intermediate results, the simultaneous maintenance of which is still necessary at the point in time when the operator is executed, (A100), - respectively selecting a minimum free memory area (81) within the global memory area (8) which is able to accommodate each intermediate result and is available within the necessary maintenance time of the intermediate result, (A200), and - Assigning the minimum free memory area (81) to the intermediate result (82) within the necessary maintenance time, (A300).
7. The method (1000) according to claim 2 and optionally in addition to any one or more of claims 3 to 6, wherein one of the algorithms (A1) from a plurality of predefined algorithms (A1, A2, A3) comprises optimizing N starting addresses for intermediate results by means of integer linear programming. i , i = 1, ..., N, where - The i-th intermediate result has a memory requirement of s i , and according to the memory requirements s i Occupies the starting address o in memory i and the final address e i The memory area between - pairs (u, v) of intermediate results u and v, the simultaneous maintenance of which is necessary, are stored in the conflict set C, - the memory areas occupied by these intermediate results u and v must not overlap, and - The optimization aims to minimize the occurrence of the highest final address e i .
8. The method (1000) according to claim 7, comprising the steps of: - providing a global memory area (8) having a predetermined size (80), which corresponds to at least MaxLB, wherein MaxLB corresponds to the combined memory requirement of all intermediate results, the simultaneous maintenance of which is still necessary at the maximum at the point in time of executing the operator, (B100), and - Optimize (B200) the starting address according to the following equation: i : -and i =o i +s i , -0≤o i ≤MaxLB-s i , -o u ≥e v or v ≥e u ,, if (u, v)∈C.
9. The method (1000) of claim 6 or 8, wherein in response to the optimization not leading to a solution, the size of the global memory area is increased to a value between MaxLB and 2*MaxLB.
10. The method (1000) according to claim 2 and optionally in addition to any one or more of claims 3 to 9, wherein one of the algorithms (A1) from a plurality of predefined algorithms (A1, A2, A3) for performing memory planning comprises at least the following steps: - determining the intermediate result which is estimated to require the largest storage space and / or those layers (O1, O2) of the neural network which, during their calculation, are estimated to require the largest storage space for the intermediate result, (C100), - create an initial memory plan for these intermediate results, without considering the remaining intermediate results, (C200), and - Placing the remaining intermediate results using another algorithm for memory planning, (C300).
11. The method (1000) according to claim 10, wherein the algorithm for memory planning according to claim 6 is selected for creating an initial memory plan and / or for placing remaining intermediate results.
12. The method (1000) according to claim 2 and optionally in addition to any one or more of claims 3 to 11, further comprising the following steps: -selecting intermediate results that should not be included in the memory plan but should be dynamically saved on the stack, (301), and - executing a plurality of predefined algorithms for memory planning taking into account only intermediate results which are not to be stored on the stack (302).
13. The method (1000) according to any of the preceding claims, wherein for debugging purposes the planning proposal is determined under the additional boundary condition that a memory area once assigned for one intermediate result is not reused for another intermediate result.
14. The method (1000) according to any one of the preceding claims, further comprising: -Converting the program code in the target language into an executable program code (7), (700); and - executing the executable program code (7) on at least one computer and / or at least one computing instance so that at least one input of the neural network is transformed into at least one output of the neural network, (800).
15. A computer program comprising machine-readable instructions, which, when executed on one or more computers and / or computing instances, cause the one or more computers and / or computing instances to perform the method (1000) according to any one of claims 1 to 14.
16. A machine-readable data carrier and / or a download product having a computer program according to claim 15.
17. One or more computers and / or computing instances equipped with a computer program according to claim 15 and / or a machine-readable data carrier according to claim 16.