Memory allocation method and device, equipment and storage medium

Abstract

The invention relates to the technical field of image processing, and discloses a memory allocation method, device and equipment and a storage medium, the method comprises the following steps: in response to a data processing request, performing model reasoning on to-be-processed data through a neural network model, when a current operator is a static output operator, obtaining a static sub-network to which the current operator belongs, calculating an optimal memory size corresponding to the static sub-network, and allocating a memory block for the current operator according to the optimal memory size; when the current operator is the static output operator, the static sub-network to which the current operator belongs is firstly obtained, and then the static sub-network is used as the allocation object to apply for the minimum memory meeting the memory requirements of the plurality of static output operators from the memory pool; therefore, the memory can be applied for all the static output operators in the static sub-network only through one-time application, the memory allocation frequency is reduced, memory use is reduced, and the model reasoning efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a memory allocation method, device, equipment and storage medium. Background Art

[0002] The relevant memory allocation method is to use a dynamic memory pool allocation algorithm for operators whose input is not fixed size, and a static memory pool allocation algorithm for operators whose input is fixed size. However, as deep network models become more and more complex, there are more and more operators with non-fixed input sizes, which causes the above memory allocation method to continuously request memory from the memory pool, causing the memory pool to become larger and larger, reducing inference efficiency. Summary of the invention

[0003] The main purpose of this application is to provide a memory allocation method, device, equipment and storage medium, aiming to solve the technical problem that the related memory allocation method will continuously apply for memory from the memory pool, the memory pool will become larger and larger, and the reasoning efficiency will be reduced.

[0004] To achieve the above object, the present application provides a memory allocation method, the memory allocation method comprising:

[0005] In response to a data processing request, performing model reasoning on the data to be processed through a neural network model, wherein the neural network model includes a plurality of operators, and the neural network model performs model reasoning through the plurality of operators;

[0006] When the current operator is a static output operator, obtaining a static sub-network to which the current operator belongs, wherein the static sub-network includes a plurality of static output operators, and the static output operator is an operator that can obtain output data without reasoning;

[0007] An optimal memory size corresponding to the static subnetwork is calculated, and a memory block is allocated to the current operator according to the optimal memory size, wherein the optimal memory size is a minimum memory size that satisfies the memory requirements of multiple static output operators.

[0008] Optionally, calculating the optimal memory size corresponding to the static subnetwork includes:

[0009] Calculating the memory size required for output of each static output operator in the static subnetwork;

[0010] Get the life cycle of each static output operator;

[0011] The optimal memory size corresponding to the static sub-network is calculated according to the life cycle of each static output operator and the memory size required for the output of each static output operator.

[0012] Optionally, the calculating the optimal memory size corresponding to the static sub-network according to the life cycle of each static output operator and the memory size required for output of each static output operator includes:

[0013] The life cycle of each static output operator is used as the first dimension constraint, and the memory size required for output by each static output operator is used as the second dimension constraint;

[0014] Constructing a two-dimensional solution model according to the first-dimensional constraint and the second-dimensional constraint, wherein a solution target of the two-dimensional solution model is to obtain a minimum two-dimensional space under the first-dimensional constraint and the second-dimensional constraint;

[0015] The optimal memory size corresponding to the static subnetwork is calculated according to the two-dimensional solution model.

[0016] Optionally, allocating a memory block to the current operator according to the optimal memory size includes:

[0017] Allocate a corresponding optimal memory block to the static subnetwork according to the optimal memory size;

[0018] Calculating the offset of each static output operator in the minimum two-dimensional space, wherein the offset is used to represent the position offset value of each static output operator under the second dimensional constraint;

[0019] Calculate and allocate the absolute memory address of each static output operator according to the offset;

[0020] A memory block is allocated for the current operator in the optimal memory block according to the absolute memory address.

[0021] Optionally, allocating a corresponding optimal memory block to the static subnetwork according to the optimal memory size includes:

[0022] Traversing the free memory linked list of the memory pool to determine whether there is an optimal memory block that satisfies the optimal memory size;

[0023] If it exists, the optimal memory block is removed from the free memory linked list, added to the used memory linked list of the memory pool, and the optimal memory block is allocated to the static subnetwork;

[0024] If not, apply for a memory block of the optimal memory size from the storage system, add the applied memory block to the used memory linked list of the memory pool, and allocate the applied memory block to the static sub-network.

[0025] Optionally, after allocating a memory block for the current operator according to the optimal memory size, the method further includes:

[0026] Check whether other operators connected to each static output operator in the static subnetwork have allocated memory;

[0027] When other operators connected to each static output operator in the static sub-network have allocated memory, the memory block allocated to the static sub-network is released, and the released memory block is added to the free memory linked list of the memory pool.

[0028] Optionally, after performing model inference on the data to be processed by the neural network model in response to the data processing request, the method further includes:

[0029] When the current operator is a dynamic output operator, the current operator is inferred to obtain the memory size required for the current operator to output, wherein the dynamic output operator is an operator that needs to be inferred to obtain output data;

[0030] A memory block is allocated to the current operator according to the memory size required for the output of the current operator.

[0031] Optionally, before performing model inference on the data to be processed by the neural network model in response to the data processing request, the method further includes:

[0032] Obtaining a configuration file corresponding to the neural network model, and performing sub-network division on the neural network model according to the configuration file to obtain a sub-network division result;

[0033] According to the sub-network division result, the operators in the neural network model are divided into a static output operator set and a dynamic output operator set.

[0034] In addition, to achieve the above purpose, the present application also proposes a memory allocation device, the memory allocation device comprising:

[0035] A model reasoning module, used for performing model reasoning on the data to be processed through a neural network model in response to a data processing request, wherein the neural network model includes a plurality of operators, and the neural network model performs model reasoning through the plurality of operators;

[0036] A belonging acquisition module, used for acquiring the static sub-network to which the current operator belongs when the current operator is a static output operator, wherein the static sub-network includes a plurality of static output operators, and the static output operator is an operator that can obtain output data without reasoning;

[0037] A memory allocation module is used to calculate the optimal memory size corresponding to the static sub-network and allocate a memory block to the current operator according to the optimal memory size, wherein the optimal memory size is the minimum memory size that meets the memory requirements of multiple static output operators.

[0038] In addition, to achieve the above objectives, the present application also proposes a memory allocation device, which includes a memory, a processor, and a memory allocation program stored in the memory and executable on the processor, wherein the memory allocation program is configured to implement the memory allocation method described above.

[0039] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, on which a memory allocation program is stored, and when the memory allocation program is executed by a processor, the memory allocation method as described above is implemented.

[0040] One or more technical solutions proposed in this application have at least the following technical effects:

[0041] In the present application, it is disclosed that in response to a data processing request, a model inference is performed on the processed data through a neural network model. When the current operator is a static output operator, the static sub-network to which the current operator belongs is obtained, the optimal memory size corresponding to the static sub-network is calculated, and a memory block is allocated to the current operator according to the optimal memory size; because when the current operator is a static output operator, the present application first obtains the static sub-network to which the current operator belongs, and then applies to the memory pool for the minimum memory that meets the memory requirements of multiple static output operators with the static sub-network as the allocation object, so that only one application is needed to apply for memory for all static output operators in the static sub-network, thereby reducing the number of memory allocations, reducing memory usage, and improving model inference efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 This is a flowchart of the first embodiment of the memory allocation method of the present application;

[0045] Figure 2 This is a flowchart of a second embodiment of the memory allocation method of the present application;

[0046] Figure 3 This is a flowchart of a third embodiment of the memory allocation method of the present application;

[0047] Figure 4A schematic diagram of a two-dimensional solution model of an embodiment of a memory allocation method of the present application;

[0048] Figure 5 This is a flowchart of a fourth embodiment of the memory allocation method of the present application;

[0049] Figure 6 This is a specific flow chart of an embodiment of the memory allocation method of the present application;

[0050] Figure 7 This is a schematic diagram of the module structure of the memory allocation device according to an embodiment of the present application;

[0051] Figure 8 A schematic diagram of the device structure of the hardware operating environment involved in the memory allocation method in the embodiment of the present application.

[0052] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0054] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0055] At present, the relevant memory allocation method is to use a dynamic memory pool allocation algorithm for operators whose input is not fixed size, and to use a static memory pool allocation algorithm for operators whose input is fixed size. However, as deep network models become more and more complex, there are more and more operators with non-fixed input sizes, which will cause the above memory allocation method to continuously request memory from the memory pool, causing the memory pool to become larger and larger, reducing the inference efficiency.

[0056] Therefore, in order to overcome the above-mentioned defects, the present application provides a solution, which includes: in response to a data processing request, performing model inference on the data to be processed through a neural network model, and when the current operator is a static output operator, obtaining the static sub-network to which the current operator belongs, calculating the optimal memory size corresponding to the static sub-network, and allocating a memory block to the current operator according to the optimal memory size; because when the current operator is a static output operator, the present application first obtains the static sub-network to which the current operator belongs, and then applies to the memory pool for the minimum memory that meets the memory requirements of multiple static output operators with the static sub-network as the allocation object, so that only one application is needed to apply for memory for all static output operators in the static sub-network, thereby reducing the number of memory allocations, reducing memory usage, and improving model reasoning efficiency.

[0057] It should be noted that the execution subject of the embodiment of the present application can be a memory allocation device with data processing, network communication and program running functions, such as a computer, or other electronic devices that can achieve the same or similar functions, and this embodiment does not impose any restrictions on this.

[0058] Based on this, the embodiment of the present application provides a memory allocation method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the memory allocation method of the present application.

[0059] In a first embodiment, the memory allocation method includes:

[0060] Step S10: In response to the data processing request, performing model inference on the data to be processed through the neural network model;

[0061] The neural network model includes multiple operators, and the neural network model performs model inference through the multiple operators.

[0062] It should be understood that the data to be processed may be image data, audio data, and text data to be processed, and this embodiment does not limit this. For ease of understanding, the application scenario of this embodiment may be an image processing scenario, wherein the image processing scenario includes but is not limited to computer vision, image recognition, image segmentation, image enhancement, video processing, automatic driving, and the like, in which deep learning techniques (such as neural network models) are required to extract useful information from images or perform image conversion. In a specific implementation, the specific application scenario of the embodiment of the present application may be to allocate memory for operators in a neural network model when performing image processing through a neural network model.

[0063] It is understandable that the neural network model can be composed of multiple layers, each layer includes multiple nodes or neurons. In the specific implementation, these layers and nodes transmit and process data by performing specific mathematical operations (i.e., operators). In the specific implementation, model reasoning of the data to be processed through the neural network model can be to use a pre-trained neural network model to process and analyze the input data to be processed to obtain the corresponding output results. The process of processing and analyzing the input data to be processed involves multiple operators (or layers, functions, operations, etc.) inside the neural network model. Multiple operators work together to gradually convert the input image data into the desired output results through a series of mathematical calculations and transformations.

[0064] For ease of understanding, the following examples are given, but are not intended to limit the present application. As an example, model reasoning may include the following steps:

[0065] 1. Input processing: Before inputting the image data to be processed into the neural network model, the image data to be processed can be preprocessed, such as resizing, normalizing, cropping, etc., to ensure that the input data meets the input requirements of the neural network model, and then the image data to be processed can be input into the neural network model.

[0066] 2. Forward propagation: The image data to be processed is forward propagated through the neural network model. In this process, the data passes through each operator (layer) in the model in turn. Each operator applies a specific mathematical transformation or operation to the data, such as convolution, pooling, activation function, etc. The purpose of these transformations is to extract useful features or information from the input data and gradually convert it into a higher-level abstract representation.

[0067] 3. Calculate output: After being processed by multiple operators, the output of the model will eventually be obtained. This output may be the classification result of the image content, the bounding box of the target in the image, the mask of the image segmentation, etc., which depends on the design of the model and the target task. This embodiment does not limit this.

[0068] Step S20: when the current operator is a static output operator, obtaining the static sub-network to which the current operator belongs;

[0069] The static subnetwork includes a plurality of static output operators, and the static output operators are operators that can obtain output data without reasoning.

[0070] In this embodiment, the operators of the neural network model are divided into static output operators and dynamic output operators, wherein the static output operator is an operator that can obtain output data without reasoning, for example, the convolution operator. Since the convolution operator has a specific calculation formula, it is only necessary to know the input data and convolution parameters of the convolution operator to calculate the output data without actual reasoning. The dynamic output operator is an operator that needs to be reasoned to obtain output data, for example, the downsampling operator in point cloud processing. Since the output of the downsampling operator is determined by the distribution of the input (e.g., in areas with sparse point clouds, the downsampling operator will retain more points to maintain the shape characteristics of the area; and in areas with dense point clouds, the downsampling operator will remove more points to reduce data redundancy), therefore, the output data of the downsampling operator needs to be obtained through reasoning based on the input data.

[0071] It should be understood that in order to adapt to more scenarios and meet the actual needs of users, in this embodiment, the user can enter the layer name regarded as a dynamic output node and / or a static output node by filling in the configuration file to divide the operator in the neural network model into dynamic output nodes and static output nodes, and the user can also set the static sub-network to which the static output operator belongs by filling in the configuration file, and this embodiment does not limit this. In a specific implementation, before the step S10, it also includes: obtaining a configuration file corresponding to the neural network model, and performing sub-network division on the neural network model according to the configuration file to obtain a sub-network division result; dividing the operators in the neural network model into a static output operator set and a dynamic output operator set according to the sub-network division result, wherein the configuration file corresponding to the neural network model can be a configuration file filled in in advance by the user, and the configuration file is used to specify the layer name of the dynamic output node and / or the static output node. For example, the layer name regarded as the dynamic output operator is determined according to the configuration file, and then a plurality of sub-networks are divided according to the layer name with the dynamic output operator as the starting node, wherein the sub-network can be a static sub-network, and then the operators in the neural network model are divided into a static output operator set and a dynamic output operator set according to the sub-network.

[0072] It should be understood that since the static output operator can calculate the memory size required for the operator output before reasoning the operator, in this embodiment, a static sub-network can be composed of multiple static output operators, and then the static sub-network is used as the allocation object to apply to the memory pool for the minimum memory that meets the memory requirements of multiple static output operators, so that only one application is required for all static output operators in the static sub-network, thereby reducing the number of memory allocations and reducing memory usage; and since the dynamic output operator must be inferred before the operator can obtain the memory size required for the operator output, memory can only be allocated to the dynamic output operators one by one.

[0073] Step S30: Calculate the optimal memory size corresponding to the static sub-network, and allocate a memory block to the current operator according to the optimal memory size, wherein the optimal memory size is the minimum memory size that meets the memory requirements of multiple static output operators.

[0074] It can be understood that the optimal memory size corresponding to the static subnetwork can be calculated by calculating the memory size required for the output of each static output operator in the static subnetwork, and the optimal memory size corresponding to the static subnetwork is calculated according to the memory size required for the output of each static output operator. For example, the maximum memory size is selected from the memory size required for the output of each static output operator, and the maximum memory size is multiplied by a preset multiple to obtain the optimal memory size corresponding to the static subnetwork, wherein the preset multiple can be pre-set, and this embodiment does not limit this.

[0075] Of course, in order to improve the accuracy of the optimal memory size calculation, in this embodiment, the optimal memory size corresponding to the static subnetwork can also be calculated in combination with the life cycle of each static output operator and the memory size required for output, and this embodiment does not limit this.

[0076] It should be understood that allocating a memory block to the current operator according to the optimal memory size may be allocating a memory block to the static subnetwork according to the optimal memory size, and allocating memory to the current operator from the memory block allocated to the static subnetwork.

[0077] Of course, in this embodiment, if a memory block has been allocated to the static subnetwork before, there is no need to calculate the optimal memory size corresponding to the static subnetwork, but directly allocate memory to the current operator from the memory block allocated to the static subnetwork. That is, when a memory block has not been allocated to the static subnetwork, the optimal memory size corresponding to the static subnetwork is calculated, a memory block is allocated to the static subnetwork according to the optimal memory size, and memory is allocated to the current operator from the memory block allocated to the static subnetwork; when a memory block has been allocated to the static subnetwork, memory is directly allocated to the current operator from the memory block allocated to the static subnetwork.

[0078] Furthermore, in order to reduce memory usage, in this embodiment, the memory blocks corresponding to the operators whose life cycles have ended are released. Since the static output operators in this embodiment are combined into memory blocks applied for in the form of static subnetworks, this embodiment releases the memory blocks allocated to the static subnetworks. After step S30, it also includes: checking whether other operators connected to each static output operator in the static subnetwork have been allocated memory; when other operators connected to each static output operator in the static subnetwork have been allocated memory, releasing the memory blocks allocated to the static subnetwork, and adding the released memory blocks to the free memory linked list of the memory pool.

[0079] It should be understood that when other operators connected to each static output operator in the static subnetwork have allocated memory, it means that the output data of each static output operator in the static subnetwork is no longer used by any subsequent operator. Therefore, it can be determined that the life cycle of each static output operator in the static subnetwork has ended, and the memory block allocated to the static subnetwork can be released, and the released memory block is added to the free memory linked list of the memory pool.

[0080] For ease of understanding, the following examples are given, but are not intended to limit the present application. As an example, assume that the neural network model is a convolutional neural network (CNN) model, and the CNN model includes a convolution operator corresponding to a convolution layer, an activation operator corresponding to an activation layer, and a pooling operator corresponding to a pooling layer.

[0081] In the relevant memory allocation method, since the convolution operator, activation operator and pooling operator can accept input of any size, the convolution operator, activation operator and pooling operator are operators whose input is not a fixed size, and a dynamic memory pool allocation algorithm is required for memory allocation. That is, when the CNN model is used to perform model inference on the processed image data, it is necessary to apply for memory for the convolution operator, activation operator and pooling operator in the memory pool respectively, that is, at least three times of memory need to be applied to the memory pool.

[0082] However, when the memory allocation method of the present application is adopted, since the convolution operator, activation operator and pooling operator all have specific calculation methods, the output data can be calculated by knowing the input data and operator parameters without actual reasoning. Therefore, the convolution operator, activation operator and pooling operator are all static output operators, and the convolution operator, activation operator and pooling operator constitute a static sub-network. When the CNN model is used to perform model reasoning on the image data to be processed, if the current operator is detected for the first time as a static output operator (that is, any one of the convolution operator, activation operator and pooling operator), the optimal memory size corresponding to the static sub-network is directly calculated, and then a memory block is applied to the memory pool according to the minimum memory size, wherein the optimal memory size is the minimum memory size that meets the memory requirements of the convolution operator, activation operator and pooling operator; when the current operator is subsequently detected as a static output operator, there is no need to apply for memory from the memory pool, but the memory resources of the previously applied memory block are directly allocated to the current operator. In summary, the memory allocation method of the present application only requires one application to apply for memory for all static output operators in the static subnetwork. Compared with the related memory allocation method that requires at least three applications to the memory pool, the present application significantly reduces the number of memory allocations.

[0083] In this embodiment, when the current operator is a static output operator, the static sub-network to which the current operator belongs is first obtained, and then the minimum memory that meets the memory requirements of multiple static output operators is applied to the memory pool with the static sub-network as the allocation object, so that only one application is needed to apply for memory for all static output operators in the static sub-network, thereby reducing the number of memory allocations, reducing memory usage, and improving model reasoning efficiency.

[0084] Reference Figure 2 , Figure 2 This is a flow chart of the second embodiment of the memory allocation method of the present application, based on the above Figure 1 The first embodiment shown provides a second embodiment of the memory allocation method of the present application.

[0085] In the second embodiment, the step S30 includes:

[0086] Step S301: Calculate the memory size required for output of each static output operator in the static sub-network.

[0087] It should be understood that, in this embodiment, the memory allocated to the static output operator is used to store the output data (such as output size) of the static output operator. Therefore, the memory size required for calculating the output of each static output operator in the static subnetwork can be the output data of each static output operator in the static subnetwork, and the memory size required for the output of each static output operator is determined according to the output data.

[0088] In a specific implementation, the static output operator is an operator that can calculate output data through a formula. For example, the static output operator can be a convolution operator, and the calculation formula of the convolution operator is:

[0089] H_out=(H_in+2*padh–dilation_h*(kh–1)-1) / stride_h)

[0090] W_out=(W_in+2*padw–dilation_w*(kw–1)-1) / stride_w)

[0091] Where H_out is the height of the output feature map, W_out is the width of the output feature map, H_in is the height of the input feature map, W_in is the width of the input feature map, padh is the number of pixels padded in the height direction of the input feature map, padw is the number of pixels padded in the width direction of the input feature map, padding is to keep the size of the feature map unchanged or change it in a specific way after the convolution operation, dilation_h is the dilation rate of the dilated convolution in the height direction, dilation_w is the dilation rate of the dilated convolution in the width direction, dilated convolution allows spaces to be inserted between the elements of the convolution kernel, thereby increasing the receptive field without increasing the number of parameters, kh is the height of the convolution kernel, kw represents the width of the convolution kernel, stride_h is the stride in height, stride_w is the stride in width, and the stride determines the distance the convolution kernel slides on the input feature map.

[0092] From the above content, it can be seen that the output data of the static output operator (such as the height and width of the output feature map) can be calculated through the formula based on the input data (such as the height and width of the input feature map) and the operator parameters (such as the convolution parameters, the number of pixels filled in the height direction of the input feature map, etc.).

[0093] Step S302: Obtain the life cycle of each static output operator.

[0094] It should be noted that the life cycle can be the period of time during which the operator output data remains valid in the memory. This period starts from the time the operator starts executing and generates output data, and ends when the output data is no longer used by any subsequent operator and can be safely released.

[0095] It should be understood that the life cycle of each static output operator can be obtained by analyzing the data flow in the static sub-network to determine how the output data of each static output operator is used by subsequent operators. Specifically, it can include determining which operators are dependent on the output of the current operator and how these dependencies affect the use and release of memory; based on data flow analysis and the execution order of operators, the life cycle of each static output operator is calculated.

[0096] Step S303: Calculate the optimal memory size corresponding to the static sub-network according to the life cycle of each static output operator and the memory size required for output of each static output operator.

[0097] It can be understood that calculating the optimal memory size corresponding to the static sub-network according to the life cycle of each static output operator and the memory size required for each static output operator to output can be calculating the optimal memory size corresponding to the static sub-network according to the life cycle of each static output operator and the memory size required for each static output operator to output through a preset optimization algorithm, wherein the preset optimization algorithm can be pre-set to solve the minimum memory size that meets the memory requirements of multiple static output operators, and this embodiment does not limit this.

[0098] Step S304: Allocate a memory block for the current operator according to the optimal memory size.

[0099] It should be understood that allocating a memory block to the current operator according to the optimal memory size may be first allocating an optimal memory block satisfying the optimal memory size to the static subnetwork in the memory pool, and then allocating a memory block to the current operator in the optimal memory block.

[0100] This embodiment calculates the optimal memory size corresponding to the static sub-network in combination with the life cycle of each static output operator and the memory size required for output, thereby improving the accuracy of the optimal memory size calculation.

[0101] Reference Figure 3 , Figure 3 This is a flow chart of the third embodiment of the memory allocation method of the present application. Based on the above second embodiment, the third embodiment of the memory allocation method of the present application is proposed.

[0102] In the third embodiment, the step S303 includes:

[0103] Step S3031: The life cycle of each static output operator is used as the first dimension constraint, and the memory size required for output of each static output operator is used as the second dimension constraint.

[0104] It should be understood that in order to further improve the accuracy of the optimal memory size calculation, in this embodiment, the life cycle of each static output operator and the memory size required for the output of each static output operator are used as optimization constraints, and the minimum memory size that meets the memory requirements of multiple static output operators is obtained as the optimization goal. A two-dimensional solution model is constructed, and the optimal memory size corresponding to the static sub-network is calculated through the two-dimensional solution model.

[0105] Step S3032: constructing a two-dimensional solution model according to the first-dimensional constraint and the second-dimensional constraint;

[0106] The solution goal of the two-dimensional solution model is to obtain the minimum two-dimensional space under the first dimensional constraint and the second dimensional constraint.

[0107] For ease of understanding, the following examples are given, but are not intended to limit the present application. As an example, assume that the life cycle is the horizontal axis of the two-dimensional matrix, the memory size is the vertical axis of the two-dimensional matrix, and the total area of ​​the matrix (i.e., the total memory usage multiplied by the maximum value of the life cycle) is as small as possible to construct a two-dimensional matrix.

[0108] For ease of understanding, refer to Figure 4 This invention is provided for illustration, but is not intended to limit the present application. Figure 4 This is a schematic diagram of a two-dimensional solution model for an embodiment of the memory allocation method of the present application. As an example, in the figure, it is assumed that a matrix is ​​used to represent the memory block, the length is the memory size, and the width is the life cycle. During the life cycle, the operator will always occupy a certain memory block, and the corresponding memory block will not be released until the end of the life cycle. Rectangular blocks can be constructed through the life cycle and memory size of the operator, and the goal of the memory allocation solution is to accommodate more rectangular blocks in the memory life cycle diagram. The constraint of the problem is that there is no collision between rectangular blocks. In this embodiment, the optimal memory size corresponding to the static subnetwork can be calculated using the memory pool + greedy algorithm, or the optimal memory size corresponding to the static subnetwork can be calculated using a heuristic algorithm:

[0109] The first method is to use a greedy algorithm, such as the Bestfit algorithm, to select the smallest memory block that meets the conditions from the memory pool each time. For example, record the offset memory length, such as allocating A, offset 10, allocating B, offset 10 (record the starting offset 10, 20), allocating C, offset 5, then the total memory offset is 25, when allocating D, A can be released, so a memory block of 10 is freed, 10 just meets the requirements, so this 10 is given to D, and the global offset is recorded as 10. In this way, when allocating memory to the operator later, there are two ways: 1. Consider continuing to offset: 2. Use the already offset free memory. The final total offset is the optimal memory size corresponding to the static subnetwork.

[0110] The second method is to use heuristic algorithms to get as close to the global optimum as possible. The reasoning order of operators A, B, C, D starting from B and C in the above figure can be changed. It can be changed by optimizing the topological sorting, such as designing rules to allocate the larger memory first, or the longer life cycle first. The goal is to arrange as many matrix blocks as possible in the smallest memory.

[0111] Step S3033: Calculate the optimal memory size corresponding to the static sub-network according to the two-dimensional solution model.

[0112] It is understandable that the optimal memory size corresponding to the static subnetwork can be calculated according to the two-dimensional solution model by combining the life cycle of each static output operator and the memory size required for the output of each static output operator into a two-dimensional matrix, and solving to obtain the two-dimensional matrix with the smallest area. The vertical axis coordinate corresponding to the two-dimensional matrix with the smallest area is the optimal memory size corresponding to the static subnetwork. In a specific implementation, for example, the optimal memory size corresponding to the static subnetwork is calculated according to the two-dimensional solution model through a heuristic algorithm such as a greedy algorithm, a genetic algorithm, and simulated annealing.

[0113] This embodiment uses the life cycle of each static output operator and the memory size required for each static output operator to output as optimization constraints, and takes the minimum memory size that meets the memory requirements of multiple static output operators as the optimization goal, constructs a two-dimensional solution model, and calculates the optimal memory size corresponding to the static subnetwork through the two-dimensional solution model, thereby further improving the accuracy of the optimal memory size calculation.

[0114] In the third embodiment, the step S304 includes:

[0115] Step S3041: allocating a corresponding optimal memory block to the static sub-network according to the optimal memory size.

[0116] It should be understood that in order to achieve optimal allocation of memory blocks, in this embodiment, the corresponding optimal memory blocks are first allocated to the static sub-network according to the optimal memory size, and then the offset of each static output operator in the minimum two-dimensional space is calculated, and the memory block is allocated to the current operator according to the offset.

[0117] It can be understood that according to the optimal memory size calculated previously (i.e., the minimum amount of memory that meets the life cycle and memory size requirements of all static output operators), a continuous optimal memory block is allocated to the static subnetwork, the size of the optimal memory block is equal to the optimal memory size, and the optimal memory block is used to store all necessary output data during the execution of the static subnetwork.

[0118] Furthermore, in order to improve the efficiency of memory block allocation, in this embodiment, a free memory linked list is used to record unallocated memory blocks in the memory pool, a used memory linked list is used to record allocated memory blocks in the memory pool, and when the memory blocks in the memory pool are insufficient, new memory is applied for. The step S3041 includes: traversing the free memory linked list of the memory pool to see whether there is an optimal memory block that meets the optimal memory size; if so, removing the optimal memory block from the free memory linked list, adding it to the used memory linked list of the memory pool, and allocating the optimal memory block to the static subnetwork; if not, applying for a memory block of the optimal memory size from the storage system, adding the applied memory block to the used memory linked list of the memory pool, and allocating the applied memory block to the static subnetwork.

[0119] It should be noted that the memory pool can be a memory allocation method. Specifically, before the memory is actually used, a certain number of memory blocks are first applied for allocation to the memory pool for backup. When there is a new memory demand, a part of the memory blocks are allocated from the memory pool. If the memory blocks in the memory pool are not enough, new memory is applied to the storage system.

[0120] Step S3042: Calculate the offset of each static output operator in the minimum two-dimensional space;

[0121] The offset is used to represent the position offset value of each static output operator under the second dimension constraint.

[0122] It should be understood that after determining the optimal memory block, it is necessary to calculate the offset of each static output operator in the minimum two-dimensional space (time-memory), wherein the offset may refer to the offset value from a certain reference point (such as the starting address of the memory block) to the storage location of the operator output data in the two-dimensional space. In this embodiment, the offset of the vertical axis (i.e., the memory size) is the main concern, so the offset here is actually the position offset value under the second dimension constraint (memory size).

[0123] In this embodiment, the calculation of the offset depends on the execution order and memory requirements of the static output operator. In a specific implementation, the offset of each operator can be accumulated and calculated in sequence starting from the start address of the memory block according to the execution order and memory requirements of the operator.

[0124] Step S3043: Calculate and allocate the absolute memory address of each static output operator according to the offset.

[0125] It can be understood that after obtaining the offset of each static output operator, the offset can be added to the starting address of the memory block to obtain the absolute memory address of each static output operator. This absolute address is unique and is used to access and modify the output data of the operator during program execution.

[0126] Step S3044: Allocate a memory block for the current operator in the optimal memory block according to the absolute memory address.

[0127] It should be understood that, according to the calculated absolute memory address, a memory block is allocated in the optimal memory block for the static output operator currently being processed. The size of this memory block is equal to the memory size required for the output of the operator, and the starting address is equal to the absolute memory address calculated previously.

[0128] This embodiment first allocates the corresponding optimal memory block to the static subnetwork according to the optimal memory size, then calculates the offset of each static output operator in the minimum two-dimensional space, and allocates the memory block to the current operator according to the offset, thereby achieving optimal allocation of memory blocks.

[0129] Reference Figure 5 , Figure 5 This is a flow chart of the fourth embodiment of the memory allocation method of the present application. Based on the above embodiments, the fourth embodiment of the memory allocation method of the present application is proposed.

[0130] In the fourth embodiment, after step S10, the method further includes:

[0131] Step S20': when the current operator is a dynamic output operator, infer the current operator to obtain the memory size required for the current operator output;

[0132] The dynamic output operator is an operator that needs to be inferred to obtain output data.

[0133] It should be understood that as neural network models become more and more complex, some operators have emerged that cannot directly calculate output data through formulas, and must rely on input data for reasoning to obtain output data. For example, the downsampling operator in point cloud processing, because the output of the downsampling operator is determined by the distribution of the input (such as in areas where the point cloud is sparse, the downsampling operator will retain more points to maintain the shape characteristics of the area; and in areas where the point cloud is dense, the downsampling operator will remove more points to reduce data redundancy), therefore, the output data of the downsampling operator needs to be obtained through reasoning. The related memory allocation method usually calculates the memory required by each operator first, then allocates memory to each operator, and then performs reasoning through the operator after the memory is allocated. Obviously, the related memory allocation method cannot allocate memory for dynamic output operators.

[0134] Therefore, in order to overcome the above defects, in this embodiment, when the current operator is a dynamic output operator, the output data of the current operator is first inferred, and then the memory size required for the dynamic output operator to output is determined based on the output data of the current operator, and then a memory block is allocated to the current operator based on the memory size required for the current operator to output.

[0135] Step S30': allocating a memory block to the current operator according to the memory size required for the output of the current operator.

[0136] It can be understood that allocating a memory block to the current operator according to the memory size required for the current operator output can be to traverse the free memory list of the memory pool to see whether there is a target memory block, wherein the target memory block is a memory block that satisfies the memory size required for the current operator output; if it exists, the target memory block is removed from the free memory list, added to the used memory list of the memory pool, and the target memory block is allocated to the current operator; if it does not exist, apply for a memory block from the storage system, add the applied memory block to the used memory list of the memory pool, and allocate the applied memory block to the current operator.

[0137] The memory pool can be a memory allocation method. Specifically, before actually using the memory, a certain number of memory blocks are allocated to the memory pool for backup. When there is a new memory demand, a part of the memory blocks are allocated from the memory pool. If the memory blocks in the memory pool are not enough, new memory is applied to the storage system. The free memory list is used to record the unallocated memory blocks in the memory pool, and the used memory list is used to record the allocated memory blocks in the memory pool.

[0138] Furthermore, in order to reduce memory usage, in this embodiment, the memory block corresponding to the operator whose life cycle has ended is released. After step S30', it also includes: checking whether other operators connected to the current operator have allocated memory; when other operators connected to the current operator have allocated memory, releasing the memory block allocated to the current operator, and adding the released memory block to the free memory linked list of the memory pool.

[0139] It should be understood that when other operators connected to the current operator have allocated memory, it means that the output data of the current operator is no longer used by any subsequent operators. Therefore, it can be determined that the life cycle of the current operator has ended, the memory block allocated to the current operator can be released, and the released memory block can be added to the free memory list of the memory pool.

[0140] In this embodiment, when the current operator is a dynamic output operator, the output data of the current operator is first inferred, and then the memory size required for the dynamic output operator to output is determined based on the output data of the current operator, and then a memory block is allocated to the current operator based on the memory size required for the current operator to output, thereby allocating memory to operators that need to obtain output data through inference.

[0141] For ease of understanding, refer to Figure 6 This invention is provided for illustration, but is not intended to limit the present application. Figure 6 This is a specific flow chart of an embodiment of the memory allocation method of the present application. In the figure, as an example, the memory allocation method may include the following steps:

[0142] 1. Input the data to be processed into the neural network model and start model reasoning;

[0143] 2. Determine whether the current operator is a dynamic output operator, where a dynamic output operator is an operator that needs to be inferred to obtain output data;

[0144] a. When the current operator is a dynamic output operator, the memory allocation method specifically includes:

[0145] 1) Reasoning the current operator to obtain the memory size required for the output of the current operator;

[0146] 2) Apply to the memory pool for a memory block of the size required for the current operator output;

[0147] 3) Determine whether there is a memory block in the free memory list of the memory pool that meets the memory size required for the current operator output;

[0148] 4) If it exists, remove the qualified memory block from the free memory list, add it to the used memory list of the memory pool, and allocate the qualified memory block to the current operator;

[0149] 5) If it does not exist, apply for a memory block from the storage system, add the applied memory block to the used memory list of the memory pool, and allocate the applied memory block to the current operator;

[0150] 6) Determine whether other operators connected to the current operator have allocated memory;

[0151] 7) If yes, it means that the life cycle of the current operator has ended and it can be put back into the memory pool, releasing the memory block allocated to the current operator and adding the released memory block to the free memory list of the memory pool.

[0152] b. When the current operator is a static output operator, the memory allocation method specifically includes:

[0153] 1) Get the static sub-network to which the current operator belongs. The static sub-network consists of multiple static output operators;

[0154] 2) Calculate the memory size required for the output of each static output operator in the static subnetwork;

[0155] 3) Calculate the optimal memory size corresponding to the static subnetwork according to the life cycle of each static output operator and the memory size required for the output of each static output operator;

[0156] 4) Determine whether there is a memory block that meets the optimal memory size in the free memory list of the memory pool;

[0157] 5) If it exists, remove the qualified memory block from the free memory list, add it to the used memory list of the memory pool, and allocate the qualified memory block to the static subnetwork;

[0158] 6) If it does not exist, apply for a memory block from the storage system, add the applied memory block to the used memory list, and allocate the applied memory block to the static subnetwork;

[0159] 7) Allocate a memory block for the current operator according to the absolute memory address of each static output operator;

[0160] 8) Determine whether other operators connected to the static sub-network have allocated memory;

[0161] 9) If yes, it means that the life cycle of each static output operator in the static subnetwork has ended and can be put back into the memory pool, releasing the memory block allocated to the static subnetwork and adding the released memory block to the free memory list of the memory pool.

[0162] 3. Determine whether the neural network model is completed. If the neural network model is not completed, the next operator of the neural network model can be used as the current operator, and return to step 2 to continue reasoning until the neural network model is completed. After the neural network model is completed, the reasoning ends. Of course, after the current input reasoning ends, if there is new input data, the allocation state of the memory pool can be maintained and reasoning can continue.

[0163] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the memory allocation method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0164] This application also provides a memory allocation device, please refer to Figure 7 , the memory allocation device comprises:

[0165] A model reasoning module 10 is used to perform model reasoning on the data to be processed through a neural network model in response to a data processing request, wherein the neural network model includes a plurality of operators, and the neural network model performs model reasoning through the plurality of operators;

[0166] The belonging acquisition module 20 is used to acquire the static sub-network to which the current operator belongs when the current operator is a static output operator, wherein the static sub-network includes a plurality of static output operators, and the static output operator is an operator that can obtain output data without reasoning;

[0167] The memory allocation module 30 is used to calculate the optimal memory size corresponding to the static sub-network and allocate a memory block to the current operator according to the optimal memory size, wherein the optimal memory size is the minimum memory size that meets the memory requirements of multiple static output operators.

[0168] The memory allocation device provided by the present application adopts the memory allocation method in the above embodiment, which can solve the technical problem that the related memory allocation method will continuously apply for memory from the memory pool, and the memory pool will become larger and larger, thereby reducing the reasoning efficiency. Compared with the related art, the beneficial effects of the memory allocation device provided by the present application are the same as the beneficial effects of the memory allocation method provided by the above embodiment, and the other technical features in the memory allocation device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0169] The present application provides a memory allocation device, which includes: at least one processor; and a memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the memory allocation method in the above-mentioned embodiment 1.

[0170] Reference below Figure 8 , which shows a schematic diagram of the structure of a memory allocation device suitable for implementing an embodiment of the present application. The memory allocation device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The memory allocation device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0171] like Figure 8 As shown, the memory allocation device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a ROM (Read Only Memory) 1002 or a program loaded from a storage device 1003 to a RAM (Random Access Memory) 1004. In RAM 1004, various programs and data required for the operation of the memory allocation device are also stored. The processing device 1001, ROM 1002, and RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, an LCD (Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. Communication device 1009 can allow memory allocation device to carry out wireless or wired communication with other equipment to exchange data.Although the memory allocation device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown.Can implement or have more or less systems alternatively.

[0172] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0173] The memory allocation device provided by the present application adopts the memory allocation method in the above embodiment, which can solve the technical problem that the related memory allocation method will continuously apply for memory from the memory pool, and the memory pool will become larger and larger, thereby reducing the reasoning efficiency. Compared with the related art, the beneficial effects of the memory allocation device provided by the present application are the same as the beneficial effects of the memory allocation method provided by the above embodiment, and the other technical features in the memory allocation device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0174] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0175] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0176] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the memory allocation method in the above-mentioned embodiment.

[0177] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory), optical fiber, CD-ROM (CD-Read Only Memory, portable compact disk read-only memory), optical storage device, magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0178] The computer-readable storage medium may be included in the memory allocation device; or may exist independently without being assembled into the memory allocation device.

[0179] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the memory allocation device, the memory allocation device executes the memory allocation method described above.

[0180] The computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a LAN (Local Area Network) or a WAN (Wide Area Network), or it can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0181] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0182] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0183] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned memory allocation method, and can solve the technical problem that the related memory allocation method will continuously apply for memory from the memory pool, and the memory pool will become larger and larger, reducing the reasoning efficiency. Compared with the related art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the memory allocation method provided in the above-mentioned embodiment, which will not be repeated here.

[0184] The above descriptions are only some embodiments of the present application, and are not intended to limit the protection scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the protection scope of the present application.

Claims

1. A memory allocation method, characterized in that: The memory allocation method comprises: In response to a data processing request, performing model reasoning on the data to be processed through a neural network model, wherein the neural network model includes a plurality of operators, and the neural network model performs model reasoning through the plurality of operators; When the current operator is a static output operator, obtaining a static sub-network to which the current operator belongs, wherein the static sub-network includes a plurality of static output operators, and the static output operator is an operator that can obtain output data without reasoning; An optimal memory size corresponding to the static subnetwork is calculated, and a memory block is allocated to the current operator according to the optimal memory size, wherein the optimal memory size is a minimum memory size that satisfies the memory requirements of multiple static output operators.

2. The memory allocation method according to claim 1, wherein: The calculating the optimal memory size corresponding to the static sub-network includes: Calculating the memory size required for output of each static output operator in the static subnetwork; Get the life cycle of each static output operator; The optimal memory size corresponding to the static sub-network is calculated according to the life cycle of each static output operator and the memory size required for the output of each static output operator.

3. The memory allocation method according to claim 2, characterized in that: The calculating the optimal memory size corresponding to the static sub-network according to the life cycle of each static output operator and the memory size required for output of each static output operator includes: The life cycle of each static output operator is used as the first dimension constraint, and the memory size required for output by each static output operator is used as the second dimension constraint; Constructing a two-dimensional solution model according to the first-dimensional constraint and the second-dimensional constraint, wherein a solution target of the two-dimensional solution model is to obtain a minimum two-dimensional space under the first-dimensional constraint and the second-dimensional constraint; The optimal memory size corresponding to the static subnetwork is calculated according to the two-dimensional solution model.

4. The memory allocation method according to claim 3, characterized in that: The allocating a memory block to the current operator according to the optimal memory size includes: Allocate a corresponding optimal memory block to the static subnetwork according to the optimal memory size; Calculating the offset of each static output operator in the minimum two-dimensional space, wherein the offset is used to represent the position offset value of each static output operator under the second dimensional constraint; Calculate and allocate the absolute memory address of each static output operator according to the offset; A memory block is allocated for the current operator in the optimal memory block according to the absolute memory address.

5. The memory allocation method according to claim 4, characterized in that: The allocating a corresponding optimal memory block to the static subnetwork according to the optimal memory size includes: Traversing the free memory linked list of the memory pool to determine whether there is an optimal memory block that satisfies the optimal memory size; If it exists, the optimal memory block is removed from the free memory linked list, added to the used memory linked list of the memory pool, and the optimal memory block is allocated to the static subnetwork; If not, apply for a memory block of the optimal memory size from the storage system, add the applied memory block to the used memory linked list of the memory pool, and allocate the applied memory block to the static sub-network.

6. The memory allocation method according to any one of claims 1 to 5, characterized in that: After allocating a memory block for the current operator according to the optimal memory size, the method further includes: Check whether other operators connected to each static output operator in the static subnetwork have allocated memory; When other operators connected to each static output operator in the static sub-network have allocated memory, the memory block allocated to the static sub-network is released, and the released memory block is added to the free memory linked list of the memory pool.

7. The memory allocation method according to any one of claims 1 to 5, characterized in that: After the method performs model inference on the data to be processed by the neural network model in response to the data processing request, it also includes: When the current operator is a dynamic output operator, inferring the current operator to obtain the memory size required for the current operator to output, wherein the dynamic output operator is an operator that needs to be inferred to obtain output data; A memory block is allocated to the current operator according to the memory size required for the output of the current operator.

8. The memory allocation method according to any one of claims 1 to 5, characterized in that: Before the method performs model inference on the data to be processed by the neural network model in response to the data processing request, the method further includes: Obtaining a configuration file corresponding to the neural network model, and performing sub-network division on the neural network model according to the configuration file to obtain a sub-network division result; According to the sub-network division result, the operators in the neural network model are divided into a static output operator set and a dynamic output operator set.

9. A memory allocation device, characterized in that: The memory allocation device comprises: A model reasoning module, used for performing model reasoning on the data to be processed through a neural network model in response to a data processing request, wherein the neural network model includes a plurality of operators, and the neural network model performs model reasoning through the plurality of operators; A belonging acquisition module, used for acquiring the static sub-network to which the current operator belongs when the current operator is a static output operator, wherein the static sub-network includes a plurality of static output operators, and the static output operator is an operator that can obtain output data without reasoning; A memory allocation module is used to calculate the optimal memory size corresponding to the static sub-network and allocate a memory block to the current operator according to the optimal memory size, wherein the optimal memory size is the minimum memory size that meets the memory requirements of multiple static output operators.

10. A memory allocation device, characterized in that: The memory allocation device comprises: a memory, a processor, and a memory allocation program stored in the memory and executable on the processor, wherein the memory allocation program implements the memory allocation method according to any one of claims 1 to 8 when executed by the processor.

11. A storage medium, characterized in that: The storage medium stores a memory allocation program, and when the memory allocation program is executed by a processor, the memory allocation method according to any one of claims 1 to 8 is implemented.

Images