Method, apparatus and device for running control of network model

Abstract

This invention provides a method, apparatus, and device for controlling the operation of a network model. The method includes: determining the operating data type of a target network layer among multiple network layers included in the network model; if the operating data type is a static data type, obtaining target static data of the target network layer from a first storage space, wherein the static storage space in the network model's operation scenario is divided into a first storage space and a second storage space; and running the target network layer using at least the target static data. This invention solves the problem of low storage resource utilization in the static storage space supporting network model operation, thereby improving the storage resource utilization of the static storage space supporting network model operation.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computers, and more specifically, to a method, apparatus, and device for controlling the operation of a network model. Background Technology

[0002] In recent years, thanks to the significant improvement in computing power and the ease of access to big data, deep learning technology and network models have made great progress. More and more problems such as image processing and natural language analysis can be well solved through deep learning technology.

[0003] Deep learning network models currently typically run on general-purpose GPUs (Graphics Processing Units), and deploying these models is a crucial aspect of deep learning applications. Therefore, designing efficient methods to support network model execution naturally becomes a strong demand. The inherent computational characteristics of network models place specific requirements on storage resources. GPU on-chip storage is generally implemented using SRAM (Static Random Access Memory). SRAM offers higher read / write efficiency but is also more expensive. Furthermore, SRAM resources are often limited in devices and need to be fully utilized. Therefore, the design and use of on-chip storage resources in inference devices deploying deep learning network models are particularly critical. Summary of the Invention

[0004] This invention provides a method, apparatus, and device for controlling the operation of a network model, to at least address the problem of low storage resource utilization in static storage space supporting network model operation in related technologies.

[0005] According to an embodiment of the present invention, a method for controlling the operation of a network model is provided, comprising: determining the operating data type of a target network layer among a plurality of network layers included in the network model; if the operating data type is a static data type, obtaining target static data of the target network layer from a first storage space, wherein the static storage space in the operation scenario of the network model is divided into a first storage space and a second storage space, the first storage space being used to store static data used by the network model during operation, and the second storage space being used to store dynamic data used and output by the network model during operation; and running the target network layer using at least the target static data.

[0006] In an exemplary embodiment, obtaining the target static data of the target network layer from the first storage space includes: determining the target operation type of the target network layer from multiple operation types of network layers that use static data; determining the target storage subspace corresponding to the target operation type from multiple operation types and multiple storage subspaces that have a corresponding relationship; and reading the target static data of the target network layer from the target storage subspace.

[0007] In an exemplary embodiment, determining the target storage subspace corresponding to the target operation type from a plurality of operation types and a plurality of storage subspaces with corresponding relationships includes: when the target operation type is a convolution operation, determining a first subspace among the plurality of storage subspaces as the target storage subspace; and when the target operation type is an activation operation, determining a second subspace among the plurality of storage subspaces as the target storage subspace.

[0008] In an exemplary embodiment, reading the target static data of the target network layer from the target storage subspace includes: when the target operation type is a convolution operation, reading the convolution weight data and bias data of the target network layer from the first subspace as the target static data; and when the target operation type is an activation operation, reading the activation coefficient data of the target network layer from the second subspace as the target static data.

[0009] In one exemplary embodiment, the method further includes: determining a current allocation direction for allocating storage space for the output data of the target network layer in the second storage space, wherein the current allocation direction is opposite to the previous allocation direction for allocating storage space for data in the second storage space; and allocating target storage space for the output data of the target network layer in the second storage space according to the current allocation direction, wherein the target storage space is used to store the target output data output by the target network layer.

[0010] In one exemplary embodiment, determining the current allocation direction for allocating storage space for the output data of the target network layer in the second storage space includes: determining a reference pointer used in the previous allocation of storage space for data in the second storage space from a first pointer and a second pointer, wherein the first pointer is used to indicate the top start position of allocating storage space from the top to the bottom of the second storage space, and the second pointer is used to indicate the bottom start position of allocating storage space from the bottom to the top of the second storage space; if the reference pointer is the first pointer, determining the second pointer as the target pointer and determining the allocation direction indicated by the second pointer as the current allocation direction; if the reference pointer is the second pointer, determining the first pointer as the target pointer and determining the allocation direction indicated by the first pointer as the current allocation direction.

[0011] In one exemplary embodiment, allocating target storage space in the second storage space for the output data of the target network layer according to the current allocation direction includes: allocating the target storage space in the second storage space from the start position indicated by the target pointer according to the current allocation direction; and updating the target pointer according to the end position of the target storage space.

[0012] In an exemplary embodiment, after allocating target storage space in the second storage space for the output data of the target network layer according to the current allocation direction, the method further includes: recording the remaining usage count corresponding to the target storage space, wherein the initial value of the remaining usage count is the number of network layers that use the output data of the target network layer as input data; decrementing the remaining usage count by 1 when the output data of the target network layer is used as input data; and releasing the target storage space and updating the target pointer when the remaining usage count is 0.

[0013] In one exemplary embodiment, determining the current allocation direction for allocating storage space for the output data of the target network layer in the second storage space includes: determining the data attributes of the output data of the target network layer; and determining the current allocation direction for allocating storage space for the output data of the target network layer in the second storage space when the data attributes indicate that the target network layer is not included in the same group of network layers in the network model.

[0014] In an exemplary embodiment, after determining the data attributes of the output data of the target network layer, the method further includes: when the data attributes indicate that the target network layer is included in the same group of network layers in the network model and the same group of network layers has been allocated storage space, determining the storage space corresponding to the output data of the target network layer from the storage space already allocated for the network layer group to which the same group of network layers belongs, wherein the output data of the same group of network layers and the output data of the target network layer are both input to the same network layer; when the data attributes indicate that the target network layer is included in the same group of network layers in the network model and the same group of network layers has not been allocated storage space, determining the target network layer and the same group of network layers as a reference network layer group, wherein the output data of the same group of network layers and the output data of the target network layer are both input to the same network layer; determining a current allocation direction for allocating storage space for the output data of the reference network layer group in the second storage space; allocating storage space for the output data of the reference network layer group in the second storage space according to the current allocation direction; and dividing storage space for each network layer included in the reference network layer group in the storage space allocated for the reference network layer group.

[0015] According to another embodiment of the present invention, a virtual device for controlling the operation of a network model is provided, comprising: a first determining module, configured to determine the operating data type of a target network layer among a plurality of network layers included in the network model; an acquiring module, configured to acquire target static data of the target network layer from a first storage space when the operating data type is a static data type, wherein the static storage space in the operation scenario of the network model is divided into a first storage space and a second storage space, the first storage space being used to store static data used by the network model during operation, and the second storage space being used to store dynamic data used and output by the network model during operation; and a running module, configured to run the target network layer using at least the target static data.

[0016] According to another embodiment of the present invention, a network model operation control device is provided, comprising: a processor and a static memory, wherein the static memory is divided into a first storage space and a second storage space, the first storage space is used to store static data of the network model during operation, and the second storage space is used to store dynamic data of the network model during operation, the network model comprising multiple network layers; the processor is configured to determine the operating data type of a target network layer among the multiple network layers; if the operating data type is a static data type, to obtain target static data of the target network layer from the first storage space; and to run the target network layer using at least the target static data.

[0017] In an exemplary embodiment, the first storage space includes: a first subspace and a second subspace, wherein the first subspace is used to store weight data and bias data of network layers whose operation type is convolution; and the second subspace is used to store activation coefficient data of network layers whose operation type is activation.

[0018] In one exemplary embodiment, the processor includes a convolution processing unit and an activation processing unit, wherein the convolution processing unit is configured to obtain target weight data and target bias data of the target network layer whose operation type is convolution operation from the first subspace as the target static data; and the activation processing unit is configured to obtain target activation coefficient data of the target network layer whose operation type is activation operation from the second subspace as the target static data.

[0019] In one exemplary embodiment, the processor includes: a data acquisition port and a data processing unit, wherein the data acquisition port is configured to determine a current allocation direction for allocating storage space in the second storage space for the output data of the target network layer, wherein the current allocation direction is opposite to the previous allocation direction for allocating storage space for data in the second storage space; the data processing unit is configured to allocate target storage space in the second storage space for the output data of the target network layer according to the current allocation direction, wherein the target storage space is used to store the target output data output by the target network layer.

[0020] According to yet another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0021] According to yet another embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0022] Through this invention, the static storage space in the network model's runtime environment is divided into a first storage space and a second storage space. The first storage space stores the static data used by the network model during operation, while the second storage space stores the dynamic data used and output by the network model during operation. For a target network layer among the multiple network layers included in the network model, if its data type is a static data type, the target static data of that target network layer can be obtained from the first storage space, and the target network layer can be run using at least the obtained target static data. This allows each network layer of the network model to make more rational use of the static storage space in the runtime environment, satisfying the storage allocation needs of the network model's operation while fully utilizing the storage space, thus better supporting the efficient operation of the network model. Therefore, this invention solves the problem of low storage resource utilization in the static storage space supporting network model operation, achieving the effect of improving the storage resource utilization of the static storage space supporting network model operation. Attached Figure Description

[0023] Figure 1 This is a hardware structure block diagram of a mobile terminal for a network model operation control method according to an embodiment of the present invention.

[0024] Figure 2 This is a flowchart of a network model operation control method according to an embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram of the network layer operation process in a network model according to an embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of the storage space allocation process of the network layer in the network model according to an embodiment of the present invention. Figure 1 ;

[0027] Figure 5 This is a schematic diagram of the storage space allocation process of the network layer in the network model according to an embodiment of the present invention. Figure 2 ;

[0028] Figure 6 This is a schematic diagram of the storage space allocation process of the network layer in the network model according to an embodiment of the present invention. Figure 3 ;

[0029] Figure 7 This is a schematic diagram of the grouping process of a network layer in a network model according to an embodiment of the present invention;

[0030] Figure 8 This is a schematic diagram of the storage space allocation process of the network layer in the network model according to an embodiment of the present invention. Figure 4 ;

[0031] Figure 9 This is a structural block diagram of a virtual device for controlling the operation of a network model according to an embodiment of the present invention;

[0032] Figure 10 This is a structural block diagram of a network model operation control device according to an embodiment of the present invention;

[0033] Figure 11 This is a schematic diagram of a convolutional neural network operating device according to an embodiment of the present invention. Detailed Implementation

[0034] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0036] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a network model operation control method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0037] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the network model operation control method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-described networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0038] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0039] This embodiment provides a method for controlling the operation of a network model running on the aforementioned mobile terminal. Figure 2 This is a flowchart of a network model operation control method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0040] Step S202: Determine the running data type of the target network layer among the multiple network layers included in the network model;

[0041] Step S204: When the above-mentioned running data type is a static data type, the target static data of the target network layer is obtained from the first storage space. In the running scenario of the above-mentioned network model, the static storage space is divided into the first storage space and the second storage space. The first storage space is used to store the static data used by the above-mentioned network model during operation, and the second storage space is used to store the dynamic data used and output by the above-mentioned network model during operation.

[0042] Step S206: Run the target network layer using at least the target static data described above.

[0043] Through the above steps, the static storage space in the network model's runtime environment is divided into a first storage space and a second storage space. The first storage space stores the static data used by the network model during operation, while the second storage space stores the dynamic output data used by the network model during operation. For a target network layer among the multiple network layers included in the network model, if its data type is a static data type, the target static data of that target network layer can be obtained from the first storage space. At least the obtained target static data is used to run the target network layer. This allows each network layer of the network model to make more rational use of the static storage space in the runtime environment, satisfying the storage allocation needs of the network model's operation while making full use of the storage space, thus better supporting the efficient operation of the network model. Therefore, this solves the problem of low storage resource utilization in the static storage space supporting network model operation, achieving the effect of improving the storage resource utilization of the static storage space supporting network model operation.

[0044] The entity performing the above steps can be any electronic device with network model operation control functions, such as terminal devices, servers, cloud service platforms, databases, etc., but is not limited to these.

[0045] Optionally, in this embodiment, the network model may include, but is not limited to, any network model that may require the use of static storage space to store data, such as: convolutional neural network models, recurrent neural network models, deep neural network models, generative adversarial network models, etc. In this embodiment, the operation and control process of the network model is described and explained using a convolutional neural network model as an example. The processing methods for other types of network models are similar and will not be repeated here.

[0046] Taking convolutional neural network (CNN) models as an example, the computational characteristics of CNN models include: First, the outputs of network layers in the network model are generally consumed immediately, with most being used as the input for the next layer immediately after output, and only used once as the input for the next layer; Second, the outputs of a small number of network layers are used as the input for other layers multiple times; In addition, for network models with a concat (connection function) mechanism, the data blocks output by several network layers are combined into a whole and used as the input for other layers. Figure 3 This is a schematic diagram of the network layer operation process in a network model according to an embodiment of the present invention, such as... Figure 3As shown, data that remains unchanged after the network structure is determined can be called static data, which may include, but is not limited to, weights, biases, and activation coefficients (such as PreLU coefficients). There is also data that changes with different inputs, which can be called dynamic data. This includes the dynamic data used by the network model during operation, i.e., input data (such as feature input), and the dynamic data output by the network model during operation, i.e., output data (such as feature output), etc.

[0047] Optionally, in this embodiment, the network layers in the network model may include, but are not limited to, convolutional layers (Conv), activation layers (Active), pooling layers (Pool), etc. Different types of network layers can be further classified into different data types based on whether they use static data. For example, convolutional layers and activation layers can be classified as static data type network layers, while pooling layers can be classified as non-static data type network layers.

[0048] Optionally, in this embodiment, the operating scenario of the network model may include, but is not limited to, any device or platform capable of running a network model. For example, computer equipment, terminal equipment, server equipment, cloud service platforms, databases, etc.

[0049] Optionally, in this embodiment, the static storage space may include, but is not limited to, the SRAM configured in the runtime scenario. This SRAM is divided into a first storage space (which may be, but is not limited to, represented as SRAM_S) and a second storage space (which may be, but is not limited to, represented as SRAM_D). The first storage space SRAM_S may be used to store static data used by the network model during runtime, such as weights, biases, activation coefficients, etc. The second storage space SRAM_D may be used to store dynamic data used and output by the network model during runtime, such as input data, output data, etc.

[0050] Optionally, in this embodiment, in addition to acquiring static data, if the network layer also needs to use dynamic data during operation, the dynamic data used can be acquired from the second storage space, and the acquired static and dynamic data can be used to run the network layer. Taking a convolutional layer as an example, static data such as weights and biases are acquired from the first storage space, and dynamic data such as feature inputs are acquired from the second storage space to run the convolutional layer, and the dynamic data such as the output feature data is stored in the second storage space. Generally, the feature output of one convolutional layer will be used as the feature input of other layers.

[0051] In an exemplary embodiment, in step S204 above, the target static data of the target network layer can be obtained from the first storage space through, but is not limited to, the following steps:

[0052] S11, determine the target operation type of the target network layer from the operation types of multiple network layers that use static data;

[0053] S12, determine the target storage subspace corresponding to the target operation type from multiple operation types and multiple storage subspaces that have corresponding relationships;

[0054] S13, read the target static data of the target network layer from the target storage subspace.

[0055] Optionally, in this embodiment, for network layers that use static data, the first storage space can be further subdivided according to the operation type of the network layer, and a dedicated storage subspace can be allocated for each type of network layer. The division of storage subspaces can further improve the rationality of the use of static storage space.

[0056] In one exemplary embodiment, in step S12 above, the target storage subspace corresponding to the target operation type can be determined from multiple operation types and multiple storage subspaces with corresponding relationships through the following steps:

[0057] S21, when the target operation type is convolution operation, the first subspace among the plurality of storage subspaces is determined as the target storage subspace;

[0058] S22, when the target operation type is activation operation, the second subspace among the plurality of storage subspaces is determined as the target storage subspace.

[0059] Optionally, in this embodiment, a first subspace (SRAM_C) is allocated in the first storage space for the network layer performing convolution operations to store the static data used for convolution operations. A second subspace (SRAM_A) is allocated in the first storage space for the network layer performing activation operations to store the static data used for activation operations. Allocating private storage subspaces in the first storage space for different operation types of network layers to store their corresponding static data makes the use of static storage space more efficient.

[0060] In one exemplary embodiment, in step S13 above, different static data can be read from different storage subspaces in, but is not limited to, the following ways:

[0061] S31, when the target operation type is a convolution operation, read the convolution weight data and bias data of the target network layer from the first subspace as the target static data;

[0062] S32, if the target operation type is activation operation, read the activation coefficient data of the target network layer from the second subspace as the target static data.

[0063] Optionally, in this embodiment, the first subspace allocated for the convolution operation can store the convolution weight data and bias data used in the convolution operation, and the corresponding convolution weight data and bias data can be read to run the target network layer as target static data.

[0064] Optionally, in this embodiment, the second subspace allocated for activation operation can store activation coefficient data used in activation operation, and the corresponding activation coefficient data can be read to run the target network layer as target static data.

[0065] In one exemplary embodiment, the second storage space may be used during the operation of the network model through, but is not limited to, the following steps:

[0066] S41, determine the current allocation direction for allocating storage space for the output data of the target network layer in the second storage space, wherein the current allocation direction is opposite to the previous allocation direction for allocating storage space for data in the second storage space;

[0067] S42, according to the current allocation direction, allocate target storage space in the second storage space for the output data of the target network layer, wherein the target storage space is used to store the target output data output by the target network layer.

[0068] Optionally, in this embodiment, the allocation direction for allocating storage space for the network layer's output data in the second storage space may include, but is not limited to, allocation from top to bottom and allocation from bottom to top, which are opposite allocation directions. The allocation principle for the network layer's output data in the second storage space can be summarized as "staggered allocation with space in the middle," meaning that if the previous allocation was from top to bottom, the next allocation will be from bottom to top, and vice versa. This makes the use of the second storage space more rational, increases its utilization rate, and can support more diverse network model structures.

[0069] Optionally, in this embodiment, for the first network layer in the network model, the previous allocation of storage space for data in the second storage space may, but is not limited to, allocating storage space for its input data in the second storage space. For non-first network layers in the network model, the previous allocation of storage space for data in the second storage space may, but is not limited to, allocating storage space for the output data of its upper-level network layer in the second storage space.

[0070] Optionally, in this embodiment, the connection methods of the network layers in the network model include various structures, such as: sequence layer connection structures, connection structures for multiple uses of output data, residual network connection structures, concat connection structures, etc. For different connection structures, the above process can be used to allocate the storage space for the output data of the network layers.

[0071] Optionally, in this embodiment, during the allocation of storage space for the output data of the network layer, the reading location corresponding to the input data of the network layer can also be specified.

[0072] For example, in the context of sequential layer connection structures, some parts of a convolutional neural network are combinations of layers with single dynamic input and single dynamic output. This type of connection can be called a sequential layer connection structure. Figure 4 This is a schematic diagram of the storage space allocation process of the network layer in the network model according to an embodiment of the present invention. Figure 1 ,like Figure 4 As shown, the dynamic input and output data sizes of each network layer can first be calculated. The input data of layer 1 (Input1) starts at address 0 at the top of SRAM_D. The output data of layer 1 (Output1) is set by subtracting the output size from the bottom (for simplicity, it is assumed that the address increases from the top to the bottom). From the connection structure of the network layers, we know that the output address of layer 1 is the input of the next layer (layer 2). This address is set as the input of layer 2 (Input2), and the output of layer 2 (Output2) is set to the top of SRAM_D. Similarly, the output of layer 2 is set as the input address of layer 3 (Input3), and the output of layer 3 (Output3) is set to the bottom minus the output size. The above process is repeated to complete the input and output configuration of each layer.

[0073] As can be seen, the above allocation process generally follows the principle of "staggered top and bottom, with empty space in the middle", which will leave the largest possible continuous storage space, thus providing as much support as possible for the large storage requirements of the network.

[0074] It should be noted that the initial top address 0 in the above process is an example of initialization expression. More generally, it can be determined by the input of the output layer, and the staggered principle is also used to determine it.

[0075] In one exemplary embodiment, in step S41 above, the current allocation direction for allocating storage space for the output data of the target network layer in the second storage space can be determined, but is not limited to, by the following means:

[0076] S51, determine the reference pointer used in the last allocation of storage space for data in the second storage space from the first pointer and the second pointer, wherein the first pointer is used to indicate the top start position of allocating storage space from the top to the bottom of the second storage space, and the second pointer is used to indicate the bottom start position of allocating storage space from the bottom to the top of the second storage space.

[0077] S52-1, when the reference pointer is the first pointer, the second pointer is determined as the target pointer, and the allocation direction indicated by the second pointer is determined as the current allocation direction;

[0078] S52-2, when the reference pointer is the second pointer, the first pointer is determined as the target pointer, and the allocation direction indicated by the first pointer is determined as the current allocation direction.

[0079] Optionally, in this embodiment, the allocation direction of storage space and the starting position of allocation in each allocation direction can be managed by setting pointers, but not limited to this method.

[0080] Optionally, in this embodiment, two pointers are set: a first pointer and a second pointer. The first pointer indicates the starting position of allocating storage space from the top to the bottom of the second storage space, and the second pointer indicates the starting position of allocating storage space from the bottom to the top of the second storage space. First, the pointer used in the previous storage space allocation is determined as the reference pointer. Then, the pointer that can be used for this storage space allocation is the other pointer besides the reference pointer, which is used as the target pointer, and the allocation direction indicated by it is used as the current allocation direction to control the allocation of storage space.

[0081] In one exemplary embodiment, in step S42 above, target storage space may be allocated to the output data of the target network layer in, but is not limited to, the following manner:

[0082] S61, in the second storage space, starting from the starting position indicated by the target pointer, allocate the target storage space for the output data of the target network layer according to the current allocation direction;

[0083] S62, Update the target pointer according to the end position of the target storage space.

[0084] Optionally, in this embodiment, the starting position and allocation direction of the allocation can be determined according to the target pointer, and storage space that matches the data volume of the output data of the target network layer can be allocated from the determined starting position according to the determined allocation direction as the target storage space.

[0085] Optionally, in this embodiment, after allocating the target storage space, the target pointer can be updated according to the end position of the target storage space so that it points to the new start position.

[0086] In one exemplary embodiment, after step S42 described above, the use of the target storage space can also be controlled in, but is not limited to, the following ways:

[0087] S71, record the remaining number of uses corresponding to the target storage space, wherein the initial value of the remaining number of uses is the number of network layers that use the output data of the target network layer as input data;

[0088] S72, if the output data of the target network layer is used as input data, the remaining number of uses is reduced by 1;

[0089] S73, if the remaining usage count is 0, release the target storage space and update the target pointer.

[0090] Optionally, in this embodiment, the remaining usage count corresponding to the target storage space is equivalent to a countdown timer for releasing the target storage space. The initial value of the remaining usage count is the number of network layers that use the output data of the target network layer as input data. Each time the output data of the target network layer is used as input data, the remaining usage count is decremented by 1 until the remaining usage count is reduced to 0. Then, the target storage space is released and the target pointer is updated to point to the top of the target storage space, so that the target storage space can be allocated and used again.

[0091] In an optional embodiment, for connection structures where output data is used multiple times, some network layers in the network model may have their output data used as input by multiple layers. For the output data of network layers with a usage count greater than 1, storage space can be allocated through, but is not limited to, the following process: First, determine the usage count of the output data. If it is greater than 1, it indicates that the output data needs to be used multiple times and cannot be discarded (overwritten) after one use, unlike sequential layers. For such output data, the allocation of SRAM_D still follows the principle of "staggered top and bottom, with empty spaces in the middle." Figure 5 This is a schematic diagram of the storage space allocation process of the network layer in the network model according to an embodiment of the present invention. Figure 2 ,like Figure 5As shown, space management and maintenance pointers ptT (equivalent to the first pointer mentioned above) and ptB (equivalent to the second pointer mentioned above) can be added to SRAM_D. ptT represents the allocation start pointer pointing to the top of SRAM_D, and ptB represents the allocation start pointer pointing to the bottom of SRAM_D. ptT increments downwards, and ptB decrements upwards. Each allocated memory space is incremented with corresponding control information, Cuse = n (equivalent to the remaining usage count mentioned above; this value is determined by the network layer connections, and the initial value is the number of layers using its output data as input). When a data block in the memory space is used as input, its Cuse is decremented by 1, and then it is checked whether it is 0. If it is 0, the memory space reclamation process is initiated.

[0092] The storage space reclamation process includes: If the storage space to be reclaimed is above the ptT pointer, the ptT pointer is decremented by the data block size to point to the starting address of the storage space to be reclaimed. Simultaneously, the Cuse value of the upper-level storage space is checked. If it is zero, the above process is repeated for storage space reclamation; otherwise, the storage space reclamation process ends. If the storage space to be reclaimed is below the ptB pointer, the ptB pointer is incremented by the size of the storage space to be reclaimed to point to the address after the end of the reclaimed storage space. Simultaneously, the Cuse value of the next storage block is checked. If it is zero, the above storage space reclamation process is repeated; otherwise, the storage space reclamation process ends.

[0093] For example, in the connection structure of residual networks, the residual method is a common way to reuse storage space multiple times. Figure 6 This is a schematic diagram of the storage space allocation process of the network layer in the network model according to an embodiment of the present invention. Figure 3 ,like Figure 6As shown, the network structure of the residual network includes: convolutional layer 1, convolutional layer 2, convolutional layer 3, and a residual layer (Sum(Residual)) that performs residual calculation using the outputs of convolutional layer 1 and convolutional layer 3. The output of convolutional layer 1 is used twice: as the input to convolutional layer 2 and as the input to the residual layer. Analysis of the network structure reveals that the output data block of convolutional layer 1 (i.e., the storage space for the output data of convolutional layer 1) has a Cuse = 2. The input and output of convolutional layer 1 are allocated. Assuming the input is at the top of SRAM_D with a usage count of 1, the output of convolutional layer 1, Output1, is allocated at the bottom, and the usage count Cuse = 2 is recorded, updating ptB. After this allocation, the input count of Input1 is decremented by 1 to zero, a recycling check is performed, and ptT is reset to 0. Convolutional layer 2 is allocated; the input block uses the previous output Output1, and the output Output2 is allocated on ptT, which is maintained. This allocation decrements the usage count of Output1 by 1. Output1's Cuse is 1 (non-zero), so no memory reclamation is performed, and ptB points to the beginning of this data block. Convolutional layer 3 is allocated, using the output Output2 from convolutional layer 2 as input. The output needs to be allocated in ptB, and ptB is maintained. This allocation indicates that Output2 can be reclaimed after use. After reclamation, ptT points back to 0. Note that at this point, there are two data blocks at the bottom, Output1 and Output3, both with a usage count of 1.

[0094] The residual layer is allocated as follows: Input 1 uses Output1, Input 2 uses Output3, and the output is allocated using ptT. After allocation, the usage counts of Output1 and Output3 are both 0. At this point, ptB is recycled, starting with Output3. Because it immediately follows ptB and its usage count is 0, it is recycled, ptB increments, pointing to Output1. Then, the usage count of Output1 also becomes 0, and recycling continues, with ptB pointing to the last position. It can be seen that during the entire memory allocation process described above, a contiguous maximum storage area is still reserved for subsequent use.

[0095] In one exemplary embodiment, in step S41 above, the current allocation direction can also be determined in, but is not limited to, the following ways:

[0096] S81, determine the data attributes of the output data of the target network layer;

[0097] S82, when the data attribute is used to indicate that the target network layer is not included in the same group of network layers in the network model, determine the current allocation direction for allocating storage space for the output data of the target network layer in the second storage space.

[0098] Optionally, in this embodiment, the data attributes of the output data of the target network layer may, but are not limited to, indicating whether the target network layer includes other network layers in the network model, and that the output data of the other network layers and the output data of the target network layer are both input to the same network layer. The target network layer may have already run network layers in the same group. For each network layer belonging to the same group, the storage space for the output data can be allocated uniformly for the entire group when allocating storage space for the first network layer in the group that needs to allocate storage space for its output data.

[0099] Optionally, in this embodiment, firstly, the data attributes of the output data of the target network layer are determined. If the data attributes indicate that the target network layer is not included in the network model, then the current allocation direction for allocating storage space for the output data of the target network layer in the second storage space is determined. For other cases, other methods can be used to allocate storage space.

[0100] Optionally, in this embodiment, the data attributes of the output data of the target network layer may also include, but are not limited to, attributes that may affect the allocation of storage space, such as the number of times the output data of the target network layer is used and its position in the network model.

[0101] In one exemplary embodiment, after step S81 described above, storage space may be allocated in different ways for different situations through, but is not limited to, the following steps:

[0102] S91, when the data attribute is used to indicate that the target network layer is included in the network model and the same group of network layers has been allocated storage space, the storage space corresponding to the output data of the target network layer is determined from the storage space already allocated for the network layer group to which the same group of network layers belongs, wherein the output data of the same group of network layers and the output data of the target network layer are both input to the same network layer.

[0103] S92, when the data attribute indicates that the target network layer is included in the same group of network layers in the network model and the same group of network layers has not been allocated storage space, the target network layer and the same group of network layers are determined as a reference network layer group, wherein the output data of the same group of network layers and the output data of the target network layer are both input to the same network layer; the current allocation direction for allocating storage space for the output data of the reference network layer group in the second storage space is determined; storage space is allocated for the output data of the reference network layer group in the second storage space according to the current allocation direction; and storage space is divided for each network layer included in the reference network layer group in the storage space allocated to the reference network layer group.

[0104] Optionally, in this embodiment, the storage space for the output data of each network layer belonging to the same group is centrally allocated. If the data attributes of the target network layer are used to indicate the network layers in the same group that include the target network layer in the network model, and the network layers in the same group have been allocated storage space, then the storage space for the output data of the target network layer can be determined from the storage space already allocated to the network layers in the same group.

[0105] Optionally, in this embodiment, if the target network layer has network layers in the same group, but the network layers in the same group have not been allocated storage space, then the storage space for the output data of the entire network layer group can be allocated according to the above principle of "staggered at the top and bottom, with empty space in the middle", and then the storage space for the output data of each network layer can be divided in the allocated storage space.

[0106] For example, in the connection structure of the concat function, the concat unit combines data blocks from different path calculations into a single data block. In subsequent network layer calculations, this single data block is processed and not calculated on a separate block. To support this, the concept of a "group" can be introduced, i.e., a network layer group, where all data blocks form a group. Storage management of data blocks within a group follows the principle of unified allocation and unified release. The allocation and reclamation of storage space used by each group can be performed according to the above process. Each group can also maintain management information for the data blocks within the group and complete the calculation of the compilation addresses of the data blocks within the group. Figure 7 This is a schematic diagram of the grouping process of the network layer in the network model according to an embodiment of the present invention, such as... Figure 7 As shown, the management and control information for each group's storage space includes: the number of blocks in the group and the size of each block in the group. Each allocated storage space can also be marked with its own group and the block index within that group.

[0107] Figure 8 This is a schematic diagram of the storage space allocation process of the network layer in the network model according to an embodiment of the present invention. Figure 4 ,like Figure 8 As shown, in a Concat structure, convolutional layer 1 calculates the input and outputs Output1. This output is then processed by three branches: convolutional layer 11 (conv11), convolutional layer 12 (conv12), and convolutional layer 13 (conv13) to obtain different data blocks. The three data blocks are then connected by a Concat unit and subsequently processed uniformly.

[0108] The connection structure of the concatenation function (Concat) can allocate storage space through, but is not limited to, the following process: For convolutional layer 1, allocate storage space, assuming the input is at the bottom, allocate the output, placing the output at the top, and record the Output1 usage count as 3. For convolutional layer 11, allocate storage space, using Output1 as input. The output block, based on pre-processed layer information, belongs to the Concat group; allocate storage space according to the group size, placing it at the bottom, and setting the group usage count to 1. Decrement the Output1 usage count by 1, resulting in 2. For the data block of convolutional layer 11, acquire storage space for each network layer within the group. The output data block Output11 has an index of 0 within the group and is allocated at the beginning address of the group. For convolutional layer 12, allocate storage space, using Output1 as input. The output block belongs to the Concat group, and group storage has already been allocated; calculate the address within the group. Decrement the Output1 usage count by 1, resulting in 1. For the data block of convolutional layer 12, acquire the address within the group. The output data block Output12 has an index of 1 within the group, and its address is set to the group address plus the size of all blocks before that index. Storage space is allocated for convolutional layer 13. The input uses Output1, and the output block belongs to the Concat group. The group storage has been allocated, and the intra-group address is calculated. Output1 uses a counter decremented by 1 to 0. After this calculation, Output1 is reclaimed. The intra-group address is obtained for the data block of convolutional layer 12. The intra-group index of the output data block Output13 is 2, and the address is set to the group address plus the size of all blocks before this index.

[0109] This completes the address allocation of the storage space for the connection structure of the Concat function. Subsequently, the entire group participates in the operation as a whole data block and also performs memory management operations as a whole.

[0110] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0111] This embodiment also provides a virtual device for controlling the operation of a network model. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0112] Figure 9 This is a structural block diagram of a virtual device for controlling the operation of a network model according to an embodiment of the present invention, such as... Figure 9 As shown, the device includes:

[0113] The first determining module 902 is used to determine the running data type of the target network layer among the multiple network layers included in the above network model;

[0114] The acquisition module 904 is used to acquire the target static data of the target network layer from the first storage space when the above running data type is a static data type. In the running scenario of the above network model, the static storage space is divided into the first storage space and the second storage space. The first storage space is used to store the static data used by the above network model during operation, and the second storage space is used to store the dynamic data used and output by the above network model during operation.

[0115] Run module 906 is used to run the target network layer using at least the target static data described above.

[0116] In one exemplary embodiment, the above-described acquisition module includes:

[0117] The first determining unit is used to determine the target operation type of the target network layer from multiple operation types of network layers that use static data.

[0118] The second determining unit is used to determine the target storage subspace corresponding to the target operation type from multiple operation types and multiple storage subspaces that have a corresponding relationship;

[0119] The reading unit is used to read the target static data of the target network layer from the target storage subspace.

[0120] In one exemplary embodiment, the second determining unit described above is configured to:

[0121] When the target operation type is convolution operation, the first subspace among the multiple storage subspaces is determined as the target storage subspace.

[0122] When the target operation type is activation operation, the second subspace among the multiple storage subspaces is determined as the target storage subspace.

[0123] In one exemplary embodiment, the above-described reading unit is used for:

[0124] When the target operation type is convolution operation, the convolution weight data and bias data of the target network layer are read from the first subspace as the target static data.

[0125] When the target operation type is activation operation, the activation coefficient data of the target network layer is read from the second subspace as the target static data.

[0126] In one exemplary embodiment, the above-described apparatus further includes:

[0127] The second determining module is used to determine the current allocation direction of allocating storage space for the output data of the target network layer in the second storage space, wherein the current allocation direction is opposite to the previous allocation direction of allocating storage space for data in the second storage space.

[0128] The allocation module is used to allocate target storage space in the second storage space according to the current allocation direction, wherein the target storage space is used to store the target output data of the target network layer.

[0129] In one exemplary embodiment, the second determining module described above includes:

[0130] The third determining unit is used to determine from the first pointer and the second pointer the reference pointer used in the last allocation of storage space for data in the second storage space, wherein the first pointer is used to indicate the top starting position of the allocation of storage space from the top to the bottom of the second storage space, and the second pointer is used to indicate the bottom starting position of the allocation of storage space from the bottom to the top of the second storage space.

[0131] The fourth determining unit is used to determine the second pointer as the target pointer when the reference pointer is the first pointer, and to determine the allocation direction indicated by the second pointer as the current allocation direction.

[0132] The fifth determining unit is used to determine the first pointer as the target pointer when the reference pointer is the second pointer, and to determine the allocation direction indicated by the first pointer as the current allocation direction.

[0133] In one exemplary embodiment, the above-mentioned allocation module includes:

[0134] The allocation unit is configured to allocate the target storage space in the second storage space from the starting position indicated by the target pointer according to the current allocation direction for the output data of the target network layer.

[0135] The update unit is used to update the target pointer according to the end position of the target storage space.

[0136] In one exemplary embodiment, the above-described apparatus further includes:

[0137] The recording module is used to record the remaining number of times the target storage space is used after allocating the target storage space in the second storage space according to the current allocation direction, and the initial value of the remaining number of times is the number of network layers that use the output data of the target network layer as input data.

[0138] The first processing module is used to decrement the remaining number of uses by 1 when the output data of the target network layer is used as input data.

[0139] The second processing module is used to release the target storage space and update the target pointer when the remaining usage count is 0.

[0140] In one exemplary embodiment, the second determining module described above includes:

[0141] The sixth determining unit is used to determine the data attributes of the output data of the target network layer mentioned above;

[0142] The seventh determining unit is used to determine the current allocation direction of allocating storage space for the output data of the target network layer in the second storage space when the above data attribute indicates that the target network layer is not included in the same group of network layers in the above network model.

[0143] In one exemplary embodiment, the above-described apparatus further includes:

[0144] The third determining module is used to determine the storage space corresponding to the output data of the target network layer from the storage space already allocated to the network layer group to which the target network layer is located, after determining the data attributes of the output data of the target network layer. In the case that the data attributes indicate that the target network layer is included in the network model and that the target network layer has been allocated storage space, the output data of the target network layer and the output data of the target network layer are both input to the same network layer.

[0145] The processing module is configured to: determine the target network layer and the network layers in the same group as a reference network layer group when the data attributes indicate that the target network layer is included in the network model and the network layers in the same group have not been allocated storage space; wherein the output data of the network layers in the same group and the output data of the target network layer are both input to the same network layer; determine the current allocation direction for allocating storage space for the output data of the reference network layer group in the second storage space; allocate storage space for the output data of the reference network layer group in the second storage space according to the current allocation direction; and divide storage space for each network layer included in the reference network layer group in the storage space allocated to the reference network layer group.

[0146] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0147] This embodiment also provides a network model operation control device. Figure 10 This is a structural block diagram of the network model operation control device according to an embodiment of the present invention, such as... Figure 10 As shown, the device includes: a processor 1002 and a static memory 1004, wherein,

[0148] The static memory 1004 is divided into a first storage space 1004-2 and a second storage space 1004-4. The first storage space 1004-2 is used to store static data of the network model during operation, and the second storage space 1004-4 is used to store dynamic data of the network model during operation. The network model includes multiple network layers.

[0149] The processor 1002 is configured to determine the running data type of the target network layer among the plurality of network layers; if the running data type is a static data type, to obtain the target static data of the target network layer from the first storage space 1004-2; and to run the target network layer using at least the target static data.

[0150] Typical convolutional neural network operations in deep learning include convolution, activation, pooling, accumulation, and residual operations. Running a network model requires a suitable runtime environment, particularly efficient utilization of static storage devices within that environment.

[0151] With the aforementioned device, the static storage space in the network model's runtime environment is divided into a first storage space and a second storage space. The first storage space stores static data during network model operation, while the second storage space stores dynamic data. For a target network layer among the multiple network layers included in the network model, if it belongs to a data type that uses static data, the target static data of that target network layer can be obtained from the first storage space. At least the obtained target static data is used to run the target network layer. This allows each network layer of the network model to more rationally utilize the static storage space in the runtime environment, satisfying the storage allocation needs of the network model while fully utilizing the storage space, thus better supporting the efficient operation of the network model. Therefore, this solves the problem of low storage resource utilization in the static storage space supporting network model operation, achieving the effect of improving the storage resource utilization of the static storage space supporting network model operation.

[0152] In one exemplary embodiment, the first storage space includes: a first subspace and a second subspace, wherein,

[0153] The first subspace mentioned above is used to store the weight and bias data of network layers that perform convolution operations.

[0154] The aforementioned second subspace is used to store the activation coefficient data of network layers whose operation type is activation operation.

[0155] In one exemplary embodiment, the processor includes: a convolution processing unit and an activation processing unit, wherein,

[0156] The aforementioned convolution processing unit is used to obtain target weight data and target bias data of the target network layer whose operation type is convolution operation from the aforementioned first subspace as the aforementioned target static data;

[0157] The activation processing unit described above is used to obtain the target activation coefficient data of the target network layer whose operation type is activation operation from the second subspace as the target static data.

[0158] In one exemplary embodiment, the processor includes: a data acquisition port and a data processing unit, wherein,

[0159] The aforementioned data acquisition port is used to determine the current allocation direction of the storage space allocated for the output data of the target network layer in the aforementioned second storage space, wherein the current allocation direction is opposite to the previous allocation direction of the storage space allocated for the data in the aforementioned second storage space.

[0160] The aforementioned data processing unit is configured to allocate target storage space in the second storage space for the output data of the target network layer according to the aforementioned current allocation direction, wherein the target storage space is used to store the target output data output by the target network layer.

[0161] In one exemplary embodiment, the data acquisition port is configured to: determine, from a first pointer and a second pointer, a reference pointer used in the previous allocation of storage space for data in the second storage space, wherein the first pointer indicates the top start position of allocating storage space from the top to the bottom of the second storage space, and the second pointer indicates the bottom start position of allocating storage space from the bottom to the top of the second storage space; when the reference pointer is the first pointer, determine the second pointer as the target pointer and determine the allocation direction indicated by the second pointer as the current allocation direction; when the reference pointer is the second pointer, determine the first pointer as the target pointer and determine the allocation direction indicated by the first pointer as the current allocation direction.

[0162] In an exemplary embodiment, the data processing unit is configured to: allocate the target storage space in the second storage space from the starting position indicated by the target pointer according to the current allocation direction for the output data of the target network layer; and update the target pointer according to the ending position of the target storage space.

[0163] In one exemplary embodiment, the processor is further configured to:

[0164] After allocating target storage space for the output data of the target network layer in the second storage space according to the current allocation direction, the remaining usage counts corresponding to the target storage space are recorded, wherein the initial value of the remaining usage counts is the number of network layers that use the output data of the target network layer as input data.

[0165] If the output data of the target network layer is used as input data, the remaining number of uses will be reduced by 1.

[0166] If the remaining usage count is 0, release the target storage space and update the target pointer.

[0167] In one exemplary embodiment, the above-mentioned data acquisition port is used for:

[0168] Determine the data attributes of the output data of the target network layer mentioned above;

[0169] When the aforementioned data attributes are used to indicate the same group of network layers that do not include the target network layer in the aforementioned network model, the current allocation direction for allocating storage space for the output data of the target network layer in the aforementioned second storage space is determined.

[0170] In one exemplary embodiment, the processor is further configured to:

[0171] After determining the data attributes of the output data of the target network layer, if the data attributes are used to indicate that the target network layer is included in the network model and the network layer in the same group has been allocated storage space, the storage space corresponding to the output data of the target network layer is determined from the storage space already allocated for the network layer group to which the network layer in the same group belongs, wherein the output data of the network layer in the same group and the output data of the target network layer are both input to the same network layer;

[0172] When the aforementioned data attributes are used to indicate that the target network layer is included in the same group of network layers in the aforementioned network model and that no storage space has been allocated to the same group of network layers, the target network layer and the same group of network layers are determined as a reference network layer group, wherein the output data of the same group of network layers and the output data of the target network layer are both input to the same network layer; the current allocation direction for allocating storage space for the output data of the reference network layer group in the aforementioned second storage space is determined; storage space for the output data of the reference network layer group is allocated in the aforementioned second storage space according to the aforementioned current allocation direction; and storage space is partitioned for each network layer included in the reference network layer group in the storage space allocated to the reference network layer group.

[0173] In one optional embodiment, an optional convolutional neural network running device is provided, which can support various operations during the operation of the convolutional neural network model, such as: convolution operation, convolution operation + activation operation, convolution operation + activation operation + pooling + SUM. Figure 11 This is a schematic diagram of a convolutional neural network operating device according to an embodiment of the present invention, such as... Figure 11 As shown, the device includes: DDR storage, SRAM_D storage, SRAM_C storage, SRAM_A storage, and convolution processing unit (Conv), activation processing unit (Activate), pooling processing unit (POOL), accumulation processing unit (ADD), etc.; it can perform various typical applications of convolutional neural network calculations.

[0174] In this device, the computing units Conv and Activate are equipped with separate SRAM areas. During the computing process, these areas are used to store static data obtained from DDR for convenient computing use; while the other input required for computing processing, dynamic data, can be obtained from the dynamic data cache SRAM_D.

[0175] For the convolution processing unit, the required weights and bias data are loaded from DDR into SRAM_C, and the corresponding input data is loaded from SRAM_D to perform convolution calculation; the output data stream flows to the activation processing unit.

[0176] For the activation processing unit, upon receiving input, it can bypass and directly output to SRAM_D, or perform calculations according to the given activation type. If static data is required, it can be loaded into its private static storage area SRAM_A. The calculation result can be output to SRAM_D, and it also supports data stream output to the next pooling processing unit.

[0177] For the pooling processing unit, after receiving the input, it can bypass and output directly to SRAM_D, or it can perform specified pooling type processing and finally output to the SRAM_D area.

[0178] For the accumulation processing unit, two inputs can be obtained from SRAM_D according to the instructions, accumulated, and then output to SRAM_D.

[0179] By combining the aforementioned functional units, convolution calculations, convolution + activation processing, convolution + activation + pooling processing, convolution + pooling processing, and the combination of the above functions with residuals, etc., can be performed. Through supporting the computation of these typical network layers, the aforementioned device can support the execution of deep learning network models.

[0180] In this optional embodiment, based on the characteristics of deep learning networks, the aforementioned device structure supporting network model operation is provided. Furthermore, considering the storage requirements of upper-layer network operations, the aforementioned SRAM_D storage space can adopt a "staggered top and bottom, with empty space in the middle" usage principle, which further improves the utilization rate of storage resources, enables the device to support a wider range of networks, and achieves a better energy efficiency ratio.

[0181] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0182] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0183] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0184] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0185] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0186] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0187] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for controlling the operation of a network model, characterized in that, include: Determine the operational data type of the target network layer among the multiple network layers included in the network model; When the running data type is a static data type, the target static data of the target network layer is obtained from the first storage space. In the running scenario of the network model, the static storage space is divided into the first storage space and the second storage space. The first storage space is used to store the static data used by the network model during operation, and the second storage space is used to store the dynamic data used and output by the network model during operation. The target network layer is run using at least the target static data; The method further includes: determining a current allocation direction for allocating storage space for the output data of the target network layer in the second storage space, wherein the current allocation direction is opposite to the previous allocation direction for allocating storage space for data in the second storage space; allocating target storage space for the output data of the target network layer in the second storage space according to the current allocation direction, wherein the target storage space is used to store the target output data output by the target network layer; determining the current allocation direction for allocating storage space for the output data of the target network layer in the second storage space includes: determining a reference pointer used in the previous allocation of storage space for data in the second storage space from a first pointer and a second pointer, wherein the first pointer is used to indicate the top start position of allocating storage space from the top to the bottom of the second storage space, and the second pointer is used to indicate the bottom start position of allocating storage space from the bottom to the top of the second storage space; if the reference pointer is the first pointer, determining the second pointer as the target pointer, and determining the allocation direction indicated by the second pointer as the current allocation direction; if the reference pointer is the second pointer, determining the first pointer as the target pointer, and determining the allocation direction indicated by the first pointer as the current allocation direction.

2. The method according to claim 1, characterized in that, The step of obtaining the target static data of the target network layer from the first storage space includes: The target operation type of the target network layer is determined from the operation types of multiple network layers that use static data; From multiple operation types and multiple storage subspaces that have corresponding relationships, determine the target storage subspace corresponding to the target operation type; Read the target static data of the target network layer from the target storage subspace.

3. The method according to claim 2, characterized in that, Determining the target storage subspace corresponding to the target operation type from multiple operation types and multiple storage subspaces with corresponding relationships includes: When the target operation type is convolution operation, the first subspace among the plurality of storage subspaces is determined as the target storage subspace; When the target operation type is an activation operation, the second subspace among the plurality of storage subspaces is determined as the target storage subspace.

4. The method according to claim 3, characterized in that, The step of reading the target static data of the target network layer from the target storage subspace includes: When the target operation type is a convolution operation, the convolution weight data and bias data of the target network layer are read from the first subspace as the target static data; When the target operation type is an activation operation, the activation coefficient data of the target network layer is read from the second subspace as the target static data.

5. The method according to claim 1, characterized in that, The step of allocating target storage space in the second storage space for the output data of the target network layer according to the current allocation direction includes: In the second storage space, starting from the starting position indicated by the target pointer, the target storage space is allocated for the output data of the target network layer according to the current allocation direction; The target pointer is updated based on the end position of the target storage space.

6. The method according to claim 1, characterized in that, After allocating target storage space for the output data of the target network layer in the second storage space according to the current allocation direction, the method further includes: Record the remaining number of times the target storage space is used, wherein the initial value of the remaining number of times is the number of network layers that use the output data of the target network layer as input data; If the output data of the target network layer is used as input data, the remaining number of uses is decremented by 1; If the remaining usage count is 0, release the target storage space and update the target pointer.

7. The method according to claim 1, characterized in that, Determining the current allocation direction for allocating storage space in the second storage space for the output data of the target network layer includes: Determine the data attributes of the output data of the target network layer; When the data attribute is used to indicate that the target network layer is not included in the same group of network layers in the network model, the current allocation direction for allocating storage space for the output data of the target network layer in the second storage space is determined.

8. The method according to claim 7, characterized in that, After determining the data attributes of the output data of the target network layer, the method further includes: When the data attribute is used to indicate that the target network layer is included in the network model, and the same group of network layers has been allocated storage space, the storage space corresponding to the output data of the target network layer is determined from the storage space already allocated for the network layer group to which the same group of network layers belongs, wherein the output data of the same group of network layers and the output data of the target network layer are both input to the same network layer; When the data attribute indicates that the target network layer is included in the same group of network layers in the network model and the same group of network layers has not been allocated storage space, the target network layer and the same group of network layers are identified as a reference network layer group, wherein the output data of the same group of network layers and the output data of the target network layer are both input to the same network layer; the current allocation direction for allocating storage space for the output data of the reference network layer group in the second storage space is determined; storage space for the output data of the reference network layer group is allocated in the second storage space according to the current allocation direction; and storage space is partitioned for each network layer included in the reference network layer group in the storage space allocated to the reference network layer group.

9. A virtual device for controlling the operation of a network model, characterized in that, include: The first determining module is used to determine the running data type of the target network layer among the multiple network layers included in the network model; The acquisition module is used to acquire the target static data of the target network layer from the first storage space when the running data type is a static data type. In the running scenario of the network model, the static storage space is divided into the first storage space and the second storage space. The first storage space is used to store the static data used by the network model during operation, and the second storage space is used to store the dynamic data used and output by the network model during operation. The execution module is used to run the target network layer using at least the target static data; The device further includes: a second determining module, configured to determine the current allocation direction for allocating storage space for the output data of the target network layer in the second storage space, wherein the current allocation direction is opposite to the previous allocation direction for allocating storage space for data in the second storage space; The allocation module is configured to allocate target storage space in the second storage space for the output data of the target network layer according to the current allocation direction, wherein the target storage space is used to store the target output data output by the target network layer; The second determining module includes: The third determining unit is used to determine from the first pointer and the second pointer the reference pointer used in the last allocation of storage space for data in the second storage space, wherein the first pointer is used to indicate the top starting position of the allocation of storage space from the top to the bottom of the second storage space, and the second pointer is used to indicate the bottom starting position of the allocation of storage space from the bottom to the top of the second storage space. The fourth determining unit is configured to, when the reference pointer is the first pointer, determine the second pointer as the target pointer and determine the allocation direction indicated by the second pointer as the current allocation direction; The fifth determining unit is configured to, when the reference pointer is the second pointer, determine the first pointer as the target pointer and determine the allocation direction indicated by the first pointer as the current allocation direction.

10. A network model operation control device, characterized in that, include: Processor and static memory, of which, The static memory is divided into a first storage space and a second storage space. The first storage space is used to store static data of the network model during operation, and the second storage space is used to store dynamic data of the network model during operation. The network model includes multiple network layers. The processor is configured to determine the running data type of a target network layer among the plurality of network layers; if the running data type is a static data type, obtain target static data of the target network layer from the first storage space; and run the target network layer using at least the target static data. The processor includes: a data acquisition port and a data processing unit, wherein, The data acquisition port is used to determine the current allocation direction of allocating storage space for the output data of the target network layer in the second storage space, wherein the current allocation direction is opposite to the previous allocation direction of allocating storage space for data in the second storage space; The data processing unit is configured to allocate target storage space in the second storage space for the output data of the target network layer according to the current allocation direction, wherein the target storage space is used to store the target output data output by the target network layer; Determining the current allocation direction for allocating storage space for the output data of the target network layer in the second storage space includes: determining, from a first pointer and a second pointer, a reference pointer used in the previous allocation of storage space for data in the second storage space, wherein the first pointer indicates the top start position of allocating storage space from the top to the bottom of the second storage space, and the second pointer indicates the bottom start position of allocating storage space from the bottom to the top of the second storage space; if the reference pointer is the first pointer, determining the second pointer as the target pointer and determining the allocation direction indicated by the second pointer as the current allocation direction; if the reference pointer is the second pointer, determining the first pointer as the target pointer and determining the allocation direction indicated by the first pointer as the current allocation direction.

11. The device according to claim 10, characterized in that, The first storage space includes: a first subspace and a second subspace, wherein, The first subspace is used to store the weight data and bias data of network layers whose operation type is convolution. The second subspace is used to store the activation coefficient data of the network layer whose operation type is activation operation.

12. The device according to claim 11, characterized in that, The processor includes: a convolution processing unit and an activation processing unit, wherein, The convolution processing unit is used to obtain target weight data and target bias data of the target network layer whose operation type is convolution operation from the first subspace as the target static data. The activation processing unit is used to obtain the target activation coefficient data of the target network layer with the operation type of activation operation from the second subspace as the target static data.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of claims 1 to 8.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 8.

Images