Convolutional neural network weight mapping and pixel-level reasoning simulation method based on memory computing architecture of memristor

Abstract

The invention provides a convolutional neural network weight mapping and pixel-level reasoning simulation method based on a memory computing architecture of a memristor, which comprises the following steps of: for any convolutional neural network input, abstracting all weight parameters of the input convolutional neural network into rectangular models with a plurality of sizes by using a mapping method, and reasonably segmenting the rectangular models to obtain a pixel-level reasoning simulation model; storing in a memristor array group; a neural network reasoning process is divided into a plurality of periods by using a simulation method, and the serial number and the row and column number of a memristor array working in each period, the output pixel value of a network layer and the scale of pixels needing to be cached and allowed to be released are simulated, so that pixel-level network full-pipeline simulation is realized. The method is suitable for any convolutional neural network, provides an important basis for hardware implementation such as on-chip cache capacity design and inter-memristor array feature map pipeline design, and has a wide application prospect.

Description

Technical Field

[0001] The present invention relates to the technical field of memristor in-memory computing and the field of convolutional neural network hardware simulation, and in particular to a method for convolutional neural network weight mapping and pixel-level reasoning simulation based on a memristor in-memory computing architecture. Background Art

[0002] We are now in the post-Moore era of integrated circuit development, and algorithm fields such as big data, artificial intelligence, and the Internet of Things are developing rapidly, gradually showing a trend of hardware being left behind by software. Traditional neural network acceleration chips will have serious storage wall problems under the von Neumann architecture, and the problem becomes more and more serious as the network scale increases. The storage and computing framework based on memristor arrays has become an effective means to break through the storage wall problem. This new computing architecture can greatly reduce the amount of neural network parameters to be moved, thereby improving the chip's reasoning speed.

[0003] Most of the common integrated computing chips currently only deploy multiplication and addition operations on the memristor array. The hardware implementation of complete network reasoning will involve a large amount of off-chip communication, which seriously affects the computing speed and efficiency. Even if the data flow of the entire network is completed on-chip, most of them serve customized specific networks and face the problem of poor versatility. There is still no good solution for how to map any convolutional neural network to a general memristor array chip. Therefore, it is of great significance to study a new method of convolutional neural network weight mapping and pixel-level reasoning simulation based on the memristor in-memory computing architecture. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention proposes for the first time a method for convolutional neural network weight mapping and pixel-level reasoning simulation based on a memristor in-memory computing architecture. For any convolutional neural network input, it can be mapped to a universal memristor array chip, and the full-pipeline simulation of the network for pixel-level reasoning can be realized, providing an important basis for hardware implementation such as on-chip cache capacity design and feature map pipeline design between memristor arrays.

[0005] The technical solution of the present invention is as follows:

[0006] A convolutional neural network weight mapping method based on a memristor in-memory computing architecture is characterized in that all weights of the input convolutional neural network are mapped into rectangular models of several sizes, and the rectangular models are reasonably divided and stored in a memristor array group. The specific steps include:

[0007] 1) First read the neural network model, assuming that there are LT layers in total, where the number of convolutional layers is L layers, and the fully connected layer is equivalent to the convolutional layer, which is included in it, and there are 0 <L≤LT;

[0008] 2) The weight parameters of each convolutional layer are abstracted into a rectangular model. The single rectangular model is represented by the matrix M i Represents, where i is the layer number of the convolutional layer, 0≤i<L, the length and width of the rectangular model of a single convolutional layer, that is, the matrix M i The number of rows H i and the number of columns W i , the expressions are:

[0009] H i =K i 2 ×Ci i

[0010] W i =Co i

[0011] Among them, K i is the side length of the convolution kernel of the i-th convolution layer. The convolution kernel is square by default. i is the number of channels of the input image of the i-th convolutional layer, Co i is the number of channels of the output image of the i-th convolutional layer; the set of rectangular models of a single convolutional layer is denoted as S i ={(H i ,W i )},S i The number of elements is n i ;

[0012] 3) Determine the hardware configuration for deploying this network, including the number of memristor arrays N and the number of rows R of a single memristor array. j and the number of columns C j , where j is the serial number of the memristor array, 0≤j<N, requiring the storage capacity of the memristor array to be greater than or equal to the total network weight parameter. The storage capacity of a single memristor array is represented by its area R j ×C j ;

[0013] 4) Map the convolutional layer weights of layer L to N memristor arrays; for the i-th convolutional layer, the rectangular model set is denoted as S i ={(H i ,W i )},S i The number of elements is n i , when initialized, n i are all 1, 0≤i<L; for the jth memristor array, the remaining area of ​​the memristor array j is initialized to R j ×C j ; Starting the convolution layer weight mapping operation from the convolution layer with layer number i=0 and the memristor array with sequence number j=0 until the convolution layer weight mapping of all L layers is completed, the steps include:

[0014] 4-1) The number of the convolutional layer to be placed is i, and the size of i is determined. When i is less than L, go to step 4-2), otherwise, the mapping operation ends;

[0015] 4-2) From the rectangular model set S of the current i-th convolutional layer i Take out a rectangular element s, if the remaining area of ​​the memristor array j to be placed can accommodate s, go to step 4-3), otherwise go to step 4-4);

[0016] 4-3) Fill the rectangular element s in step 4-2) tightly into the remaining area of ​​the memristor array j, j = (j+1) mod N; determine S i , if S i It is an empty set, that is, the rectangular model set S of the current i-th convolutional layer i All mappings have been completed, then i = i + 1, if S i If it is not an empty set, i remains unchanged; return to step 4-1);

[0017] 4-4) Split the rectangular element s in half, denoted as s1 i and s2 i , and store both in the set S i , return to step 4-2);

[0018] 5) When the mapping operation process is finished, the memristor array serial number set O of each element placed in the rectangular model set of each layer is recorded. i , 0≤i<L, the rectangular model set S of each layer i and the serial number set O of the memristor array placed in each layer i The number of elements is the same; the set O i That is the mapping result of the full convolutional layer.

[0019] The present invention also proposes a method for performing pixel-level reasoning simulation based on the result obtained by the convolutional neural network weight mapping method based on the memristor in-memory computing architecture, characterized in that the mapping result of the full convolution layer is i , 0≤i<L, conduct pixel-level reasoning simulation of convolutional neural network, divide the neural network reasoning into several cycles, simulate the serial number and number of rows and columns of the memristor array working in each cycle, the output pixel value of the network layer, and the pixel scale that needs to be cached and can be released. The specific steps include:

[0020] S1) Initialize the current cycle number k to 0;

[0021] S2) Determine whether the accumulated pixels of the input images of all convolutional layers have reached or exceeded the area covered by the current convolution kernel. If so, and the convolutional layer has not calculated all the pixels, then the convolutional layer meets the working conditions in period k. If not, then the convolutional layer does not meet the working conditions in period k. Based on this, count the layer numbers of all convolutional layers that meet the working conditions in period k, and use the set P k Indicates; if the number of convolutional layers that meet the working conditions is 0, that is The simulation process ends if the number of convolution layers that meet the working conditions is Then proceed to step 3);

[0022] S3) Let Q be the set of all convolutional layers whose cycle k satisfies the working conditions and can work without conflict. k , which is an empty set at initialization. According to the principle of low layer number priority, the layers are sorted from P in ascending order. k Take out an element p and determine the array number set O placed on this layer p With Q k The array sequence number set O where all elements are placed q Whether the union of has the same elements, q∈Q k If yes, it means that the layer number p occupies at least one of the same arrays as the lower layer number that is retained, and the layer is changed to be unavailable for working in the current cycle k; if no, p is added to the set Q k In; until P k All elements in have been taken out and judged, and finally the layer numbers Q of all convolutional layers that meet the working conditions and can work without conflict in period k are obtained. k .

[0023] S4) Calculate Q k The new pixel values ​​output by all convolutional layers in cycle k are stored in the cache, and then the convolution kernel window is moved to the next position;

[0024] S5) Calculate Q k All the layer numbers that all the convolutional layers in the network depend on, and calculate the number of pixels that these dependent layers are allowed to release from the cache, release these pixels from the cache space, and update the cache parameter amount of period k;

[0025] S6) k = k + 1; proceed to step 2).

[0026] The present invention proposes a method for weight mapping and pixel-level reasoning simulation of a convolutional neural network based on a memristor in-memory computing architecture. For any convolutional neural network input, a mapping method is used to abstract all weight parameters of the input convolutional neural network into rectangular models of several sizes, and after reasonable segmentation, they are stored in a memristor array group; a simulation method is used to divide the neural network reasoning process into several cycles, and the sequence number and number of rows and columns of the memristor array working in each cycle, the output pixel value of the network layer, the pixel scale that needs to be cached and allowed to be released are simulated, and a pixel-level full-pipeline simulation of the network is realized. The present invention is applicable to any convolutional neural network, and provides an important basis for hardware implementations such as on-chip cache capacity design and feature map pipeline design between memristor arrays. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Flow chart of convolutional neural network weight mapping in an embodiment of the present invention;

[0028] Figure 2 It is a flowchart of the pixel-level reasoning simulation of the convolutional neural network in an embodiment of the present invention;

[0029] Figure 3 This is an example diagram of judging whether the convolution layer is working in the pixel-level reasoning simulation method in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to more clearly illustrate the purpose, technical solutions and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings. The description here is only used to explain the present invention and is not used to limit the present invention.

[0031] The present invention proposes a method for convolutional neural network weight mapping and pixel-level reasoning simulation based on a memristor in-memory computing architecture, which provides an important basis for hardware implementations such as on-chip cache capacity design and feature map pipeline design between memristor arrays.

[0032] Currently, only separate multiplication and addition operations are deployed on the memristor array. The network-level weight deployment and pipeline simulation are not yet mature. Accelerating the neural network requires frequent off-chip communication, which is inefficient. Therefore, we propose a method for convolutional neural network weight mapping and pixel-level reasoning simulation based on the memristor in-memory computing architecture. The specific steps of the convolutional neural network weight mapping method based on the memristor in-memory computing architecture are as follows:

[0033] 1) First read the neural network model. Suppose there are LT layers in total, LT = 100, where the number of convolutional layers is L, L = 80. The fully connected layer can also be equivalent to the convolutional layer and counted in it, with 0 <L≤LT。

[0034] 2) The weight parameters of each convolutional layer are abstracted into a rectangular model. The single rectangular model is represented by the matrix Mi Indicates, where i is the layer number of the current convolutional layer, 0≤i<L. The length and width of the rectangular model of a single convolutional layer, that is, the matrix M i The number of rows H i and the number of columns W i , the expressions are:

[0035] H i =K i 2 ×Ci i

[0036] W i =Co i

[0037] Among them, K i is the side length of the convolution kernel of the i-th convolution layer, and the convolution kernel is square by default. i is the number of channels of the input image to the i-th convolutional layer. i is the number of channels of the output image of the i-th convolutional layer; the set of rectangular models of a single convolutional layer is denoted as S i ={(H i ,W i )},S i The number of elements is n i .

[0038] 3) Determine the hardware configuration for deploying this network, including the number of memristor arrays N, N = 32, and the number of rows of a single memristor array R j and the number of columns C j , where j is the serial number of the current memristor array, 0≤j<N, requiring the storage capacity of the memristor array to be greater than or equal to the total network weight parameter, and preferably leaving a 5% margin. It is conceivable that the size of all memristor arrays is the same, all 256*512, that is, R j =256, C j =512,0≤j<N.

[0039] 4) Next, we start mapping the convolutional layer weights of layer L to the memristor array. For the i-th convolutional layer, the rectangular model set is denoted as S i ={(H i ,W i )},S i The number of elements is n i , when initialized, n i are all 1, 0≤i<L; for the jth memristor array, the remaining area of ​​the memristor array j is initialized to R j ×C j ; The convolution layer weight mapping operation starts from the convolution layer with layer number i=0 and the memristor array with sequence number j=0 until the convolution layer weight mapping of all L layers is completed; the flow chart is as follows Figure 1 The steps include:

[0040] 4-1) The number of the convolutional layer to be placed is i, and the size of i is determined. When i is less than L, go to step 4-2), otherwise, the mapping operation ends;

[0041] 4-2) From the rectangular model set S of the current i-th convolutional layer i Take out a rectangular element s, if the remaining area of ​​the memristor array j to be placed can accommodate s, go to step 4-3), otherwise go to step 4-4);

[0042] 4-3) Fill the rectangular element s in step 4-2) tightly into the remaining area of ​​the memristor array j, j = (j+1) mod N; determine S i , if S i It is an empty set, that is, the rectangular model set S of the current i-th convolutional layer i All mappings have been completed, then i = i + 1, if S i If it is not an empty set, i remains unchanged; return to step 4-1);

[0043] 4-4) Split the rectangular element s in half, denoted as s1 i and s2 i , and store both in the set S i , return to step 4-2);

[0044] 5) When the mapping operation process is finished, the memristor array serial number set O of each element placed in the rectangular model set of each layer is recorded. i , 0≤i<L, the rectangular model set S of each layer i and the serial number set O of the memristor array placed in each layer i The number of elements is the same; the set O i That is the mapping result of the full convolutional layer.

[0045] Taking the zeroth convolutional layer as an example, the number of the currently placed convolutional layer i is 0, and the set of rectangular models of the zeroth convolutional layer is initialized to S0 = {(H0, W0)}, H0 = 512, W0 = 512, and the number of S0 elements is recorded as n0. Obviously, when initialized, n0 is 1, and the number of the currently placed memristor array j is initialized to 0.

[0046] a) The convolutional layer number i=0 is currently being placed, and the size of i is determined. i is less than L;

[0047] b) Take out a rectangular element s, i.e. (512, 512), from the rectangular model set S0 of the zeroth convolutional layer. The memristor array 0 to be placed is empty, and the remaining area is 256*512, which cannot accommodate the rectangular element s.

[0048] c) Split the rectangular element s in half, preferably vertically, because it cannot fit horizontally, so it can only be split in half horizontally into two 256*512 sub-rectangles, denoted as s1 and s2, and both are stored in the set S0;

[0049] d) Take out a rectangular element s from the rectangular model set S0 of the 0th convolutional layer, that is, s1(256,512), and there is one element s2(256,512) left in S. The serial number j of the memristor array to be placed is 0, and the remaining area can be placed;

[0050] e) Place the rectangular element s1 into the memristor array 0, j = (j+1)modN = (0+1)mod 32 = 1. Since S0 is not an empty set, another rectangular element s needs to be taken out from S0, i.e., s2(256,512). The serial number j of the memristor array to be placed is 1, and the remaining area can be accommodated. Then place the rectangular element s into the memristor array 1, j = (j+1)modN = (1+1)mod32 = 2. Since S0 is already an empty set, the weight mapping from the 0th convolutional layer to the memristor array has been completed, and the next convolutional layer is considered, i = i+1.

[0051] f) Continue mapping the next convolutional layer until i = L, then the weight mapping process ends, completing the weight mapping of the L-layer convolutional layer to the memristor array, and obtaining the mapping result O of the full convolutional layer. i , 0≤i<L, that is, the set of serial numbers of the memristor arrays where each element is placed in each layer of the rectangular model set.

[0052] After completing the mapping process of all convolutional layers to arrays, based on O i , 0≤i<L, design a pixel-level reasoning simulation method, divide the neural network reasoning into several cycles, simulate the memristor array serial number and number of rows and columns, the output pixel value of the network layer, the pixel scale that needs to be cached and allowed to be released in each cycle, and the complete example process is as follows Figure 2 As shown, the specific steps include:

[0053] S1) Initialize the current cycle number k to 0.

[0054] S2) Determine whether the accumulated pixels of all convolutional layer input images have reached or exceeded the area covered by the current convolution kernel. If so, and the convolutional layer has not calculated all the pixels, then the convolutional layer meets the working conditions in cycle k. If not, then the convolutional layer does not meet the working conditions in cycle k. Figure 3 For example, the pixels in the outer gray frame are the padding area of ​​each layer of the input image, the yellow area is the cumulative pixels of the input image, and the red frame range is the area that the 2*2 convolution kernel has passed through. In the area where the convolution kernel currently stays, Figure 3The left image has not accumulated enough pixels, so the current cycle does not work. Figure 3 There are enough graphs accumulated, so the current cycle meets the working conditions. Based on this, the layer numbers of all convolutional layers that meet the working conditions in cycle k are counted and the set P is used. k If the number of convolutional layers that meet the working conditions is 0, the simulation process ends, otherwise it goes to step 3).

[0055] S3) Let Q be the set of all convolutional layers whose cycle k satisfies the working conditions and can work without conflict. k , which is an empty set at initialization. According to the principle of low layer number priority, the layers are sorted from P in ascending order. k Take out an element p and determine the array number set O placed on this layer p With Q k The array sequence number set O where all elements are placed q Whether the union of has the same elements, q∈Q k If yes, it means that the layer number p occupies at least one of the same arrays as the lower layer number that is retained, and the layer is changed to be unavailable for working in the current cycle k; if no, p is added to the set Q k In; until P k All elements in have been taken out and judged, and finally the layer numbers Q of all convolutional layers that meet the working conditions and can work without conflict in period k are obtained. k Assume that the set P k The elements of are convolutional layer 0, convolutional layer 1 and convolutional layer 2, that is, P k ={0,1,2}. The memristor arrays placed in convolutional layer 0 are numbered 0 and 1, the memristor arrays placed in convolutional layer 1 are numbered 2 and 3, and the memristor arrays placed in convolutional layer 2 are numbered 1 and 4. That is, O0 = {0,1}, O1 = {2,3}, O2 = {1,4}. Initialization According to the principle of low layer number priority, convolutional layer 0 is checked first. At this time, Q k It is empty, obviously there is no conflict, add 0 to Q k . Next, check the convolutional layer 1, O1 = {2, 3}, Q k If all elements in are 0, then Q k The union of the array sequence numbers of all elements in is {0,1}, which has no common elements with O1={2,3}, so 1 is added to Q k Next, check the convolutional layer 2, O2 = {1, 4}, Q k If all elements in are 0 and 1, then Q kThe union of the array sequence numbers of all elements in is {0,1,2,3}, which has the same element 1 as O2={1,4}, indicating that convolution layer 2 and the lower convolution that has passed the inspection need to occupy array 1, resulting in a conflict. Therefore, convolution layer 2 is changed to cycle k and does not work, and Q is not added. k . Thus, we get the set Q of all convolutional layer numbers that do not conflict with each other. k ={0,1}.

[0056] S4) Calculate Q k All convolutional layers in cycle k output new pixel values ​​and store them in the cache, then move the convolution kernel window to the next position.

[0057] S5) Calculate Q k All the layer numbers that all the convolutional layers in depend on, and calculate the number of pixels that these dependent layers are allowed to release from the cache, release these pixels from the cache space, and update the cache parameter amount of period k.

[0058] S6) k = k + 1. Go to step 2).

[0059] Furthermore, the method for calculating the number of pixels allowed to be released from the cache in each layer in each cycle in step 5) above is specifically as follows:

[0060] S5-1) Let the current layer number be l, l=0, count all subsequent layers of layer 0, represented by set Y, and record the number of its elements as y, that is, l is the layer number of the dependent layer of all elements in Y. Each element in Y has a pixel value that has been accumulated after the dependent layer is used up, represented by set U, obviously the number of its elements is equal to y, and initialized to 0. It is worth noting that the order of each element in set Y and U must correspond one to one. Let Y of layer 0 be {1, 2, 3}, U be {0, 0, 0}, and y=3.

[0061] S5-2) In a period K, if Q K The elements in include 2 and 3, that is, layers 2 and 3 can work without conflict in period K. Assuming that U of layer 0 in period K is {2, 0, 0}, after period K ends, since layers 2 and 3 have worked once, U of their dependent layer layer 0 will be updated to {2, 1, 1}.

[0062] S5-3) At the end of period K, i.e. Q K All elements in have completed step 5-2), indicating that the cumulative pixel values ​​used by all layers have been updated. At this time, the minimum value of each element of the set U of all layers can be calculated, which represents the oldest number of pixels that the layer is allowed to release in the current cycle. If it is 0, the layer does not release any pixels in cycle K. Taking layer 0 as an example, at this time, U of layer 0 is {2, 1, 1}, and the minimum value is 1, so layer 0 is allowed to release the oldest 1 pixel value.

[0063] The present invention proposes a method for convolutional neural network weight mapping and pixel-level reasoning simulation based on a memristor in-memory computing architecture, which provides an important basis for hardware implementations such as on-chip cache capacity design and feature map pipeline design between memristor arrays. In the post-Moore era, it provides important support for breaking through the traditional von Neumann architecture and building a new type of neural network acceleration chip.

[0064] Finally, it should be noted that the purpose of publishing the embodiments is to help further understand the present invention, but those skilled in the art can understand that various substitutions and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the contents disclosed in the embodiments, and the scope of protection claimed by the present invention shall be subject to the scope defined in the claims.

Claims

1. A convolutional neural network weight mapping method based on a memristor in-memory computing architecture, characterized in that: Map all weights of the input convolutional neural network into rectangular models of several sizes, and divide them reasonably and store them in the memristor array group. The specific steps include: 1) First read the neural network model, assuming that there are LT layers in total, where the number of convolutional layers is L layers, and the fully connected layer is equivalent to the convolutional layer, which is included in it, and there are 0 <L≤LT; 2) The weight parameters of each convolutional layer are abstracted into a rectangular model. The single rectangular model is represented by the matrix M i Represents, where i is the layer number of the convolutional layer, 0≤i<L, the length and width of the rectangular model of a single convolutional layer, that is, the matrix M i The number of rows H i and the number of columns W i , the expressions are: H i =K i 2 ×Ci i IN i =What i Among them, K i is the side length of the convolution kernel of the i-th convolution layer. The convolution kernel is square by default. i is the number of channels of the input image of the i-th convolutional layer, Co i is the number of channels of the output image of the i-th convolutional layer; the set of rectangular models of a single convolutional layer is denoted as S i ={(H i ,W i )},S i The number of elements is n i ; 3) Determine the hardware configuration for deploying this network, including the number of memristor arrays N and the number of rows R of a single memristor array. j and the number of columns C j , where j is the serial number of the memristor array, 0≤j<N, requiring the storage capacity of the memristor array to be greater than or equal to the total network weight parameter. The storage capacity of a single memristor array is represented by its area R j ×C j ; 4) Map the convolutional layer weights of layer L to N memristor arrays; for the i-th convolutional layer, the rectangular model set is denoted as S i ={(H i ,W i )},S i The number of elements is n i , when initialized, n i are all 1, 0≤i<L; for the jth memristor array, the remaining area of ​​the memristor array j is initialized to R j ×C j ; Starting the convolution layer weight mapping operation from the convolution layer with layer number i=0 and the memristor array with sequence number j=0 until the convolution layer weight mapping of all L layers is completed, the steps include: 4-1) The number of the convolutional layer to be placed is i, and the size of i is determined. When i is less than L, go to step 4-2), otherwise, the mapping operation ends; 4-2) From the rectangular model set S of the current i-th convolutional layer i Take out a rectangular element s, if the remaining area of ​​the memristor array j to be placed can accommodate s, go to step 4-3), otherwise go to step 4-4); 4-3) Fill the rectangular element s in step 4-2) tightly into the remaining area of ​​the memristor array j, j = (j+1) mod N; determine S i , if S i It is an empty set, that is, the rectangular model set S of the current i-th convolutional layer i The mapping has been completed, then i = i + 1, if S i If it is not an empty set, i remains unchanged; return to step 4-1); 4-4) Split the rectangular element s in half, denoted as s1 i and s2 i , and store both in the set S i , return to step 4-2); 5) When the mapping operation process is finished, the memristor array serial number set O of each element placed in the rectangular model set of each layer is recorded. i , 0≤i<L, the rectangular model set S of each layer i and the serial number set O of the memristor array placed in each layer i The number of elements is the same; the set O i That is the mapping result of the full convolutional layer.

2. A method for performing pixel-level reasoning simulation on the result obtained by the method of claim 1, characterized in that: Mapping result O based on full convolutional layer i , 0≤i<L, conduct pixel-level reasoning simulation of convolutional neural network, divide the neural network reasoning into several cycles, simulate the serial number and number of rows and columns of the memristor array working in each cycle, the output pixel value of the network layer, and the pixel scale that needs to be cached and can be released. The specific steps include: S1) Initialize the current cycle number k to 0; S2) Determine whether the accumulated pixels of the input images of all convolutional layers have reached or exceeded the area covered by the current convolution kernel. If so, and the convolutional layer has not calculated all the pixels, then the convolutional layer meets the working conditions in period k. If not, then the convolutional layer does not meet the working conditions in period k. Based on this, count the layer numbers of all convolutional layers that meet the working conditions in period k, and use the set P k Indicates; if the number of convolutional layers that meet the working conditions is 0, that is The simulation process ends if the number of convolution layers that meet the working conditions is Then proceed to step 3); S3) Let Q be the set of all convolutional layers whose cycle k satisfies the working conditions and can work without conflict. k , which is an empty set at the time of initialization; according to the principle of low layer number priority, the layers are sorted from P in ascending order. k Take out an element p and determine the array number set O placed on this layer p With Q k The array sequence number set O where all elements are placed q Whether the union of has the same elements, q∈Q k If yes, it means that the layer number p occupies at least one of the same arrays as the lower layer number that is retained, and the layer is changed to be unavailable for working in the current cycle k; if no, p is added to the set Q k In; until P k All elements in have been taken out and judged, and finally the layer numbers Q of all convolutional layers that meet the working conditions and can work without conflict in period k are obtained. k ; S4) Calculate Q k The new pixel values ​​output by all convolutional layers in cycle k are stored in the cache, and then the convolution kernel window is moved to the next position; S5) Calculate Q k All the layer numbers that all the convolutional layers in the network depend on, and calculate the number of pixels that these dependent layers are allowed to release from the cache, release these pixels from the cache space, and update the cache parameter amount of period k; S6) k = k + 1; proceed to step 2).

3. The method according to claim 2, characterized in that The method of step 5) calculating the number of pixels allowed to be released from the cache at each dependent layer in each cycle, specifically comprises the following steps: S5-1) The current layer number is recorded as l, and all subsequent layers of this layer are counted and represented by a set Y, whose number of elements is recorded as y, i.e. l is the dependency layer number of all elements in Y. Each element in Y has a pixel value that has been accumulated to use up the dependency layer. It is represented by the set U, whose number of elements is equal to y and is initialized to 0. The order of the elements in the sets Y and U must correspond one to one. S5-2) In any period K, if a convolutional layer number ∈ Q K , that is, the convolutional layer can work without conflict. Let the convolutional layer be q. Then all dependent layers of the convolutional layer q have a subsequent layer set Y. r And the accumulated pixel value set U r , q∈Y r , r is greater than or equal to 0, and r is less than the number of all dependent layers of the convolutional layer q; r In the set Y, q r The value of the element at the corresponding position is increased by 1, which means that the number of pixels used by the convolutional layer q in all subsequent layers of the dependent layer has increased by 1; S5-3) At the end of any period K, i.e. Q K All elements in have completed step 5-2), which means that the cumulative pixel values ​​used by all layers have been updated; at this time, the minimum value of each element of the set U of all layers is obtained, which represents the oldest number of pixels allowed to be released by this layer in the current cycle. If it is 0, the layer does not release any pixels in cycle K.

Images