Method for judging whether Conv2D operator can be in-place or not

Abstract

The invention provides a method for judging In-place of Conv2D operators, which comprises the following steps of: S1, inferring In-place conditions of common Conv2D operators by considering various parameters of Conv2D in combination with inferred data streams of bottom-layer Conv2D operators and storage size conditions of an FRAM (Frequency Random Access Memory); and S2, judging whether the Conv2D operator can be in-place or not according to the condition that the Conv2D operator meets the condition, namely, if the Conv2D operator meets the condition, determining that the Conv2D operator can be in-place, otherwise, determining that the Conv2D operator cannot be in-place. According to the method, the inference data stream of the bottom layer Conv2D operator and the storage size condition of the FRAM are combined, all parameters of Conv2D are considered, the In-place condition of the common Conv2D operator is obtained, and the reduction of the memory overhead of the bottom layer inference of the model is facilitated.

Description

Technical Field

[0001] The invention belongs to the technical field of neural network optimization, and in particular relates to a method for judging whether a Conv2D operator can be in-place. Background Art

[0002] In the prior art, the in-place operation is an in-place update operation, specifically, the output of the operator directly overwrites the input to achieve the purpose of reducing memory overhead. As one of the most commonly used operators in neural networks, the Conv2D operator can bring huge benefits by reducing its memory overhead during reasoning. However, the Conv2D operator currently does not have an in-place operation, and some Conv2D operators that meet the in-place conditions have unnecessary memory overhead. Summary of the invention

[0003] In order to solve the above problems, the purpose of this application is to: involve the In-place method of the Conv2 operator in the model, aiming to save memory during reasoning and improve the model reasoning speed.

[0004] Specifically, the present invention provides a method for determining whether a Conv2D operator can be in-place, the method comprising:

[0005] S1, combining the inference data flow of the underlying Conv2D operator and the storage size of FRAM, considering various parameters of Conv2D, and inferring the In-place conditions of the commonly used Conv2D operators; further including:

[0006] S1.1, according to the calculation principle of Conv2D, its output size o_w and o_h are calculated as shown in formula (1) and formula (2):

[0007]

[0008]

[0009] in, Indicates rounding down; Obviously, the first condition for the input to be covered by the output In-place is that the input space can accommodate the output, that is, condition 0, expressed as:

[0010] o_h≤i_h and o_w≤i_w Formula (3);

[0011] S1.2, Conv2D operation consists of multiple multiplication and accumulation operations MAC. If the shape of the output of a MAC operation at the bottom layer is 1*4, when Conv2D is inferred, the load input is first loaded into FRAM. If the size of FRAM is 16KB;

[0012] Calculate the size width current_w of the input of the current MAC, expressed as:

[0013]

[0014] current_w=3*s_w+k_w formula (5)

[0015] Calculate the size of the input of the current MAC, current_h, expressed as:

[0016]

[0017] current_h=k_h Formula (7)

[0018] The boundary condition of ic, ic is the number of input channels, expressed as formula (9):

[0019] current_h*current_w*ic≤16*1024 formula (8)

[0020]

[0021] S1.3, according to the boundary conditions, further consider the upper edge, left edge, and lower edge, and derive the In-place conditions respectively. Since the right edge satisfies the In-place condition is a necessary but not sufficient condition for the left edge to satisfy the In-place condition, that is, the condition that satisfies the In-place condition of the left edge must satisfy the In-place condition of the right edge, so the right edge is not considered;

[0022] S2, judging whether the Conv2D operator can be in-place according to the conditions met, that is, if the above conditions are met, it can be in-place, otherwise it cannot.

[0023] The step S1.3 further comprises:

[0024] 1) Consider the upper edge:

[0025] The first multiplication-accumulation MAC operation, that is, the upper left corner, has pad values ​​in both w and h directions, and obtains 1*4*oc numbers. The total number of numbers loaded into the temporary space is (h_current-p_top)*(w_current-p_left)*ic, so condition 1 must be met, expressed as:

[0026] Condition 1: if((h_current-p_top)*(w_current-p_left)*ic<1*4*oc){

[0027] in_place = false;

[0028] },

[0029] That is, it can be expressed as formula (10):

[0030] 1*4*oc≤(h_current-p_top)*(w_current-p_left)*ic formula (10);

[0031] Slide and get 1*4*oc number. The number newly loaded into the temporary space is (h_current-p_top)*(4*s_w)*ic. Therefore, condition 2 must be met, which can be expressed as:

[0032] Condition 2: if((h_current-p_top)*(4*s_w)*ic<1*4*oc){

[0033] in_place = false;

[0034] },

[0035] That is, it can be expressed as formula (11):

[0036] 1*4*oc≤(h_current-p_top)*(4*s_w)*ic formula (11);

[0037] The last MAC operation, i.e. the upper right corner, obtains 1*4*oc numbers. The number newly loaded into the temporary space is (h_current-p_top)*(4*s_w-p_right)*ic, so condition 3 must be met, expressed as:

[0038] Condition 3: if((h_current-p_top)*(4*s_w-p_right)*ic<1*4*oc){

[0039] in_place = false;

[0040] },

[0041] That is, it can be expressed as formula (12):

[0042] 1*4*oc≤(h_current-p_top)*(4*s_w-p_right)*ic formula (12);

[0043] Condition 3 includes condition 2, ignoring condition 2, that is, if condition 3 is met, condition 2 must be met; but conditions 0 and 1 still need to be considered;

[0044] 2) Consider the left edge:

[0045] The first MAC operation is the same as condition 1;

[0046] Slide, get 1*4*oc number, the number newly loaded into the temporary space is (1*s_h)*(w_current-p_left)*ic, so it is necessary to meet condition 4, expressed as:

[0047] Condition 4: if((1*s_h)*(w_current-p_left)*ic<1*4*ocr){

[0048] in_place = false;

[0049] },

[0050] That is, it can be expressed as formula (13):

[0051] 1*4*oc≤(1*s_h)*(w_current-p_left)*ic formula (13);

[0052] The last MAC operation obtains 1*4*oc numbers, and the number of new numbers loaded into the temporary space is (1*s_h-p_bottom)*(w_current-p_left)*ic, so condition 5 needs to be met, which is expressed as:

[0053] Condition 5: if((1*s_h-p_bottom)*(w_current-p_left)*ic<1*4*ocr){

[0054] in_place = false;

[0055] },

[0056] That is, it can be expressed as formula (14):

[0057] 1*4*oc≤(1*s_h-p_bottom)*(w_current-p_left)*ic formula (14);

[0058] Condition 5 includes condition 4, and condition 4 is ignored; that is, if condition 5 is satisfied, condition 4 must be satisfied; but conditions 0, 1, and 3 still need to be considered;

[0059] 3) Consider the lower edge:

[0060] The first MAC operation, i.e. the lower left corner, is the same as condition 5;

[0061] Slide and get 1*4*oc number. The number newly loaded into the temporary space is (h_current-p_bottom)*(4*s_w)*ic. Therefore, condition 6 needs to be met, which can be expressed as:

[0062] Condition 6: if((h_current-p_bottom)*(4*s_w)*ic<1*4*oc){

[0063] in_place = false;

[0064] };

[0065] That is, it can be expressed as formula (15):

[0066] 1*4*oc≤(h_current-p_bottom)*(4*s_w)*ic formula (15);

[0067] The last MAC operation, i.e. the lower right corner, gets 1*4*oc. The number newly loaded into the temporary space is (h_current-p_bottom)*(4*s_w-p_right)*ic. Therefore, condition 7 needs to be met, which is expressed as:

[0068] Condition 7: if((h_current-p_bottom)*(4*s_w-p_right)*ic<1*4*oc){

[0069] in_place = false;

[0070] };

[0071] That is, it can be expressed as formula (16):

[0072] 1*4*oc≤(h_current-p_bottom)*(4*s_w-p_right)*ic formula (16);

[0073] Condition 7 includes condition 6, and condition 6 is ignored, that is, if condition 7 is satisfied, condition 6 must be satisfied; but conditions 0, 1, 3, and 5 still need to be considered;

[0074] In summary, conditions 0, 1, 3, 5 and 7 need to be considered simultaneously.

[0075] Therefore, the advantage of the present application is that: this method combines the inference data flow of the underlying Conv2D operator and the storage size of FRAM, considers various parameters of Conv2D, and derives the In-place conditions of commonly used Conv2D operators, which helps to reduce the memory overhead of the underlying reasoning of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0077] Figure 1 It is a schematic diagram of the process of this method. DETAILED DESCRIPTION

[0078] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0079] An embodiment of the present application relates to a method for determining whether a Conv2D operator can be in-place. The method combines the inference data flow of the underlying Conv2D operator and the storage size of FRAM, considers various parameters of Conv2D, and derives the in-place conditions of commonly used Conv2D operators, aiming to reduce the memory overhead of the underlying reasoning of the Conv2D operator.

[0080] like Figure 1 As shown, the method includes:

[0081] S1, combining the inference data flow of the underlying Conv2D operator and the storage size of FRAM, considering various parameters of Conv2D, and inferring the In-place conditions of the commonly used Conv2D operators; further including:

[0082] S1.1, according to the calculation principle of Conv2D, its output size o_w and o_h are calculated as shown in formula (1) and formula (2):

[0083]

[0084]

[0085] in, Indicates rounding down; Obviously, the first condition for the input to be covered by the output In-place is that the input space can accommodate the output, that is, condition 0, expressed as:

[0086] o_h≤i_h and o_w≤i_w Formula (3);

[0087] S1.2, Conv2D operation consists of multiple multiplication and accumulation operations MAC. If the shape of the output of a MAC operation at the bottom layer is 1*4, when Conv2D is inferred, the load input is first loaded into FRAM. If the size of FRAM is 16KB;

[0088] Calculate the size width current_w of the input of the current MAC, expressed as:

[0089]

[0090] current_w=3*s_w+k_w formula (5)

[0091] Calculate the size of the input of the current MAC, current_h, expressed as:

[0092]

[0093] current_h=k_h Formula (7)

[0094] The boundary condition of ic, ic is the number of input channels, expressed as formula (9):

[0095] current_h*current_w*uc≤16*1024 formula (8)

[0096]

[0097] S1.3, according to the boundary conditions, further consider the upper edge, left edge, and lower edge, and derive the In-place conditions respectively. Since the right edge satisfies the In-place condition is a necessary but not sufficient condition for the left edge to satisfy the In-place condition, that is, the condition that satisfies the In-place condition of the left edge must satisfy the In-place condition of the right edge, so the right edge is not considered;

[0098] 1) Consider the upper edge:

[0099] The first multiplication-accumulation MAC operation, that is, the upper left corner, has pad values ​​in both w and h directions, and obtains 1*4*oc numbers. The total number of numbers loaded into the temporary space is (h_currect-p_top)*(w_current-p_left)*ic, so condition 1 must be met, expressed as:

[0100] Condition 1: if((h_current-p_top)*(w_current-p_left)*ic<1*4*oc){

[0101] in_place = false;

[0102] },

[0103] That is, it can be expressed as formula (10):

[0104] 1*4*oc≤(h_currect-p_top)*(w_current-p_left)*ic formula (10);

[0105] Slide and get 1*4*oc number. The number newly loaded into the temporary space is (h_currect-p_top)*(4*s_w)*ic. Therefore, condition 2 must be met, which can be expressed as:

[0106] Condition 2: if((h_current-p_top)*(4*s_w)*ic<1*4*oc){

[0107] in_place = false;

[0108] },

[0109] That is, it can be expressed as formula (11):

[0110] 1*4*oc≤(h_current-p_top)*(4*s_w)*ic formula (11);

[0111] The last MAC operation, i.e. the upper right corner, obtains 1*4*oc numbers. The number newly loaded into the temporary space is (h_current-p_top)*(4*s_w-p_right)*ic, so condition 3 must be met, expressed as:

[0112] Condition 3: if((h_current-p_top)*(4*s_w-p_right)*ic<1*4*oc){

[0113] in_place = false;

[0114] },

[0115] That is, it can be expressed as formula (12):

[0116] 1*4*oc≤(1*s_h-p_bottom)*(w)current-p_left)*ic formula (12);

[0117] Condition 3 includes condition 2, ignoring condition 2, that is, if condition 3 is met, condition 2 must be met; but conditions 0 and 1 still need to be considered;

[0118] 2) Consider the left edge:

[0119] The first MAC operation is the same as condition 1;

[0120] Slide, get 1*4*oc number, the number newly loaded into the temporary space is (1*s_h)*(w_current-p_left)*ic, so it is necessary to meet condition 4, expressed as:

[0121] Condition 4: if((1*s_h)*(w_current-p_left)*ic<1*4*ocr){

[0122] in_place = false;

[0123] },

[0124] That is, it can be expressed as formula (13):

[0125] 1*4*oc≤(1*s_h)*(w_current-p_left)*ic formula (13);

[0126] The last MAC operation obtains 1*4*oc numbers, and the number of new loads into the temporary space is (1*s_h-p_bpttpm)*(w_current-p_left)*ic, so condition 5 needs to be met, which is expressed as:

[0127] Condition 5: if((1*s_h-p_bottom)*(w_current-p_left)*ic<1*4*ocr){

[0128] in_place = false;

[0129] },

[0130] That is, it can be expressed as formula (14):

[0131] 1*4*oc≤(1*s_h-p_bottom)*(w)current-p_left)*ic formula (14);

[0132] Condition 5 includes condition 4, and condition 4 is ignored; that is, if condition 5 is satisfied, condition 4 must be satisfied; but conditions 0, 1, and 3 still need to be considered;

[0133] 3) Consider the lower edge:

[0134] The first MAC operation, i.e. the lower left corner, is the same as condition 5;

[0135] Slide and get 1*4*oc number. The number newly loaded into the temporary space is h_currect-p_bottom)*(4*s_w)*ic, so condition 6 needs to be met, which is expressed as:

[0136] Condition 6: if((h_current-p_bottom)*(4*s_w)*ic<1*4*oc){

[0137] in_place = false;

[0138] };

[0139] That is, it can be expressed as formula (15):

[0140] 1*4*oc≤(h_current-p_bottom)*(4*s_w)*ic formula (15);

[0141] The last MAC operation, i.e. the lower right corner, gets 1*4*oc. The number newly loaded into the temporary space is (h_current-p_bottom)*(4*x_w-p_right)*ic. Therefore, condition 7 needs to be met, which is expressed as:

[0142] Condition 7: if((h_current-p_bottom)*(4*s_w-p_right)*ic<1*4*oc){

[0143] in_place = false;

[0144] };

[0145] That is, it can be expressed as formula (16):

[0146] 1*4*oc≤(h_current-p_bottom)*(4*s_w-p_right)*ic formula (16);

[0147] Condition 7 includes condition 6, and condition 6 is ignored, that is, if condition 7 is satisfied, condition 6 must be satisfied; but conditions 0, 1, 3, and 5 still need to be considered;

[0148] The summary needs to consider conditions 0, 1, 3, 5 and 7 at the same time;

[0149] S2, judging whether the Conv2D operator can be in-place according to the conditions met, that is, if the above conditions are met, it can be in-place, otherwise it cannot.

[0150] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for determining whether a Conv2D operator can be in-place, characterized in that: The method comprises: S1, combining the inference data flow of the underlying Conv2D operator and the storage size of FRAM, considering various parameters of Conv2D, and inferring the In-place conditions of the commonly used Conv2D operators; further including: S1.1, according to the calculation principle of Conv2D, its output size o_w and o_h are calculated as shown in formula (1) and formula (2): in, Indicates rounding down; Obviously, the first condition for the input to be covered by the output In-place is that the input space can accommodate the output, that is, condition 0, expressed as: o_h≤i_h and o_w≤i_w Formula (3); S1.2, Conv2D operation consists of multiple multiplication and accumulation operations MAC. If the shape of the output of a MAC operation at the bottom layer is 1*4, when Conv2D is inferred, the load input is first loaded into FRAM. If the size of FRAM is 16KB; Calculate the size width current_w of the input of the current MAC, expressed as: current w=3*s w+kw formula (5) Calculate the size of the input of the current MAC, current_h, expressed as: current_h=k_h Formula (7) The boundary condition of ic, ic is the number of input channels, expressed as formula (9): current_h*current_w*ic≤16*1024 formula (8) S1.3, according to the boundary conditions, further consider the upper edge, left edge, and lower edge, and derive the In-place conditions respectively. Since the right edge satisfies the In-place condition is a necessary but not sufficient condition for the left edge to satisfy the In-place condition, that is, the condition that satisfies the In-place condition of the left edge must satisfy the In-place condition of the right edge, so the right edge is not considered; S2, judging whether the Conv2D operator can be in-place according to the conditions met, that is, if the above conditions are met, it can be in-place, otherwise it cannot.

2. According to claim 1, a method for determining whether a Conv2D operator can be in-place, characterized in that: The step S1.3 further comprises: 1) Consider the upper edge: The first multiplication-accumulation MAC operation, that is, the upper left corner, has pad values ​​in both w and h directions, and obtains 1*4*oc numbers. The total number of numbers loaded into the temporary space is (h_current-p_top)*(w_current-p_left)*ic, so condition 1 must be met, expressed as: Condition 1: if((h_current-p_top)*(w_current-p_left)*ic<1*4*oc){In_place=false;}, That is, it can be expressed as formula (10): 1*4*oc≤(h_current-p_top)*(w_current-p_left)*ic formula (10); Slide and get 1*4*oc number. The number newly loaded into the temporary space is (h_current-p_top)*(4*s_w)*ic. Therefore, condition 2 must be met, which can be expressed as: Condition 2: if((h_current-p_top)*(4*s_w)*ic<1*4*oc){In_place=false;}, That is, it can be expressed as formula (11): 1*4*oc≤(h_current-p_top)*(4*s_w)*ic formula (11); The last MAC operation, i.e. the upper right corner, obtains 1*4*oc numbers, and the new numbers loaded into the temporary space are (h_current-p_top)*(4*s_w-p_right)*ic, so condition 3 must be met, expressed as: Condition 3: if((h_current-p_top)*(4*s_w-p_right)*ic<1*4*oc){In_place=false;}, That is, it can be expressed as formula (12): 1*4*oc≤(h_current-p_top)*(4*s_w-p_right)*ic formula (12); Condition 3 includes condition 2, ignoring condition 2, that is, if condition 3 is met, condition 2 must be met; but conditions 0 and 1 still need to be considered; 2) Consider the left edge: The first MAC operation is the same as condition 1; Slide, get 1*4*oc number, the number newly loaded into the temporary space is (1*s_h)*(w_current-p_left)*ic, so it is necessary to meet condition 4, expressed as: Condition 4: if((1*s_h)*(w_current-p_left)*ic<1*4*ocr){In_place=false;}, That is, it can be expressed as formula (13): 1*4*oc≤(1*s_h)*(w_current-p_left)*ic formula (13); The last MAC operation obtains 1*4*oc numbers, and the number of new numbers loaded into the temporary space is (1*s_h-p_bottom)*(w_current-p_left)*ic, so condition 5 needs to be met, which is expressed as: Condition 5: if((1*s_h-p_bottom)*(w_current-p_left)*ic<1*4*ocr){In_place=false;}, That is, it can be expressed as formula (14): 1*4*oc≤(1*s_h-p_bottom)*(w_current-p_left)*ic formula (14); Condition 5 includes condition 4, and condition 4 is ignored; that is, if condition 5 is satisfied, condition 4 must be satisfied; but conditions 0, 1, and 3 still need to be considered; 3) Consider the lower edge: The first MAC operation, i.e. the lower left corner, is the same as condition 5; Slide and get 1*4*oc number. The number newly loaded into the temporary space is (h_current-p_bottom)*(4*s_w)*ic. Therefore, condition 6 needs to be met, which can be expressed as: Condition 6: if((h_current-p_bottom)*(4*s_w)*ic<1*4*oc){In_place=false;}; That is, it can be expressed as formula (15): 1*4*oc≤(h_current-p_bottom)*(4*s_w)*ic formula (15); The last MAC operation, i.e. the lower right corner, gets 1*4*oc. The number newly loaded into the temporary space is (h_current-p_bottom)*(4*s_w-p_right)*ic. Therefore, condition 7 needs to be met, which is expressed as: Condition 7: if((h_current-p_bottom)*(4*s_w-p_right)*ic<1*4*oc){In_place=false;}; That is, it can be expressed as formula (16): 1*4*oc≤(h_current-p_bottom)*(4*s_w-p_right)*ic formula (16); Condition 7 includes condition 6, and condition 6 is ignored, that is, if condition 7 is satisfied, condition 6 must be satisfied; but conditions 0, condition 1, condition 3, and condition 5 still need to be considered; In summary, conditions 0, 1, 3, 5 and 7 need to be considered simultaneously.

Images