Operation processing method, device, device for operation processing and storage medium

Abstract

The embodiment of the present invention provides an operation processing method, device, device for operation processing and storage medium, which are applied to a convolutional neural network model, wherein the method includes: extracting a loop structure to be converted from a pooling operator of the convolutional neural network model; determining the operands involved in the loop calculation in the loop structure; defining vector data matching the data type of the operand, wherein each vector data contains a preset number of operands; converting the loop body statement in the loop structure that performs loop calculation on the operands into a single instruction multiple data stream SIMD instruction that performs loop calculation on the vector data; executing the converted loop structure, and in at least one loop calculation, performing loop calculation on the preset number of operands in parallel by executing the SIMD instruction. The embodiment of the present invention can optimize the loop structure, reduce the number of loops, and thereby improve the performance of the deep learning model.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to an operation processing method, an operation processing device, a device for operation processing, and a storage medium. Background Art

[0002] Deep learning is a hot topic in the field of artificial intelligence and is widely used in search technology, graphics and image processing, data mining, machine learning, machine translation, natural language processing, etc. Typical deep learning models include CNN (Convolutional Neural Networks), which are mainly used in graphics and image processing.

[0003] CNN-based deep learning models usually use a large number of loop structures. When the number of loops is large, not only is the number of loops large, making the loop operation more time-consuming, but a large number of loop calculations require multiple memory accesses, which seriously affects the performance of the deep learning model. Summary of the invention

[0004] In view of the above problems, an embodiment of the present invention is proposed to provide an operation processing method that overcomes the above problems or at least partially solves the above problems, so as to optimize the loop structure in the convolutional neural network model, reduce the number of loops, and thereby improve the performance of the deep learning model.

[0005] Correspondingly, an embodiment of the present invention further provides an operation processing device, a device for operation processing, and a storage medium to ensure the implementation and application of the above method.

[0006] In order to solve the above problems, an embodiment of the present invention discloses an operation processing method, which is applied to a convolutional neural network model. The method includes:

[0007] Extracting a loop structure to be converted from a pooling operator of the convolutional neural network model;

[0008] Determine the number of operations involved in the loop calculation in the loop structure;

[0009] Defining vector data matching the data type of the operands, wherein each vector data includes a preset number of operands;

[0010] Converting a loop body statement in the loop structure that performs loop calculation on the operand into a single instruction multiple data (SIMD) instruction that performs loop calculation on the vector data;

[0011] The converted loop structure is executed, and in at least one loop calculation, the loop calculation is performed in parallel on the preset number of operands by executing the SIMD instruction.

[0012] The embodiment of the present invention further discloses a memory access device, which is applied to a convolutional neural network model, and the device includes:

[0013] A structure determination module, used to extract the cyclic structure to be converted from the pooling operator of the convolutional neural network model;

[0014] A data determination module, used for determining the number of operands involved in loop calculation in a loop structure;

[0015] A vector definition module, used for defining vector data matching the data type of the operands, wherein each vector data contains a preset number of operands;

[0016] An instruction conversion module, used for converting a loop body statement in the loop structure for performing loop calculation on the operand into a single instruction multiple data (SIMD) instruction for performing loop calculation on the vector data;

[0017] The instruction execution module is used to execute the converted loop structure, and in at least one loop calculation, perform loop calculation on the preset number of operands in parallel by executing the SIMD instruction.

[0018] The embodiment of the present invention also discloses a device for operation processing, which is applied to a convolutional neural network model. The device includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations:

[0019] Extracting a loop structure to be converted from a pooling operator of the convolutional neural network model;

[0020] Determine the number of operations involved in the loop calculation in the loop structure;

[0021] Defining vector data matching the data type of the operands, wherein each vector data includes a preset number of operands;

[0022] Converting a loop body statement in the loop structure that performs loop calculation on the operand into a single instruction multiple data (SIMD) instruction that performs loop calculation on the vector data;

[0023] The converted loop structure is executed, and in at least one loop calculation, the loop calculation is performed in parallel on the preset number of operands by executing the SIMD instruction.

[0024] The embodiment of the present invention further discloses a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, one or more operation processing methods described in the embodiment of the present invention can be implemented.

[0025] The embodiments of the present invention include the following advantages:

[0026] The embodiment of the present invention optimizes the loop structure in the pooling operator of the convolutional neural network (CNN) model based on the single instruction multiple data (SIMD) instruction, and converts the loop body statement in the loop structure that performs loop calculations on operands into SIMD instructions that perform loop calculations on vector data. Wherein, each vector data contains a preset number of operands. Thus, by executing the converted loop structure, the preset number of operands can be looped in parallel by executing the SIMD instruction in at least one loop calculation. Through the embodiment of the present invention, the original serial loop operation is changed to a parallel loop operation, which can reduce the number of loop calculations, improve the efficiency of the loop calculation, and improve the performance of the deep learning model. In particular, for the pooling operator that uses a large number of loop structures, the computational efficiency of the pooling operator can be greatly improved, and thus the performance of the deep learning model based on CNN can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flowchart of steps of an operation processing method embodiment of the present invention;

[0028] Figure 2 is a structural block diagram of an operation processing device embodiment of the present invention;

[0029] Figure 3 is a block diagram of a device 800 for data processing according to the present invention. DETAILED DESCRIPTION

[0030] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] Method Embodiment

[0032] Reference Figure 1 , shows a flowchart of a step of an operation processing method embodiment of the present invention, the method can be applied to a convolutional neural network model, and the method can specifically include the following steps:

[0033] Step 101: extracting a loop structure to be converted from a pooling operator of the convolutional neural network model;

[0034] Step 102, determining the number of operands involved in the loop calculation in the loop structure;

[0035] Step 103: define vector data matching the data type of the operand, wherein each vector data contains a preset number of operands;

[0036] Step 104: converting a loop body statement in the loop structure for performing loop calculation on the operand into a single instruction multiple data (SIMD) instruction for performing loop calculation on the vector data;

[0037] Step 105 : Execute the converted loop structure, and in at least one loop calculation, perform loop calculation on the preset number of operands in parallel by executing the SIMD instruction.

[0038] Pooling is an essential key algorithm in most CNN-based deep learning models. Pooling divides the input image into several rectangular regions and outputs the maximum or average value for each sub-region. Pooling can reduce the number of model parameters and prevent overfitting. The loop structure in the pooling operator usually has a large number of loops, which makes the loop operation more time-consuming, and a large number of loop calculations require multiple memory accesses, which seriously affects the performance of the deep learning model.

[0039] To solve this problem, an embodiment of the present invention optimizes the loop structure in the pooling operator of the convolutional neural network model based on the SIMD (Single Instruction Multiple Data) instruction to reduce the number of loop calculations and improve the performance of the deep learning model. SIMD is an important technology in the design of CPU (Central Processing Unit) and an extension of the basic instruction set of the CPU. SIMD executes the same instruction at the same time in a synchronous manner. Taking the addition instruction as an example, after the SISD (Single instruction, Single data) CPU decodes the addition instruction, the execution unit first accesses the memory to obtain the first operand; then accesses the memory again to obtain the second operand; and then the sum operation can be performed. In a SIMD-type CPU, after the instruction is decoded, several execution units access the memory at the same time and obtain all operands at one time for calculation.

[0040] The embodiment of the present invention converts the loop body statements for performing loop calculations on operands in the loop structure into SIMD instructions for performing loop calculations on vector data, thereby changing the original serial loop operation into a parallel loop operation, so as to reduce the number of loop calculations, improve the efficiency of the loop calculations, and improve the performance of the deep learning model. In particular, for the pooling operators that use a large number of loop structures, the computational efficiency of the pooling operators can be greatly improved, thereby improving the performance of the deep learning model based on CNN.

[0041] Furthermore, before optimizing the loop structure, the embodiment of the present invention first extracts the loop structure to be converted from the pooling operator of the convolutional neural network model. There are usually multiple loop structures in the convolutional neural network model, and the embodiment of the present invention extracts the loop structure to be converted from them for optimization. The loop structure to be converted refers to the loop structure that has a greater impact on the performance of the convolutional neural network model.

[0042] Optionally, the number of loops of the loop structure to be converted exceeds a first threshold, and / or the number of times the loop structure to be converted is called exceeds a second threshold.

[0043] When a loop structure has a large number of loops or is called a large number of times, the execution efficiency of the loop structure will directly affect the performance of the entire convolutional neural network model. A large number of loop structures are usually used in the pooling operator of the convolutional neural network model. Therefore, an embodiment of the present invention selectively optimizes the loop structure in the pooling operator of the convolutional neural network model, extracts loop structures whose loop times exceed a first threshold from the pooling operator of the convolutional neural network model, and / or extracts loop structures whose calls exceed a second threshold from the pooling operator of the convolutional neural network model as loop structures to be converted, and optimizes the loop structures to be converted, thereby significantly improving the performance of the convolutional neural network model.

[0044] In an optional embodiment of the present invention, the SIMD instruction may be an MSA instruction in a MIPS (Microprocessor without interlocked piped stages) platform.

[0045] Currently, the main computer architectures can be divided into two categories, CISC (Complex Instruction Set Computing) and RISC (Reduced Instruction Set Computing). The MIPS platform belongs to a RISC architecture.

[0046] The MSA (MIPS SIMD Architecture) instruction on the MIPS platform is a SIMD (Single Instruction Multiple Data)-based instruction.

[0047] The MSA instruction is a SIMD instruction on the MIPS platform. The MSA instruction is 128 bits and has a large instruction bit width. It can perform a loop calculation on multiple operands at the same time. For architectures based on the MIPS platform, such as the LoongArch architecture, the embodiments of the present invention can use the MSA instruction to optimize the loop structure executed on the MIPS platform, so that in the process of one loop calculation, a loop calculation can be performed on multiple operands at the same time, and the original serial loop operation is changed to a parallel loop operation, which can reduce the number of loop calculations and improve the efficiency of the loop calculation. For deep learning models that use a large number of loop structures, the performance of the deep learning models can be improved.

[0048] It is understandable that the embodiment of the present invention does not limit the application platform of the convolutional neural network model. For example, the application platform of the convolutional neural network model can be a MIPS platform, and the SIMD instruction can be an MSA instruction in the MIPS platform. Alternatively, the application platform of the convolutional neural network model can also be an x86 platform, and the SIMD instruction can be MMX (Multi Media eXtension, multimedia extension instruction set)\SSE (Streaming SIMD Extensions, single instruction multiple data stream extension)\AVX (Advanced Vector Extensions, advanced vector extension instruction set) in the x86 platform, etc.

[0049] For ease of description, the embodiments of the present invention are mainly described by taking the application of the convolutional neural network model to the MIPS platform and the SIMD instruction as the MSA instruction in the MIPS platform as an example. The execution process under the x86 platform and other platforms is similar and can be referred to each other.

[0050] It should be noted that the embodiments of the present invention do not limit the application scenarios of the deep learning model. For example, the deep learning model can be applied to the field of artificial intelligence, such as smart education, gates, security, smart homes and other application scenarios. The deep learning model includes but is not limited to AlexNet, VGG, GoogLeNet, ResNet, etc.

[0051] Furthermore, the loop structure may be a for loop structure in a pooling operator. In a deep learning model based on CNN, the pooling operator uses a large number of for loop structures, which greatly affects the performance of the pooling operator, and further affects the performance of the deep learning model based on CNN.

[0052] In specific applications, pooling operators include the maximum pooling method MaxPooling and the average pooling method AvgPooling. The maximum pooling method MaxPooling outputs the maximum value for each sub-region; the average pooling method AvgPooling outputs the average value for each sub-region. The maximum pooling method MaxPooling is usually used in CNN-based deep learning models. The maximum pooling method MaxPooling includes forward pooling operations and reverse pooling operations. For loop structures are used extensively in both forward pooling operations and reverse pooling operations.

[0053] Example 1 shows a for loop structure commonly used in the pooling operator as follows:

[0054] for(int oc=0; oc <OC;++oc){

[0055] d[oc]=nstl::max(s[oc],d[oc]);

[0056] }

[0057] The loop structure shown in Example 1 is used to take the larger of two numbers in two large arrays and assign it a value.

[0058] Example 2 shows another for loop structure commonly used in the pooling operator:

[0059]

[0060] The loop structure shown in Example 2 is used to assign values ​​to a large array.

[0061] It should be noted that, in practical applications, in addition to the for loop structure, the deep learning model may also include other types of loop structures, such as goto loop, while loop, do...while loop, etc. In the embodiment of the present invention, the optimization of the for loop structure is mainly used as an example for explanation. The optimization methods of other types of loop structures are similar and can be referred to each other.

[0062] In order to address the problem that a large number of loop structures in the pooling operator leads to low performance of the pooling operator, which in turn affects the performance of the deep learning model, an embodiment of the present invention uses SIMD instructions to optimize the loop structure in the pooling operator of the convolutional neural network model to reduce the number of loops and memory accesses, thereby improving the performance of the deep learning model.

[0063] Specifically, the number of operands involved in the loop calculation in the loop structure is first determined.

[0064] The loop structure can be used to repeatedly execute multiple statements. Here, "multiple statements" are called loop bodies. The number of times the loop body is repeated is controlled by the loop condition, which is called the control expression.

[0065] The embodiment of the present invention determines the operands involved in the loop calculation in the loop structure, that is, determines the operands in the loop body. Taking Example 1 as an example, the operands include array s[oc] and array d[oc]. The loop body in Example 1 is used to take one data from array s[oc] and array d[oc] respectively in each loop calculation, and store the larger one into array d[oc].

[0066] In order to optimize the loop structure by using SIMD instructions (such as MSA instructions of the MIPS platform), the embodiment of the present invention defines vector data matching the data type of the operands, wherein each vector data includes a preset number of operands.

[0067] Taking Example 1 as an example, the operands include array s[oc] and array d[oc], and the data types of array s[oc] and array d[oc] are int8, that is, array s[oc] and array d[oc] store int8 type data. Taking the MSA instruction of the MIPS platform as an example, in the MSA instruction, the vector data type matching the data type int8 is v16i8, and the vector data type v16i8 indicates that each vector data contains 16 int8 type data.

[0068] Therefore, in Example 1, a vector data type of v16i8 can be defined, and each such vector data contains 16 int8 type data in the array s[oc]. Similarly, another vector data type of v16i8 can be defined, and each such vector data contains 16 int8 type data in the array d[oc]. Therefore, each time a vector data is processed, 16 int8 type data can be read from the memory at a time, and these 16 int8 type data can be processed simultaneously, which can improve the computing efficiency.

[0069] It is understandable that the embodiment of the present invention does not limit the data type of the operands involved in the loop calculation. For example, the data type of the operands may be int8, int32, float32, etc.

[0070] In another example, when the data type of the operand involved in the loop calculation is int32, the vector data type that matches this data type is v4i32, and the vector data type that matches the data type int32 can be defined as vector data of v4i32, and each such vector data contains 4 data of the int32 type. For another example, when the data type of the operand involved in the loop calculation is float32, the vector data type that matches this data type is v4f32, and the vector data type that matches the data type float32 can be defined as vector data of v4f32, and each such vector data contains 4 data of the float32 type.

[0071] Next, the loop body statements in the loop structure that perform loop calculations on the operands are converted into single instruction multiple data stream SIMD instructions that perform loop calculations on the vector data, such as converted into MIPS single instruction multiple data stream architecture MSA instructions.

[0072] For example, in Example 1, the original loop body statement "d[oc] = nstl::max(s[oc], d[oc])" can be converted into the following MSA instruction "*(v16i8*)d = _msa_max_s_b(*(v16i8*)s, *(v16i8*)d)". In this way, when the converted loop structure is executed, a preset number (16) of operands can be simultaneously calculated in parallel in one loop, that is, 16 data are taken out from the array s[oc] and the array d[oc] at one time, and the two are compared and the larger one is assigned. In this way, 16 data can be processed simultaneously in one loop. In the next loop, the next group of 16 data are taken out from the array s[oc] and the array d[oc] at one time for parallel processing.

[0073] Similarly, in Example 2, after the original loop body statement is converted into an MSA instruction, when the converted loop structure is executed, a preset number (4) of operands can be simultaneously performed in parallel in one loop calculation, that is, 4 operands can be processed simultaneously in one loop calculation, and in the next loop calculation, the next group of 4 operands can be processed simultaneously.

[0074] In an optional embodiment of the present invention, before converting the loop body statement in the loop structure that performs loop calculation on the operand into a SIMD instruction that performs loop calculation on the vector data, the method further includes: determining the operation type of the loop body statement in the loop structure;

[0075] The converting the loop body statement for performing loop calculation on the operand in the loop structure into a SIMD instruction for performing loop calculation on the vector data comprises:

[0076] Determine a SIMD instruction built-in function that matches the operation type;

[0077] Based on the SIMD instruction built-in function, a loop body statement in the loop structure that performs loop calculation on the operand is converted into a SIMD instruction that performs loop calculation on the vector data.

[0078] Furthermore, the operation type may include but is not limited to a comparison operation type and / or an assignment operation type.

[0079] The operation type of the loop body statement in the loop structure shown in Example 1 includes a comparison operation type, and the operation type of the loop body statement in the loop structure shown in Example 2 includes an assignment operation type.

[0080] The SIMD instruction built-in function may be a pre-packaged function interface, and by calling the built-in function, a loop body statement for performing loop calculations on the operands in a loop structure may be converted into a SIMD instruction for performing loop calculations on the vector data.

[0081] Taking the MSA instruction of the MIPS platform as an example, in Example 1, it can be determined that the MSA instruction built-in function that matches the operation type (comparison operation type) is "_msa_max_s_b". Based on the MSA instruction built-in function, the loop body statement in the loop structure that performs loop calculation on the operand can be converted into an MSA instruction that performs loop calculation on the vector data, such as "*(v16i8*)d=_msa_max_s_b(*(v16i8*)s,*(v16i8*)d)".

[0082] In Example 2, it can be determined that the MSA instruction built-in function that matches the operation type (assignment operation type) is "_msa_fill_w". Based on the MSA instruction built-in function, the loop body statement in the loop structure that performs loop calculation on the operand can be converted into an MSA instruction that performs loop calculation on the vector data, such as "v4f32 data_tmp=(v4f32)_msa_fill_w(0.f)".

[0083] It can be understood that, for the x86 platform, the corresponding SIMD instruction built-in function in the platform can be determined. Through the built-in function, the loop body statement in the loop structure that performs loop calculations on the operands can be converted into SIMD instructions that comply with the x86 platform and perform loop calculations on the vector data, such as MMX\SSE\AVX instructions, etc.

[0084] It should be noted that the embodiment of the present invention does not limit the type of programming language used in the loop structure, and the embodiment of the present invention takes C++ language as an example. Of course, the programming language used in the loop structure may include but is not limited to C++ language, Python language, Java language, C language, and R language.

[0085] In an optional embodiment of the present invention, before converting the loop body statement in the loop structure for performing loop calculation on the operand into a SIMD instruction for performing loop calculation on the vector data, the method further comprises: defining a pointer variable for pointing to an offset address of the operand;

[0086] In the at least one loop calculation, executing the SIMD instruction in parallel to perform loop calculation on the preset number of operands includes:

[0087] In at least one loop calculation, reading the preset number of operands according to the offset address pointed to by the pointer variable;

[0088] Performing cyclic calculations on the preset number of operands in parallel by executing the SIMD instructions;

[0089] After performing cyclic calculations on the preset number of operands in parallel by executing the SIMD instruction, the method further includes:

[0090] The offset address is moved by a length corresponding to the preset number of operands, and the moved offset address is assigned to the pointer variable, so that the preset number of operands can be read according to the moved offset address in the next loop calculation.

[0091] Taking the MSA instruction of the MIPS platform as an example, in one example, the embodiment of the present invention uses the MSA instruction to optimize the loop structure shown in Example 1, and the converted loop structure is as follows:

[0092]

[0093]

[0094] In Example 1, since the data types of operands s[oc] and d[oc] involved in the loop calculation are int8, a vector type v16i8 matching the data type is used.

[0095] In the converted loop structure of the above example 1, the first and second lines of code define pointer variables s and d used to point to the offset addresses of the operands.

[0096] Since the number of loops of the loop structure in Example 1 is OC, after the embodiment of the present invention optimizes the loop structure using the MSA instruction, one loop can process 16 operands at the same time, and the number of loops is reduced from the original OC times to OC divided by 16 times. Therefore, the quotient obtained by dividing OC by 16 can be used as the number of loops of the converted loop structure. As shown in the above converted loop structure, the third line of code "int len=OC>>4" is used to assign OC to the variable len after shifting it right by four bits, where shifting it right by four bits is equivalent to taking the quotient of dividing OC by 16. That is, the number of loops of the converted loop structure is len.

[0097] The fourth line of code "*(v16i8*)d=_msa_max_s_b(*(v16i8*)s,*(v16i8*)d)" is the MSA instruction after the original loop body statement is converted. By executing the MSA instruction, a preset number (16) of operands can be calculated in parallel in one loop calculation.

[0098] The fifth and sixth lines of code "s+=16" and "d+=16" are used to move the offset address by the length corresponding to the preset number (16) of operands, and assign the moved offset address to the pointer variables s and d, so as to read the next set of preset number (16) operands according to the moved offset address in the next loop calculation.

[0099] It should be noted that the code of the converted loop structure is written in a high-level language, so it is necessary to compile the code to obtain assembly instructions that can be recognized by the computer. Among them, the MSA instruction "*(v16i8*)d=_msa_max_s_b(*(v16i8*)s,*(v16i8*)d)" will generate MSA assembly instructions after compilation.

[0100] Taking the MSA instruction of the MIPS platform as an example, in one example, the embodiment of the present invention uses the MSA instruction to optimize the loop structure shown in Example 2, and the converted loop structure is as follows:

[0101]

[0102]

[0103] In Example 2, since the data type of the operand involved in the loop calculation is float32, the vector type v4f32 matching the data type is used.

[0104] In the converted loop structure of the above example 2, the first line of code "int len=OC>>2" is used to assign the loop count OC to the variable len after shifting it right by two bits, where shifting it right by two bits is equivalent to dividing OC by 4 and taking the quotient. That is, the loop count of the converted loop structure is len. The second line of code defines the pointer variable diff_src_offset used to point to the offset address of the operand. The third line of code "v4f32data_tmp=(v4f32)_msa_fill_w(0.f)" is the MSA instruction after the original loop body statement is converted. By executing the MSA instruction, a preset number (4) of operands can be calculated in parallel in one loop calculation. Among them, data_tmp is the vector data corresponding to the operand 0.f, and one vector data data_tmp contains 4 float32 type data 0.f. The original assignment operations "diff_src_fp32[oc]=0.f" and "diff_src[src_offset_init+oc]=0.f" can only assign one data in the array diff_src_fp32[oc] and the array diff_src[src_offset_init+oc] to 0.f each time. After using the MSA instruction optimization, the vector data data_tmp can be used to assign values ​​to the array diff_src_fp32[oc] and the array diff_src[src_offset_init+oc]. Therefore, each time, the four data in the array diff_src_fp32[oc] and the array diff_src[src_offset_init+oc] can be assigned to 0.f at the same time. The corresponding code is shown in the fourth and fifth lines of code.

[0105] The sixth and seventh lines of code "diff_src_fp32+=4" and "diff_src_offset+=4" are used to move the offset address by the length corresponding to the preset number (4) of operands, and assign the moved offset address to the pointer variable so that the next set of preset number (4) of operands can be read according to the moved offset address in the next loop calculation.

[0106] It should be noted that the MSA instruction "v4f32 data_tmp = (v4f32)_msa_fill_w (0.f)" will generate MSA assembly instructions after compilation.

[0107] In an optional embodiment of the present invention, in at least one loop calculation, performing loop calculation on the preset number of operands in parallel by executing the SIMD instruction includes:

[0108] If the number of loops of the loop structure is an integer multiple of the preset number, then in each loop calculation, the preset number of operands are looped in parallel by executing the SIMD instruction until the loop ends; or,

[0109] If the number of loops of the loop structure is not an integer multiple of the preset number, then in the last loop calculation, loop calculation is performed on the remaining operands by executing the loop body statements before the conversion.

[0110] Taking the MSA instruction of the MIPS platform as an example, in actual applications, the MSA instruction can process a preset number of operands in parallel at a time. If the number of loops is an integer multiple of the preset number, then in each loop calculation, the preset number of operands can be processed in parallel at the same time through the MSA instruction. For example, if the number of loops OC is 48, for the loop structure in Example 1, 48 loop calculations need to be performed without optimization. After optimization, each loop can process 16 operands at the same time, and only 48 / 16=3 loops need to be executed, which greatly reduces the number of loop calculations.

[0111] If the number of loops is not an integer multiple of the preset number (16), the remaining operands are looped according to the loop body statements in the loop structure before conversion. For example, if the number of loops OC is 50, for the loop structure in Example 1, after optimization, since 50 is not an integer multiple of 16 (the quotient of 50 divided by 16 is 3 and the remainder is 2), in the first three loop calculations, the MSA instruction can be used to process 16 operands in parallel each time. For the remaining two operands, the loop calculations are still performed according to the loop body statements before conversion, that is, two loop calculations are still required. When OC is 50, a total of 3+2=5 loop calculations need to be performed. Compared with the 50 loop calculations that need to be performed without optimization, the number of loop calculations is still greatly reduced.

[0112] Specifically, the remaining two operands can be calculated according to the following for loop body:

[0113]

[0114] Similarly, for Example 2, if the number of loops is not an integer multiple of the preset number (4), the preset number of operands are processed in parallel by the MSA instruction at the same time. The remaining operands are still looped and calculated according to the loop body statement before conversion. For example, the remaining operands can be calculated according to the following for loop body:

[0115]

[0116] The embodiment of the present invention optimizes the loop structure in the pooling operator of the convolutional neural network (CNN) model through SIMD instructions, and converts the loop body statements in the loop structure that perform loop calculations on operands into SIMD instructions that perform loop calculations on vector data, thereby changing the original serial loop operation into a parallel loop operation. In at least one loop calculation, executing the SIMD instruction once can perform loop calculations on a preset number of operands in parallel.

[0117] It is understandable that for different SIMD instructions in different platforms, such as MMX\SSE\AVX instructions in x86 platforms, the preset number of operands that can be calculated in parallel loops in one SIMD instruction may be different. Therefore, the preset number can be set according to the number of parallel processing bits supported by SIMD instructions in different platforms and the hardware conditions of the platform.

[0118] In summary, the embodiment of the present invention optimizes the loop structure in the pooling operator of the convolutional neural network CNN model based on the single instruction multiple data stream SIMD instruction, and converts the loop body statement in the loop structure that performs loop calculations on operands into SIMD instructions that perform loop calculations on vector data. Wherein, each vector data contains a preset number of operands. Therefore, by executing the converted loop structure, the preset number of operands can be looped in parallel by executing the SIMD instruction in at least one loop calculation. Through the embodiment of the present invention, the original serial loop operation is changed to a parallel loop operation, which can reduce the number of loop calculations, improve the efficiency of the loop calculation, and improve the performance of the deep learning model. In particular, for the pooling operator that uses a large number of loop structures, the computational efficiency of the pooling operator can be greatly improved, and then the performance of the deep learning model based on CNN can be improved.

[0119] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0120] Device Embodiment

[0121] Reference Figure 2 , shows a structural block diagram of an operation processing device embodiment of the present invention, the device can be applied to a convolutional neural network model, and the device can specifically include the following modules:

[0122] A structure determination module 201, used to extract the cyclic structure to be converted from the pooling operator of the convolutional neural network model;

[0123] A data determination module 202 is used to determine the number of operands involved in the loop calculation in the loop structure;

[0124] A vector definition module 203, used to define vector data matching the data type of the operand, wherein each vector data includes a preset number of operands;

[0125] An instruction conversion module 204, configured to convert a loop body statement in the loop structure for performing loop calculation on the operand into a single instruction multiple data (SIMD) instruction for performing loop calculation on the vector data;

[0126] The instruction execution module 205 is used to execute the converted loop structure, and in at least one loop calculation, perform loop calculation on the preset number of operands in parallel by executing the SIMD instruction.

[0127] Optionally, the instruction execution module includes:

[0128] A first execution submodule is configured to, if the number of loops of the loop structure is an integer multiple of the preset number, perform loop calculations on the preset number of operands in parallel by executing the SIMD instruction in each loop calculation until the loop ends; or

[0129] The second execution submodule is used to perform loop calculation on the remaining operands by executing the loop body statements before conversion in the last loop calculation if the loop number of the loop structure is not an integer multiple of the preset number.

[0130] Optionally, the device further comprises:

[0131] A variable definition module, used for defining a pointer variable for pointing to an offset address of the operand;

[0132] The instruction execution module comprises:

[0133] A data reading submodule, used for reading the preset number of operands according to the offset address pointed to by the pointer variable in at least one loop calculation;

[0134] An instruction execution submodule, used for performing cyclic calculation on the preset number of operands in parallel by executing the SIMD instruction;

[0135] The device also includes:

[0136] The address offset module is used to move the offset address by the length corresponding to the preset number of operands, and assign the moved offset address to the pointer variable, so as to read the preset number of operands according to the moved offset address in the next loop calculation.

[0137] Optionally, the device further comprises:

[0138] A type determination module, used to determine the operation type of the loop body statement in the loop structure;

[0139] The instruction conversion module comprises:

[0140] A function determination submodule, used to determine a SIMD instruction built-in function matching the operation type;

[0141] The instruction conversion submodule is used to convert the loop body statement for performing loop calculation on the operands in the loop structure into a SIMD instruction for performing loop calculation on the vector data based on the SIMD instruction built-in function.

[0142] Optionally, the operation type includes a comparison operation type and / or an assignment operation type.

[0143] Optionally, the number of loops of the loop structure to be converted exceeds a first threshold, and / or the number of times the loop structure to be converted is called exceeds a second threshold.

[0144] Optionally, the SIMD instruction is an MSA instruction in a microprocessor MIPS platform without internal interlocked pipeline stages.

[0145] Optionally, the loop structure is a for loop structure in a pooling operator.

[0146] The embodiment of the present invention optimizes the loop structure in the pooling operator of the convolutional neural network (CNN) model based on the single instruction multiple data (SIMD) instruction, and converts the loop body statement in the loop structure that performs loop calculations on operands into SIMD instructions that perform loop calculations on vector data. Wherein, each vector data contains a preset number of operands. Thus, by executing the converted loop structure, the preset number of operands can be looped in parallel by executing the SIMD instruction in at least one loop calculation. Through the embodiment of the present invention, the original serial loop operation is changed to a parallel loop operation, which can reduce the number of loop calculations, improve the efficiency of the loop calculation, and improve the performance of the deep learning model. In particular, for the pooling operator that uses a large number of loop structures, the computational efficiency of the pooling operator can be greatly improved, and thus the performance of the deep learning model based on CNN can be improved.

[0147] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0148] An embodiment of the present invention provides a device for operation processing, which is applied to a convolutional neural network model. The device includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations:

[0149] Extracting a loop structure to be converted from a pooling operator of the convolutional neural network model;

[0150] Determine the number of operations involved in the loop calculation in the loop structure;

[0151] Defining vector data matching the data type of the operands, wherein each vector data includes a preset number of operands;

[0152] Converting a loop body statement in the loop structure that performs loop calculation on the operand into a single instruction multiple data (SIMD) instruction that performs loop calculation on the vector data;

[0153] The converted loop structure is executed, and in at least one loop calculation, the loop calculation is performed in parallel on the preset number of operands by executing the SIMD instruction.

[0154] Figure 3 8 is a block diagram of an apparatus 800 for operation processing according to an exemplary embodiment. For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0155] Reference Figure 3 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0156] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0157] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0158] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0159] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0160] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a voice information processing mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0161] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0162] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0163] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency information processing (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0164] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0165] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0166] An embodiment of the present invention provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by a program or processor of a terminal, the terminal is enabled to execute an operation processing method, the method comprising: determining operands participating in a loop calculation in a loop structure; defining vector data matching the data type of the operands, wherein each vector data contains a preset number of operands; converting a loop body statement in the loop structure that performs a loop calculation on the operands into a MIPS single instruction multiple data stream architecture MSA instruction that performs a loop calculation on the vector data; executing the converted loop structure, and in at least one loop calculation, performing a loop calculation on the preset number of operands in parallel by executing the MSA instruction.

[0167] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0168] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0169] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0170] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a predictable manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0171] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0172] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0173] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0174] The above is a detailed introduction to an operation processing method and device, a device for operation processing, and a storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a general technician in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An operation processing method, characterized in that: Applied to a convolutional neural network model, the method comprises: Extracting a loop structure to be converted from a pooling operator of the convolutional neural network model; Determine the number of operations involved in the loop calculation in the loop structure; Defining vector data matching the data type of the operands, wherein each vector data includes a preset number of operands; Converting a loop body statement in the loop structure that performs loop calculation on the operand into a single instruction multiple data (SIMD) instruction that performs loop calculation on the vector data; Executing the converted loop structure, and in at least one loop calculation, performing loop calculation on the preset number of operands in parallel by executing the SIMD instruction; In the at least one loop calculation, executing the SIMD instruction in parallel to perform loop calculation on the preset number of operands includes: If the number of loops of the loop structure is an integer multiple of the preset number, then in each loop calculation, the preset number of operands are looped in parallel by executing the SIMD instruction until the loop ends; or, If the number of loops of the loop structure is not an integer multiple of the preset number, in the last loop calculation, loop calculation is performed on the remaining operands by executing the loop body statements before the conversion; Before converting the loop body statement for performing loop calculation on the operand in the loop structure into a SIMD instruction for performing loop calculation on the vector data, the method further comprises: Define a pointer variable for pointing to the offset address of the operand; In the at least one loop calculation, executing the SIMD instruction in parallel performs loop calculation on the preset number of operands, further comprising: In at least one loop calculation, reading the preset number of operands according to the offset address pointed to by the pointer variable; Performing cyclic calculations on the preset number of operands in parallel by executing the SIMD instructions; After performing cyclic calculations on the preset number of operands in parallel by executing the SIMD instruction, the method further includes: The offset address is moved by a length corresponding to the preset number of operands, and the moved offset address is assigned to the pointer variable, so that the preset number of operands can be read according to the moved offset address in the next loop calculation.

2. The method according to claim 1, characterized in that Before converting the loop body statement for performing loop calculation on the operand in the loop structure into a SIMD instruction for performing loop calculation on the vector data, the method further comprises: Determine the operation type of the loop body statement in the loop structure; The converting the loop body statement for performing loop calculation on the operand in the loop structure into a SIMD instruction for performing loop calculation on the vector data comprises: Determine a SIMD instruction built-in function that matches the operation type; Based on the SIMD instruction built-in function, a loop body statement in the loop structure that performs loop calculation on the operand is converted into a SIMD instruction that performs loop calculation on the vector data.

3. The method according to claim 2, characterized in that The operation type includes a comparison operation type and / or an assignment operation type.

4. The method according to any one of claims 1 to 3, characterized in that: The number of loops of the loop structure to be converted exceeds a first threshold, and / or the number of calls of the loop structure to be converted exceeds a second threshold.

5. The method according to any one of claims 1 to 3, characterized in that: The SIMD instruction is an MSA instruction in a microprocessor MIPS platform without an internal interlocking pipeline stage.

6. An operation processing device, characterized in that: Applied to a convolutional neural network model, the device comprises: A structure determination module, used to extract the cyclic structure to be converted from the pooling operator of the convolutional neural network model; A data determination module, used for determining the number of operands involved in loop calculation in a loop structure; A vector definition module, used for defining vector data matching the data type of the operands, wherein each vector data contains a preset number of operands; An instruction conversion module, used for converting a loop body statement in the loop structure for performing loop calculation on the operand into a single instruction multiple data (SIMD) instruction for performing loop calculation on the vector data; An instruction execution module, configured to execute the converted loop structure, and in at least one loop calculation, perform loop calculation on the preset number of operands in parallel by executing the SIMD instruction; The instruction execution module includes: A first execution submodule is configured to, if the number of loops of the loop structure is an integer multiple of the preset number, perform loop calculations on the preset number of operands in parallel by executing the SIMD instruction in each loop calculation until the loop ends; or A second execution submodule is used for, if the number of loops of the loop structure is not an integer multiple of the preset number, in the last loop calculation, performing loop calculation on the remaining operands by executing the loop body statements before conversion; The device also includes: A variable definition module, used for defining a pointer variable for pointing to an offset address of the operand; The instruction execution module includes: A data reading submodule, used for reading the preset number of operands according to the offset address pointed to by the pointer variable in at least one loop calculation; An instruction execution submodule, used for performing cyclic calculation on the preset number of operands in parallel by executing the SIMD instruction; The device also includes: The address offset module is used to move the offset address by the length corresponding to the preset number of operands, and assign the moved offset address to the pointer variable, so as to read the preset number of operands according to the moved offset address in the next loop calculation.

7. The device according to claim 6, characterized in that The device also includes: A type determination module, used to determine the operation type of the loop body statement in the loop structure; The instruction conversion module comprises: A function determination submodule, used to determine a SIMD instruction built-in function matching the operation type; The instruction conversion submodule is used to convert the loop body statement for performing loop calculation on the operands in the loop structure into a SIMD instruction for performing loop calculation on the vector data based on the SIMD instruction built-in function.

8. The device according to claim 7, characterized in that The operation type includes a comparison operation type and / or an assignment operation type.

9. The device according to any one of claims 6 to 8, characterized in that: The number of loops of the loop structure to be converted exceeds a first threshold, and / or the number of calls of the loop structure to be converted exceeds a second threshold.

10. The device according to any one of claims 6 to 8, characterized in that: The SIMD instruction is an MSA instruction in a microprocessor MIPS platform without an internal interlocking pipeline stage.

11. A device for operation processing, characterized in that Applied to a convolutional neural network model, the device includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations: Extracting a loop structure to be converted from a pooling operator of the convolutional neural network model; Determine the number of operations involved in the loop calculation in the loop structure; Defining vector data matching the data type of the operands, wherein each vector data includes a preset number of operands; Converting a loop body statement in the loop structure that performs loop calculation on the operand into a single instruction multiple data (SIMD) instruction that performs loop calculation on the vector data; Executing the converted loop structure, and in at least one loop calculation, performing loop calculation on the preset number of operands in parallel by executing the SIMD instruction; In the at least one loop calculation, executing the SIMD instruction in parallel to perform loop calculation on the preset number of operands includes: If the number of loops of the loop structure is an integer multiple of the preset number, then in each loop calculation, the preset number of operands are looped in parallel by executing the SIMD instruction until the loop ends; or, If the number of loops of the loop structure is not an integer multiple of the preset number, in the last loop calculation, loop calculation is performed on the remaining operands by executing the loop body statements before the conversion; Before converting the loop body statement for performing loop calculation on the operand in the loop structure into a SIMD instruction for performing loop calculation on the vector data, the method further includes: Define a pointer variable for pointing to the offset address of the operand; In the at least one loop calculation, executing the SIMD instruction in parallel performs loop calculation on the preset number of operands, further comprising: In at least one loop calculation, reading the preset number of operands according to the offset address pointed to by the pointer variable; Performing cyclic calculations on the preset number of operands in parallel by executing the SIMD instructions; After executing the SIMD instruction to perform cyclic calculations on the preset number of operands in parallel, the method further includes: The offset address is moved by a length corresponding to the preset number of operands, and the moved offset address is assigned to the pointer variable, so that the preset number of operands can be read according to the moved offset address in the next loop calculation.

12. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the operation processing method according to any one of claims 1 to 5 are implemented.

Images