Optimization of neural network model, data processing method and device, storage medium

Abstract

The present disclosure provides a neural network model optimization, data processing method and device, and storage medium, wherein the neural network model optimization method comprises: selecting at least two target network layers from the neural network model, wherein the at least two target network layers comprise a first network layer and a second network layer connected to an input end of the first network layer, and the first network layer is an element-by-element operation layer; fusing the at least two target network layers to obtain the optimized neural network model.

Description

Technical Field

[0001] The present disclosure relates to the field of deep learning, and in particular to the optimization of neural network models, data processing methods and devices, and storage media. Background Art

[0002] Currently, element-wise operation (Eltwise) layers are widely used in neural networks. The Eltwise layer performs element-level calculations on multiple inputs and outputs the results to the next network layer. Therefore, the Eltwise layer generally has multiple inputs and one output.

[0003] The operation overhead of the Eltwise layer is mainly reflected in three aspects: loading the input from the memory to the processor, the processor performing calculation operations, and writing the calculation results back to the memory. Summary of the invention

[0004] The present invention provides a neural network model optimization, data processing method and device, and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for optimizing a neural network model is provided, the method comprising: selecting at least two target network layers from the neural network model, wherein the at least two target network layers include a first network layer and a second network layer connected to an input end of the first network layer, the first network layer being an element-by-element operation layer; fusing the at least two target network layers to obtain the optimized neural network model.

[0006] In the above embodiment, at least two target network layers can be selected from the neural network model, and the at least two target network layers include a first network layer and a second network layer connected to the input end of the first network layer, wherein the first network layer can be an element-by-element operation layer. Fusion processing is performed on at least two target network layers to obtain an optimized neural network model, thereby achieving the purpose of optimizing the neural network model. In addition, by optimizing the neural network model, the execution time of the neural network model can be reduced and the execution efficiency of the neural network model can be improved. Furthermore, since the execution time of the neural network model is reduced, the throughput of the neural network model processing data per unit time is improved, and the delay and power consumption of processing the same data scale per unit time are reduced.

[0007] In some optional embodiments, the fusing the at least two target network layers comprises: fusing the element-by-element operation of the first network layer into the second network layer.

[0008] In some optional embodiments, selecting at least two target network layers from the neural network model includes: selecting a network layer with the deepest network depth from at least one network layer connected to the input end of the element-by-element operation layer as the second network layer.

[0009] In some optional embodiments, the second network layer is used to implement the first operation, and the data processing process of the optimized neural network model includes: reading the first input data of the second network layer and at least one second input data of the element-by-element operation layer from a first storage device; performing the first operation on the first input data through a fusion layer to obtain a first processing result, wherein the fusion layer is obtained by fusing the at least two target network layers; performing the element-by-element operation of the element-by-element operation layer on the first processing result and the at least one second input data through the fusion layer to obtain a second processing result; and writing the second processing result to a second storage device, wherein the output result of the optimized neural network model is obtained based on the second processing result.

[0010] In some optional embodiments, the method also includes: determining a target number of data to be processed at one time by the fusion layer obtained by the fusion processing, wherein the target number is an integer greater than 2; performing the element-by-element operation of the element-by-element operation layer on the first processing result and the at least one second input data through the fusion layer to obtain a second processing result, including: based on the target number, synchronously reading the first processing result and the at least one second input data, performing the element-by-element operation of the element-by-element operation layer to obtain the second processing result, wherein the sum of the data quantities of the first processing result and the at least one second input data is equal to the target number.

[0011] In some optional embodiments, determining the target number to be processed by the fusion layer at one time includes: obtaining test performance parameters of the neural network model under multiple candidate data quantities; and determining the target number from the multiple candidate data quantities based on the test performance parameters.

[0012] In some optional embodiments, the element-by-element operation of the first network layer includes a multiplication-addition operation based on a specific parameter; based on the target number, the first processing result and the at least one second input data are synchronously read, and the element-by-element operation of the element-by-element operation layer is processed to obtain a second processing result, including: writing the first processing result and the at least one second input data read synchronously into the first storage device of the target number; performing multiplication-addition operations on the specific parameters pre-written in at least one second storage device and the data in at least one first storage device, respectively, to obtain the second processing result.

[0013] In some optional embodiments, the element-by-element operation of the first network layer includes a multiplication-addition operation based on specific parameters; the method further includes: if at least one specific parameter in the multiplication-addition operation includes a target value, optimizing the element-by-element operation of the first network layer to obtain the optimized element-by-element operation.

[0014] In some optional embodiments, the element-by-element operation of the first network layer includes a multiply-add operation, and the method further includes: receiving a multiply-add instruction; and performing the multiply-add operation of the first network layer based on the multiply-add instruction.

[0015] According to a second aspect of an embodiment of the present disclosure, a data processing method is provided, the method comprising: reading first input data and at least one second input data from a first storage device; performing a first operation on the first input data through a fusion layer to obtain a first processing result, wherein the fusion layer is obtained by fusing a first network layer and a second network layer, the first network layer is an element-by-element operation layer, and the output end of the second network layer is connected to the input end of the first network layer; performing an element-by-element operation on the first processing result and the at least one second input data through the fusion layer to obtain a second processing result; and writing the second processing result to a second storage device.

[0016] In the above embodiment, the first input data of the second network layer and at least one second input data of the element-by-element operation layer are processed through the fusion layer, and there is no need to write the first processing result of the second network layer into the storage device after obtaining it. When the first processing result and at least one second input data are processed by the fusion layer through the element-by-element operation, there is no need to read the first processing result from the storage device, thereby improving the efficiency of data processing.

[0017] In some optional implementations, the method further includes: determining a target number of data to be processed by the fusion layer at one time, the target number being an integer greater than 2; and reading the first input data and at least one second input data from the first storage device includes: synchronously reading at least one second input data based on the target number.

[0018] In some possible implementations, determining the target number to be processed by the fusion layer at one time includes: obtaining test performance parameters of the neural network model under multiple candidate data quantities; and determining the target number from the multiple candidate data quantities based on the test performance parameters.

[0019] In some possible implementations, the element-by-element operation of the first network layer includes a multiplication-addition operation based on a specific parameter; the at least one second input data and / or the first processing result is written into at least one first storage device; the first processing result and the at least one second input data are processed by the fusion layer by an element-by-element operation to obtain a second processing result, including: performing multiplication-addition operations on the specific parameters pre-written into the at least one second storage device and the data stored in the at least one first storage device, respectively.

[0020] In a possible implementation, in response to the designated parameter corresponding to the second input data being 0, the second processing result is obtained without performing any operation on the second input data.

[0021] In a possible implementation, in response to the designated parameter corresponding to the second input data being 1 or -1, the second processing result is obtained by addition or subtraction of the second input data and the first processing result.

[0022] In a possible implementation manner, a multiplication and addition operation is performed on the first processing result and the at least one second input data to obtain the second processing result.

[0023] Optionally, a multiply-add instruction may also be received, and the multiply-add operation may be performed based on the multiply-add instruction. According to a third aspect of an embodiment of the present disclosure, a data processing method is provided, the method comprising: reading a target number of first input data from a storage device, the target number being an integer greater than 2; performing an element-by-element operation of a first network layer in a neural network model on the read target number of first input data to obtain a first element-by-element processing result; and storing the first element-by-element processing result in the storage device.

[0024] In the above embodiment, a target number of first input data greater than 2 can be read from the storage device, and the target data first input data can be processed by element-by-element operation, which reduces the number of times the input data is read and the number of times the data processing results are written back to the storage device during the element-by-element operation, thereby improving the efficiency of the element-by-element operation.

[0025] In some optional embodiments, when the number of multiple data to be processed by the element-by-element operation is greater than a preset value, the method further includes: obtaining multiple second input data from the storage device, wherein the multiple second input data include the first element-by-element processing result; performing the element-by-element operation on the multiple second input data to obtain a second element-by-element processing result.

[0026] In some optional embodiments, the method further includes: obtaining test performance parameters of the neural network model under multiple candidate data quantities; and determining the target number from the multiple candidate data quantities based on the test performance parameters.

[0027] In some optional embodiments, the element-by-element operation includes a multiplication and addition operation based on specific parameters; the element-by-element operation is performed on the target number of first input data read to obtain a first element-by-element processing result, including: writing the target number of first input data read into the target number of first storage devices; performing multiplication and addition operations on specific parameters pre-written in multiple second storage devices and the data in the target number of first storage devices to obtain the first element-by-element processing result.

[0028] In some optional embodiments, it also includes: receiving a target multiply-add instruction; performing element-by-element operation of the first network layer in the neural network model on the target number of first input data read, including: performing element-by-element operation of the first network layer in the neural network model on the target number of first input data read according to the target multiply-add instruction.

[0029] In some optional embodiments, the target number of first input data include at least one input data of the second network layer in the neural network model and at least one input data of the first network layer, wherein the output end of the second network layer is connected to the input end of the first network layer; performing element-by-element operation of the first network layer in the neural network model on the read target number of first input data to obtain a first element-by-element processing result includes: performing operation of the second network layer on at least one input data of the second network layer to obtain a first processing result; performing element-by-element operation on the first processing result and at least one input data of the first network layer to obtain a first element-by-element processing result.

[0030] According to a fourth aspect of an embodiment of the present disclosure, there is provided an optimization device for a neural network model, the device comprising: a selection module for selecting at least two target network layers from the neural network model, wherein the at least two target network layers include a first network layer and a second network layer connected to an input end of the first network layer, and the first network layer is an element-by-element operation layer; and a fusion processing module for fusing the at least two target network layers to obtain the optimized neural network model.

[0031] In some optional embodiments, the fusion processing module includes: fusing the element-by-element operation of the first network layer into the second network layer.

[0032] In some optional embodiments, the selection module includes: selecting a network layer with the deepest network depth from at least one network layer connected to an input end of the element-by-element operation layer as the second network layer.

[0033] In some optional embodiments, the second network layer is used to implement the first operation, and the apparatus further includes: a second reading module, used to read the first input data of the second network layer and at least one second input data of the element-by-element operation layer from a first storage device; a second data processing module, used to perform the first operation on the first input data through a fusion layer to obtain a first processing result, wherein the fusion layer is obtained by fusing the at least two target network layers; a third data processing module, used to perform the element-by-element operation of the element-by-element operation layer on the first processing result and the at least one second input data through the fusion layer to obtain a second processing result; and a first data writing module, used to write the second processing result to a second storage device, wherein the output result of the optimized neural network model is obtained based on the second processing result.

[0034] In some optional embodiments, the device also includes: a first determination module, used to determine the target number of data processed at one time by the fusion layer obtained by the fusion processing, and the target number is an integer greater than 2; the third data processing module includes: based on the target number, synchronously reading the first processing result and the at least one second input data, performing the element-by-element operation of the element-by-element operation layer to obtain a second processing result, wherein the sum of the data quantity of the first processing result and the at least one second input data is equal to the target number.

[0035] In some optional embodiments, the first determination module includes: obtaining test performance parameters of the neural network model under multiple candidate data quantity conditions; based on the test performance parameters, determining the target number from the multiple candidate data quantities.

[0036] In some optional embodiments, the element-by-element operation of the first network layer includes a multiplication-addition operation based on a specific parameter; the third data processing module includes: writing the first processing result and the at least one second input data read synchronously into the target number of first storage devices; performing multiplication-addition operations on the specific parameters pre-written in at least one second storage device and the data in at least one first storage device, respectively, to obtain the second processing result.

[0037] In some optional embodiments, the element-by-element operation of the first network layer includes a multiplication-addition operation based on specific parameters; the device also includes: an optimization processing module, which is used to optimize the element-by-element operation of the first network layer if at least one specific parameter in the multiplication-addition operation includes a target value, so as to obtain the element-by-element operation after optimization.

[0038] In some optional embodiments, the element-by-element operation of the first network layer includes a multiplication-addition operation, and the device further includes: a first receiving module for receiving a multiplication-addition instruction; and an execution module for executing the multiplication-addition operation of the first network layer based on the multiplication-addition instruction.

[0039] According to a fifth aspect of an embodiment of the present disclosure, a data processing device is provided, the device comprising: a third reading module, used to read first input data and at least one second input data from a first storage device; a fourth data processing module, used to perform a first operation on the first input data through a fusion layer to obtain a first processing result, wherein the fusion layer is obtained by fusing a first network layer and a second network layer, the first network layer is an element-by-element operation layer, and the output end of the second network layer is connected to the input end of the first network layer; a fifth data processing module, used to perform an element-by-element operation on the first processing result and the at least one second input data through the fusion layer to obtain a second processing result; and a second data writing module, used to write the second processing result to a second storage device.

[0040] According to a sixth aspect of an embodiment of the present disclosure, a data processing device is provided, comprising: a first reading module, used to read a target number of first input data from a storage device, wherein the target number is an integer greater than 2; a first data processing module, used to perform an element-by-element operation of a first network layer in a neural network model on the read target number of first input data to obtain a first element-by-element processing result; and a storage module, used to store the first element-by-element processing result to the storage device.

[0041] In some optional embodiments, when the number of multiple data to be processed by the element-by-element operation is greater than a preset value, the device further includes: a data acquisition module, used to acquire multiple second input data from the storage device, wherein the multiple second input data include the first element-by-element processing result; and a sixth data processing module, used to perform the element-by-element operation on the multiple second input data to obtain a second element-by-element processing result.

[0042] In some optional embodiments, the device also includes: a parameter acquisition module, used to obtain test performance parameters of the neural network model under multiple candidate data quantity conditions; a second determination module, used to determine the target number from the multiple candidate data quantities based on the test performance parameters.

[0043] In some optional embodiments, the element-by-element operation includes a multiplication-addition operation based on specific parameters; the first data processing module includes: writing the target number of first input data read into the target number of first storage devices; performing multiplication-addition operations on the specific parameters pre-written in multiple second storage devices and the data in the target number of first storage devices to obtain the first element-by-element processing result.

[0044] In some optional embodiments, it also includes: a second receiving module for receiving a target multiplication and addition instruction; the first data processing module includes: according to the target multiplication and addition instruction, performing element-by-element operation of the first network layer in the neural network model on the target number of first input data read.

[0045] In some optional embodiments, the target number of first input data includes at least one input data of the second network layer in the neural network model and at least one input data of the first network layer, wherein the output end of the second network layer is connected to the input end of the first network layer; the first data processing module includes: performing operation processing of the second network layer on at least one input data of the second network layer to obtain a first processing result; performing element-by-element operation processing on the first processing result and at least one input data of the first network layer to obtain a first element-by-element processing result.

[0046] According to a seventh aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the optimization method of the neural network model described in any one of the first aspects above.

[0047] According to an eighth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the data processing method described in the second aspect above.

[0048] According to a ninth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the data processing method described in any one of the third aspects above.

[0049] According to the tenth aspect of an embodiment of the present disclosure, there is provided an optimization device for a neural network model, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to call the executable instructions stored in the memory to implement the optimization method for the neural network model described in any one of the first aspects.

[0050] According to the eleventh aspect of an embodiment of the present disclosure, a data processing device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to call the executable instructions stored in the memory to implement the data processing method described in the second aspect.

[0051] According to the twelfth aspect of an embodiment of the present disclosure, a data processing device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to call the executable instructions stored in the memory to implement the data processing method described in any one of the third aspects.

[0052] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0054] Figure 1 is a flow chart of a method for optimizing a neural network model according to an exemplary embodiment of the present disclosure;

[0055] Figure 2A is a schematic diagram of a network structure of a neural network model according to an exemplary embodiment of the present disclosure;

[0056] Figure 2B is a schematic diagram of a fusion layer processing process after optimization of a neural network model according to an exemplary embodiment of the present disclosure;

[0057] Figure 3 is a schematic diagram of a scenario of selecting a second network layer according to an exemplary embodiment of the present disclosure;

[0058] Figure 4 is a flow chart of another method for optimizing a neural network model according to an exemplary embodiment of the present disclosure;

[0059] Figure 5A is a schematic diagram of a network structure of a neural network model according to an exemplary embodiment of the present disclosure;

[0060] Figure 5B is a schematic diagram of a network structure after optimization of a neural network model according to an exemplary embodiment of the present disclosure;

[0061] Figure 5C is a schematic diagram of a processing process of a fusion layer after optimization of a neural network model according to an exemplary embodiment of the present disclosure;

[0062] Figure 6 is a flow chart of another method for optimizing a neural network model according to an exemplary embodiment of the present disclosure;

[0063] Figure 7 is a flow chart of another method for optimizing a neural network model according to an exemplary embodiment of the present disclosure;

[0064] Figure 8 is a flow chart of another method for optimizing a neural network model according to an exemplary embodiment of the present disclosure;

[0065] Fig. 9 is a flow chart of another method for optimizing a neural network model according to an exemplary embodiment of the present disclosure;

[0066] Fig.10 is a flow chart of another method for optimizing a neural network model according to an exemplary embodiment of the present disclosure;

[0067] Fig.11 is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure;

[0068] Fig.12 is a flow chart of another data processing method according to an exemplary embodiment of the present disclosure;

[0069] Fig.13 is a flow chart of another data processing method according to an exemplary embodiment of the present disclosure;

[0070] Fig.14 is a flow chart of another data processing method according to an exemplary embodiment of the present disclosure;

[0071] Fig.15 is a flow chart of another data processing method according to an exemplary embodiment of the present disclosure;

[0072] Fig.16 is a flow chart of another data processing method according to an exemplary embodiment of the present disclosure;

[0073] Fig.17 is a block diagram of an optimization device for a neural network model according to an exemplary embodiment of the present disclosure;

[0074] Fig.18 is a block diagram of a data processing device according to an exemplary embodiment of the present disclosure;

[0075] Fig.19 is a block diagram of another data processing device according to an exemplary embodiment of the present disclosure;

[0076] Fig. 20 is a structural schematic diagram of an optimization device for a neural network model according to an exemplary embodiment of the present disclosure;

[0077] Fig.21 It is a structural schematic diagram of a data processing device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0078] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0079] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms "a", "the" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0080] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0081] At present, the application of element-by-element operation layers in neural network models is becoming more and more widespread, and the element-by-element operation layers can be used to perform data fusion, context semantic association, etc. The disclosed embodiment provides an optimization method for a neural network model, which can be used for a neural network model that includes at least an element-by-element operation layer and at least one network layer connected to the input end of the element-by-element operation layer, and through which various computer vision tasks can be performed, such as image classification, task target detection, image semantic segmentation, human key point detection, etc.

[0082] For example Figure 1 As shown, Figure 1 A method for optimizing a neural network model according to an exemplary embodiment includes the following steps:

[0083] In step 101, at least two target network layers are selected from the neural network model.

[0084] In the embodiment of the present disclosure, at least two target network layers include a first network layer and a second network layer connected to the input end of the first network layer, wherein the first network layer is an element-by-element operation layer. The second network layer includes but is not limited to other network layers in the neural network model, such as a convolutional layer, an activation function layer, a pooling layer, a normalization layer, a fully connected layer, etc.

[0085] For example, neural network models such as Figure 2A As shown, the first network layer is an element-by-element operation layer, and both network layer 1 and network layer 2 are connected to the input end of the element-by-element operation layer. Then the target network layer includes the element-by-element operation layer, and at least one of network layer 1 and network layer 2 can be used as the second network layer in the target network layer.

[0086] In step 102, the at least two target network layers are fused to obtain the optimized neural network model.

[0087] In the embodiment of the present disclosure, the fusion processing refers to fusing the element-by-element operation of the first network layer into the second network layer, which is equivalent to fusing at least two target network layers through a fusion layer.

[0088] For example, before optimization, the structure of the neural network model is Figure 2A As shown, at least two target network layers include network layer 1 and element-by-element operation layer. The element-by-element operation of the element-by-element operation layer is integrated into network layer 1. The structure of the optimized neural network model is as follows: Figure 2B As shown in FIG. 4 , at least two target network layers are fused through the fusion layer.

[0089] In the network structure before fusion, the network layer 1 needs to read the input data B, perform the first operation on the input data B, obtain the first processing result A, and then write the first processing result A to the storage device, and the element-by-element operation layer needs to read the first processing result A and the input data B, perform the element-by-element operation on the first processing result and the input data B, and write the second processing result of the element-by-element operation back to the storage device. However, after the element-by-element operation layer and the network layer 1 are fused to obtain the fusion layer, the fusion layer reads the input data B, performs the first operation on the input data B to obtain the first processing result, and performs the element-by-element operation on the first processing result and the input data B to obtain the second processing result, and writes the second processing result to the storage device, thereby omitting the writing of the first processing result A to the storage device and the read-write operation of the element-by-element operation layer reading the first processing result A and the input data B.

[0090] In the above embodiment, at least two target network layers can be selected from the neural network model, and the at least two target network layers include a first network layer and a second network layer connected to the input end of the first network layer, wherein the first network layer can be an element-by-element operation layer. Fusion processing is performed on at least two target network layers to obtain an optimized neural network model, thereby achieving the purpose of optimizing the neural network model. In addition, by optimizing the neural network model, the execution time of the neural network model can be reduced and the execution efficiency of the neural network model can be improved. Furthermore, since the execution time of the neural network model is reduced, the throughput of the neural network model processing data per unit time is improved, and the delay and power consumption of processing the same data scale per unit time are reduced.

[0091] In some optional embodiments, step 101 may include:

[0092] From at least one network layer connected to the input end of the element-by-element operation layer, select the network layer with the deepest network depth as the second network layer.

[0093] In the disclosed embodiment, the first network layer is an element-by-element operation layer. Furthermore, the second network layer in the target network layer can be selected according to the network structure of the neural network model. If there is one network layer connected to the input end of the element-by-element operation layer, the network layer can be used as the second network layer. If there are multiple network layers connected to the input end of the element-by-element operation layer, the network layer with the deepest network depth can be used as the second network layer.

[0094] For example Figure 3 As shown, network layers 1, 2 and 3 are all connected to the input end of the element-by-element operation layer. In the neural network model, the network depth of network layers 2 and 3 is N, N is a positive integer, and the network depth of network layer 1 is N+1, so network layer 1 can be used as the second network layer.

[0095] In the above embodiment, the second network layer can be selected from the neural network model according to the network structure of the neural network model, which is simple to implement and has high availability.

[0096] In some optional embodiments, the second network layer can be used to implement the first operation, and the first operation corresponds to the second network layer, for example, if the second network layer is a convolution layer, the first operation is a convolution operation. Or if the second network layer is a normalization layer, the first operation is a normalization operation.

[0097] For example Figure 4 As shown, the data processing process of the optimized neural network model may include:

[0098] In step 201, first input data of the second network layer and at least one second input data of the element-by-element operation layer are read from a first storage device.

[0099] In the embodiment of the present disclosure, the first storage device may include but is not limited to the memory of the processor. The first input data is the input data of the second network layer in the target network layer. After the second network layer performs a first operation on the first input data, a first processing result is obtained. The second input data may include other input data in the input data of the first network layer except the first processing result.

[0100] For example, the first input data of the second network layer includes D. After the second network layer performs a first operation on the first input data, a first processing result A is obtained. The multiple input data of the element-by-element operation layer include A, B and C, then the second input data may include B and C.

[0101] In step 202, the first operation is performed on the first input data through a fusion layer to obtain a first processing result.

[0102] Since the optimized neural network model integrates the element-by-element operation of the first network layer into the second network layer, a fusion layer is obtained. The first operation is first processed on the first input data through the fusion layer to obtain a first processing result.

[0103] Taking the second network layer including a convolution layer as an example, according to the first operation corresponding to the second network layer, convolution processing is performed on the first input data of the read second network layer, so as to obtain a first processing result.

[0104] In step 203, the first processing result and the at least one second input data are processed by the element-by-element operation of the element-by-element operation layer through the fusion layer to obtain a second processing result.

[0105] In the embodiment of the present disclosure, after obtaining the first processing result, the first processing result and the second input data read previously are processed by the fusion layer through the element-by-element operation of the element-by-element operation layer, so as to obtain the second processing result.

[0106] For example, the element-by-element operation includes a sum operation, and the first processing result and the second input data can be summed through the fusion layer to obtain the second processing result.

[0107] In step 204, the second processing result is written into a second storage device.

[0108] In the embodiment of the present disclosure, the second storage device may include but is not limited to the memory of the processor, wherein the second storage device may be the same as or different from the first storage device, for example, the second storage device may be the same memory or a different memory as the first storage device, which is not limited in the present disclosure. The output result of the final optimized neural network model will be obtained based on the second processing result.

[0109] In the above embodiment, the first input data of the second network layer and at least one second input data of the element-by-element operation layer are processed through the fusion layer. There is no need to write the first processing result into the storage device first and then read it out when performing the element-by-element operation, thereby improving the efficiency of data processing.

[0110] The above scheme is further illustrated as follows.

[0111] The network structure of the original neural network model is as follows: Figure 5A As shown, in Figure 5A In the example, the element-by-element operation layer has two inputs A and B. After convolutional layer 1 and convolutional layer 2 obtain A and B, they write A and B into the storage device respectively. The element-by-element operation layer reads input A and input B into the processor, then performs element-by-element operations on input A and input B, such as element-by-element sum operations, and then writes the results back to the storage device. The processing process is as follows: Figure 5B shown.

[0112] After fusing the element-by-element operation layer and the convolutional layer 2 to obtain the fusion layer, the processing of the fusion layer is as follows: Figure 5C As shown in the figure, the fusion layer reads input A, performs convolution processing on input A by convolution layer 2 to obtain B, then performs element-by-element sum operation on A and B to obtain the calculation result, and writes the calculation result to the storage device, thereby omitting the operation of convolution layer 2 storing B and the operation of element-by-element operation layer reading input B, and the result finally written back to the storage device by the fusion layer is the same as the output result of the element-by-element operation layer before optimization.

[0113] In the above embodiment, it is not necessary to write the processing result B of the convolution layer 2 into the storage device first and then read it when performing the element-by-element operation, thereby optimizing the data IO process and improving the efficiency of data processing.

[0114] In some optional embodiments, for example Figure 6 As shown, the above method may also include:

[0115] In step 205, the target number of data to be processed at one time by the fusion layer obtained by the fusion process is determined.

[0116] In the embodiment of the present disclosure, the target number may be an integer greater than 2. For example, the fusion layer may process 3, 4 or more data at one time.

[0117] Accordingly, step 203 may include:

[0118] Based on the target number, the first processing result and the at least one second input data are synchronously read, and the element-by-element operation of the element-by-element operation layer is performed to obtain a second processing result.

[0119] In an embodiment of the present disclosure, the sum of the first processing result and the data quantity of the at least one second input data is equal to the target number.

[0120] For example, the target number is 3, and the fusion layer synchronously reads 3 data, including 1 first processing result and 2 second input data.

[0121] In the above embodiment, the fusion layer performs element-by-element operations on the first processing result of the target data and the at least one second input data, thereby reducing the number of times the input data is read and the number of times the data processing results are written back to the storage device during the element-by-element operation, thereby improving the efficiency of the element-by-element operation.

[0122] In some optional embodiments, for example Figure 7 As shown, step 205 may include:

[0123] In step 205-1, test performance parameters of the neural network model under conditions of multiple candidate data quantities are obtained.

[0124] In the embodiment of the present disclosure, the number of the plurality of candidate data may be an integer greater than 2, such as 3, 4 or more. The test performance parameter may at least be used to measure the performance of the element-by-element operation layer in the neural network model.

[0125] For example, when the number of candidate data is 3, the test performance parameters of the fusion layer when performing element-by-element operations on 3 candidate data are obtained, or the test performance parameters of the fusion layer and at least one network layer connected to the input and / or output of the fusion layer when processing 3 candidate data are obtained, or the test performance parameters of the optimized neural network model when processing 3 candidate data are obtained.

[0126] In a possible implementation, the testing performance parameter may include obtaining the running time for running a plurality of candidate data to obtain the processing result.

[0127] In step 205 - 2 , the target number is determined from the plurality of candidate data quantities based on the test performance parameter.

[0128] In the disclosed embodiment, the number of candidate data corresponding to the optimal test performance parameter can be selected as the target number based on the test performance parameter. For example, the number of candidate data includes 3, 4, 5, 6, 7 and 8, the test performance parameter includes the running time, and the neural network model has the shortest running time when the number of candidate data is 8, so the target number can be 8.

[0129] In the above embodiment, the target number can be determined from multiple candidate data quantities based on the test performance parameters of the neural network model under multiple candidate data quantity conditions, which is simple to implement and has high availability.

[0130] In some optional embodiments, the element-by-element operation of the first network layer includes a multiplication-addition operation based on a specific parameter, wherein the specified parameter may be a coeff parameter. Figure 8 As shown, the above step 203 may include:

[0131] In step 203-1, the first processing result and the at least one second input data read synchronously are written into the target number of first register devices.

[0132] In the embodiment of the present disclosure, the sum of the first processing result and the data quantity of at least one second input data is the target number, and the number of first storage devices is also the target number, then the target number of data read synchronously can be written into the target number of first storage devices.

[0133] In step 203-2, the specific parameters pre-written in at least one second storage device are multiplied and added with the data in at least one first storage device to obtain the second processing result.

[0134] In the disclosed embodiment, the specific parameters have been pre-written into different second storage devices, the number of the specific parameters is also the target number, and the number of the second storage devices is also the target number. The specific parameters in the second storage device are directly multiplied and added with the data in the first storage device, so as to obtain the second processing result corresponding to the processing of the element-by-element operation performed by the fusion layer.

[0135] In the above embodiment, after synchronously reading the first processing result and the at least one second input data based on the target number, the first processing result and the at least one second input data synchronously read can be written into the first storage device of the target number, and further, the specific parameter pre-written in the at least one second storage device is multiplied and added with the data in the at least one first storage device to obtain the second processing result. There is no need to continuously update the specified parameters stored in the second storage device, which reduces the overhead of the calculation process.

[0136] In some optional embodiments, the element-by-element operation of the first network layer includes a multiplication-addition operation based on specific parameters, such as Fig. 9 As shown, the above method may also include:

[0137] In step 301, if at least one specific parameter in the multiplication-addition operation includes a target value, the element-by-element operation of the first network layer is optimized to obtain the optimized element-by-element operation.

[0138] In the embodiment of the present disclosure, the target value may include but is not limited to 1, -1 and 0.

[0139] If the target value is 1, the optimization of the element-by-element operation may include no longer performing multiplication and addition operations on the data corresponding to the target value, but only performing sum operations. For example, the number of data read synchronously is 3, including A, B, and C respectively, and the element-by-element operation before the optimization is A×coeff_A+B×coeff_B+C×coeff_C, coeff_A is 1, and the element-by-element operation after the optimization is A+B×coeff_B+C×coeff_C.

[0140] If the target value is -1, the optimization of the element-by-element operation may include no longer performing multiplication and addition operations on the data corresponding to the target value, but only performing difference operations. For example, the number of data read synchronously is 3, including A, B, and C respectively, and the element-by-element operation before the optimization is A×coeff_A+B×coeff_B+C×coeff_C, coeffB is 1, and the element-by-element operation after the optimization is A×coeff_A-B+C×coeff_C.

[0141] If the target value is 0, the optimization of the element-by-element operation may include no longer performing any operation on the data corresponding to the target value. For example, the number of data read synchronously is 3, including A, B, and C respectively, and the element-by-element operation before the optimization is A×coeff_A+B×coeff_B+C×coeff_C, coeff_C is 0, then the element-by-element operation after the optimization is A×coeff_A+B×coeff_B.

[0142] In the above embodiment, when the element-by-element operation of the first network layer includes a multiplication-addition operation based on specific parameters, if at least one specific parameter in the multiplication-addition operation includes a target value, the element-by-element operation can be optimized to achieve the purpose of data optimization processing.

[0143] The above process is further illustrated as follows.

[0144] The element-by-element operations of the first network layer include multiplication and addition operations based on specific parameters. In order to ensure the versatility of the code, the common implementation is generally written directly in the form of coeff_A×A[0]+coeff_B×B[0]+C×coeff_C+…, but in some cases, the element-by-element operations can be optimized.

[0145] For example, when the coeff parameter corresponding to the input is 1, an addition operation is performed directly, that is, A[0]+B[0]+C[0]+…, to avoid the processor from performing a multiplication operation.

[0146] For another example, when the coeff parameter corresponding to an input is 0, the input is skipped directly. If there are two inputs A and BC, coeff_A is 1, and coeff_B is 0, then inputs A and C are directly copied to the output. For another example, if there are three inputs A, B, and C, coeff_A is 1, coeff_B is 0, and coeff_C is 1, then A[0]+C[0] is directly calculated without any operation on B.

[0147] For another example, when the coeff parameter corresponding to the input is -1, a subtraction operation is performed directly. Assume there are three inputs A and B, coeff_A is 1, coeff_B is -1, and coeff_C is 0, then A[0]–B[0] is directly calculated.

[0148] In some optional embodiments, the element-by-element operation of the first network layer includes a multiply-add operation, such as Fig.10 As shown, the above method may also include:

[0149] In step 401, a multiply-add instruction is received.

[0150] In an embodiment of the present disclosure, the neural network model may receive a multiply-add instruction. In one possible implementation, the multiply-add instruction is generated by a processor and sent to the neural network model.

[0151] In step 402, based on the multiply-add instruction, a multiply-add operation of the first network layer is performed.

[0152] In the disclosed embodiment, the neural network model can directly perform the multiplication and addition operation of the first network layer based on the received multiplication and addition instruction, and there is no need to perform the multiplication operation first and then the summation operation.

[0153] In the above embodiment, the processing process of the element-by-element operation of the first network layer by the neural network model can be further optimized through the multiplication and addition instructions, and the availability is high.

[0154] For example, when some of the coeff parameters corresponding to the input are 1, while others are not 1 or 0, since most processors now support multiplication and addition operations, in this case, multiplication and addition operations can be used to complete the operation. For example, there are three inputs A, B, and C, coeff_A is 1, coeff_B is greater than 1, and coeff_C is also greater than 1, then it is written as A[0]+coeff_B×B[0]+coeff_C×C[0]. This multiplication and addition operation is completed by multiplication and addition instructions, and there is no need to use two multiplication instructions and then an addition instruction. In this case, it can be achieved through multiplication and addition operations.

[0155] The disclosed embodiment also provides a data processing method that can be used in a neural network model that includes at least an element-by-element operation layer, such as Fig.11 As shown, Fig.11 A data processing method according to an exemplary embodiment includes the following steps:

[0156] In step 501 , a target number of first input data are read from a storage device.

[0157] In the embodiment of the present disclosure, the storage device may include but is not limited to the memory of the processor. The target number may be an integer greater than 2.

[0158] In step 502, the target number of first input data are read and processed by an element-by-element operation of a first network layer in a neural network model to obtain a first element-by-element processing result.

[0159] In the embodiment of the present disclosure, the neural network model includes at least a first network layer, which is a network layer that performs element-by-element operations on input data. The target number of first input data read can be processed element-by-element to obtain a first element-by-element processing result.

[0160] In step 503, the first element-by-element processing result is stored in the storage device.

[0161] In the embodiment of the present disclosure, the first element-by-element processing result is a processing result obtained after performing element-by-element operations on the target number of first input data read. The first element-by-element processing result can be stored in the above-mentioned storage device, such as a processor memory.

[0162] For example, the target number is N, where N is an integer greater than 2. The final first element-by-element processing result can be expressed as output[0], output[1], ... output[M]. In the case of calculating output[0], taking the sum operation as an example, first calculate the first two input data, that is, output[0] = input[0][0] + input[1][0]. Write output[0] back to the memory, then read output[0] from the memory and calculate with the next input data input[2][0], and get output[0] = output[0] + input[2][0]. And so on.

[0163] So when calculating input[0][0]+input[1][0]+input[2][0], it is necessary to perform three input reads and writes, one output read, and two output writes. The more input data there is, the more output reads and output writes there will be.

[0164] If the first input data of the target number is read each time and the element-by-element operation is performed, assuming the target number is 3, then output[0] = input[0][0] + input[1][0] + input[2][0]. Three input reads and one output write back are required. Obviously, the number of reads and write backs is reduced compared to the previous processing method.

[0165] In the above embodiment, a target number of first input data greater than 2 can be read from the storage device, and the target data first input data can be processed by element-by-element operation, which reduces the number of times the input data is read and the number of times the data processing results are written back to the storage device during the element-by-element operation, thereby improving the efficiency of the element-by-element operation.

[0166] In some optional embodiments, when the number of the plurality of data to be processed by the element-by-element operation is greater than a preset value, wherein the preset value may be less than or equal to the target number, that is, the total number of data to be processed by the element-by-element operation exceeds the target number, and the element-by-element operation of the first network layer cannot be completed by reading the target number of first input data at one time, then for example Fig.12 As shown, the above method may also include:

[0167] In step 504, a plurality of second input data is obtained from the storage device.

[0168] In an embodiment of the present disclosure, multiple second input data include the first element-by-element processing result, and the first element-by-element processing result has been previously stored in a storage device. At this time, it is necessary to read the first element-by-element processing result from the storage device again and use it as one of the multiple second input data. In addition, other input data that need to be operated element-by-element can also be read from the storage device as second input data.

[0169] For example, when calculating output[0], the target number is 3, and the number of the plurality of data to be processed is 5. Then, after calculating output[0]=input[0][0]+input[1][0]+input[2][0] and obtaining the first element-by-element processing result, the first element-by-element processing result output[0] is written to the storage device. Input[3][0], input[4][0], and the previous first element-by-element processing result are read from the storage device again as the plurality of second input data.

[0170] In step 505, the element-by-element operation is performed on the plurality of second input data to obtain a second element-by-element processing result.

[0171] The element-by-element operation is performed again on the plurality of second input data to obtain a second element-by-element processing result. If the number of data to be processed is greater, the element-by-element operation is performed multiple times in the above manner.

[0172] In the above embodiment, when the number of multiple data to be processed in the element-by-element operation is large, multiple second input data can be repeatedly acquired, and the multiple second input data include the first element-by-element processing result obtained by the last element-by-element operation, thereby obtaining the second element-by-element processing result, thereby reducing the number of times the input data is read and the number of times the data processing results are written back to the storage device during the element-by-element operation, thereby improving the processing efficiency of the element-by-element operation and having high availability.

[0173] The above process is further illustrated as follows.

[0174] When there are multiple first input data of the first network layer, assume that the number is N, and all inputs are represented as input[0], input[1]…input[N]. There are many elements in each input data. For example, the first element of the first input is represented as input[0][0]. The number of outputs is only 1, which is represented as output. Assume that the element-by-element operation of the first network layer here is a summation operation. The other two are similar and will not be described in detail. If the common element-by-element operation is implemented, it will be implemented as:

[0175] First add the first two inputs

[0176] output[0]=input[0][0]+input[1][0];

[0177] output[1]=input[0][1]+input[1][1]……

[0178] After the first two inputs are accumulated, the third is accumulated:

[0179] output[0]=output[0]+input[2][0];

[0180] output[1]=output[1]+input[2][1]……

[0181] And so on, until all inputs are accumulated.

[0182] From the above example, we can find that when the first two inputs are accumulated, the result will be written back to the output. When the third input is accumulated, the output data will be read from the storage device and the result will be written back to the output after the calculation is completed.

[0183] If the number of targets is greater than 2, assuming it is 3, we can get formula 1:

[0184] output[0]=input[0][0]+input[1][0]+input[2][0] Formula 1

[0185] Formula 1 includes three input reads and one output write back. The previous process of reading two elements each time includes three input reads, one output read, and two output write backs. And the more inputs there are, the more redundant output reads and write backs there are. It can be seen that if the target number of input data is read from the storage device, more read and write back operations can be saved.

[0186] In some optional embodiments, for example Fig.13 As shown, the above method may also include:

[0187] In step 500-1, test performance parameters of the neural network model under conditions of multiple candidate data quantities are obtained.

[0188] In the disclosed embodiment, it is best not to read all the data to be processed at once. For example, when the total number of data to be processed is 100, if 100 input data are read at one time, then output[0]=input[0][0]+input[1][0]+…+input

[99] [0].

[0189] However, in the storage device, all elements corresponding to the input data input[0] are stored first, then all elements corresponding to the input data input[1] are stored, and so on. When reading input[0][0] and input[1][0], they are read in strides. If the target number reaches 100, the stride is very large, which will cause the performance of the entire neural network model to drop sharply.

[0190] In an embodiment of the present disclosure, test performance parameters of the neural network model under conditions of multiple candidate data quantities can be obtained respectively. In an optional implementation, the multiple candidate data quantities are greater than 2, and the test performance parameters can at least be used to measure the performance of the first network layer in the neural network model.

[0191] For example, when the number of candidate data is 3, the test performance parameters of the processing result are obtained when the first network layer performs element-by-element operation on the 3 candidate data, or the first network layer and at least one network layer connected to the input end and / or output end of the first network layer process 3 candidate data to obtain the test performance parameters of the processing result, or the entire neural network model processes 3 candidate data to obtain the test performance parameters of the output result.

[0192] In an optional implementation, the test performance parameter may include the running time.

[0193] In step 500 - 2 , the target number is determined from the plurality of candidate data quantities based on the test performance parameter.

[0194] In the embodiment of the present disclosure, the number of candidate data with the best test performance parameters can be selected as the target number according to the test performance parameters.

[0195] For example, when the number of candidate data is 7, the test performance parameter is optimal. Assuming that the test performance parameter includes the running time, when the number of candidate data is 7, the running time is the shortest, then the target number is 7.

[0196] In the above embodiment, under the premise of ensuring the performance of the neural network, the number of reads and writes in the process of the first network layer performing element-by-element operations is reduced, thereby improving the performance and efficiency of the neural network model.

[0197] The above process is further illustrated as follows.

[0198] When the number of first input data is large, for example, N = 100, there are 100 first input data, if the elements at corresponding positions in the 100 inputs are processed at a time, for example:

[0199] output[0]=input[0][0]+input[1][0]+…+input

[99] [0] Formula 2

[0200] In the above formula 2, it is necessary to read the first element of each of the 100 first input data and then perform the calculation. For common processors, the performance will be better when the processor reads data stored continuously from the memory. If the stride of the stride reading data is too large, the performance will be degraded.

[0201] In a storage device, such as memory, all elements of input[0] are stored continuously first, then all elements of input[1] are stored continuously, and so on. When directly accessing the elements at corresponding positions in input[0] and input[1], they are read in strides. In the above example, when the number of the first input data is 100, the stride will be very large (when accessing the 100th one, the stride size is 99 times the length of a single input), resulting in a sharp drop in performance.

[0202] Therefore, in the embodiment of the present disclosure, the two forms of processing multiple inputs at a time and gradually accumulating are combined, and the first 10 can be processed first, and then the 10th to 20th can be processed until all 100 inputs are processed, for example:

[0203] output[0]=input[0][0]+input[1][0]+…+input[9][0];

[0204] output[1]=input[0][1]+input[1][1]+…+input[9][1];

[0205] output[2]=input[0][2]+input[1][2]+…+input[9][2]

[0206] …

[0207] output[0]=output[0]+input

[10] [0]+…+input

[19] [0];

[0208] output[1]=output[1]+input

[10] [1]+…+input

[19] [1];

[0209] output[2]=output[2]+input

[10] [2]+…+input

[19] [2];

[0210] …

[0211] In this way, multiple inputs are processed at one time, a certain number of redundant output read and write operations are saved, and long stride access is avoided. In addition, the number of multiple inputs that can be processed at one time depends on the performance of the specific processor, which can be obtained by obtaining the test performance parameters of the neural network model under multiple candidate data numbers, that is, trying to process different numbers of candidate data at one time, such as 2, 3, 4, 5, etc., and then selecting the candidate data number with the best test performance parameters as the target number.

[0212] In some optional embodiments, where the element-wise operation includes a multiply-add operation based on specific parameters, e.g. Fig.14 As shown, the above step 502 may include:

[0213] In step 502-1, the target number of first input data read are written into the target number of first register devices.

[0214] In the embodiment of the present disclosure, the target number of first input data read may be written into different first storage devices respectively.

[0215] In step 502-2, a multiplication and addition operation is performed on the specific parameters pre-written in the plurality of second storage devices and the data in the target number of first storage devices to obtain the first element-by-element processing result.

[0216] In the disclosed embodiment, the specific parameters have been pre-written into different second storage devices, the number of the specific parameters is also the target number, and the number of the second storage devices is also the target number. The specific parameters in the second storage device are directly multiplied and added with the data in the first storage device, so as to obtain the first element-by-element processing result corresponding to the element-by-element operation performed by the fusion layer.

[0217] In the above embodiment, after synchronously reading the target data first input data, the synchronously read target data first input data can be written into the target number of first storage devices, and further, the specific parameters pre-written in at least one second storage device are respectively multiplied and added with the data in at least one first storage device to obtain the first element-by-element processing result. The above process does not need to continuously update the specified parameters stored in the second storage device, thereby reducing the overhead of the calculation process.

[0218] The above process is further illustrated as follows.

[0219] When processing multiple first input data at one time, the element-by-element operation includes a multiplication and addition operation based on a specific parameter coeff. If the number of inputs processed at one time is too large, an optimization method can also be adopted in reading the coeff parameter. Generally, a processor has multiple registers for storing data. Assume that there are 32 registers available in the processor, including 16 first registers and 16 second registers. The 16 second registers can be used to store the coeff parameter, and the coeff parameters corresponding to the 16 first input data are stored respectively. Then, when calculating the 16 first input data, the 16 first input data are stored in the 16 first registers, and there is no need to read the coeff parameter (because it has been placed in the second register). The specific parameters pre-written in the multiple second storage devices are directly multiplied and added with the data in the target number of first storage devices to obtain the first element-by-element processing result.

[0220] If only 8 second registers are used to store the coeff parameter, and 16 first input data need to be calculated at one time, there will be a problem of repeatedly reading coeff. This is because after calculating the first 8 first input data, the coeff in the 8 registers needs to be replaced with the coeff parameter corresponding to the next 8 first input data, and so on. The second register storing coeff needs to be continuously updated, resulting in a certain amount of overhead.

[0221] When setting the number of registers for storing the coeff parameter here, it needs to be combined with the target number of first input data to be processed at one time. For example, if the test finds that the test performance parameters of processing 8 first input data at one time are optimal, and the number of available registers is much greater than 16, then 8 registers can be used to store the coeff parameter, and the other 8 registers can be used to store the first input data read.

[0222] If the test finds that the test performance parameter is optimal when processing 16 first input data at a time, but the number of available registers is less than 32, a compromise needs to be made to process 8 first input data at a time, which will result in relatively worse performance, but the coeff parameter will be improved. In short, the specific situation should be analyzed, and the best implementation method will be finally obtained by combining the test performance parameter and the number of registers.

[0223] In some optional embodiments, for example Fig.15 As shown, the above method may also include:

[0224] In step 506, a target multiply-add instruction is received.

[0225] In the disclosed embodiment, a target multiplication-addition instruction can be directly generated according to the element-by-element operation of the first network layer, and the processor sends the target multiplication-addition instruction to the neural network model. Alternatively, when the element-by-element operation of the first network layer includes a multiplication-addition operation based on specific parameters, if at least one specific parameter in the multiplication-addition operation includes a target value, the element-by-element operation of the first network layer can be optimized to obtain the optimized element-by-element operation, and further, the target multiplication-addition instruction is generated according to the optimized element-by-element operation and sent to the neural network model. The neural network model can receive the above-mentioned target multiplication-addition instruction.

[0226] The optimization process is the same as the optimization process in step 301 above, and will not be described again here.

[0227] Accordingly, step 502 may include:

[0228] According to the target multiplication-addition instruction, the target number of first input data read are processed by element-by-element operation of the first network layer in the neural network model.

[0229] In an embodiment of the present disclosure, the neural network model can directly perform element-by-element operations of the first network layer in the neural network model on the target number of first input data read according to the received target multiplication and addition instructions.

[0230] In the above embodiment, the processing process of the first network layer performing element-by-element operations on data can be further optimized through the target multiply-add instruction, and the usability is high.

[0231] In some optional embodiments, the target number of first input data includes at least one input data of a second network layer in the neural network model and at least one input data of the first network layer, wherein the output end of the second network layer is connected to the input end of the first network layer, for example Fig.16 shown.

[0232] Accordingly, step 502 may include:

[0233] In step 502-3, the operation processing of the second network layer is performed on at least one input data of the second network layer to obtain a first processing result.

[0234] In the embodiment of the present disclosure, the operation processing of the second network layer corresponds to the second network layer. For example, when the second network layer is a normalization layer, the operation of the second network layer includes a normalization operation. The operation processing of the second network layer is first performed on at least one input data of the second network layer to obtain a first processing result.

[0235] In step 502-4, an element-by-element operation is performed on the first processing result and at least one input data of the first network layer to obtain a first element-by-element processing result.

[0236] In the embodiment of the present disclosure, the first processing result and at least one input data of the first network layer are used as the target number of first input data, and element-by-element operation is performed to obtain a first element-by-element processing result.

[0237] In the above embodiment, the element-by-element operation of the first network layer can be integrated into the second network layer, and at least one input data of the second network layer in the neural network model and at least one input data of the first network layer can be integrated and processed, thereby reducing the execution time of the neural network model and improving the execution efficiency of the neural network model. At the same time, the data processing throughput of the neural network model per unit time is improved, and the delay and power consumption of processing the same data scale per unit time are reduced.

[0238] In some optional embodiments, when processing data, if the processor supports vectorized operations, vectorization should be used to implement element-by-element operations of the first network layer.

[0239] The processor can complete the calculation in the following formula with one vectorized instruction.

[0240] output[0]=input[0][0]+input[1][0]

[0241] output[1]=input[0][1]+input[1][1]

[0242] output[2]=input[0][2]+input[1][2]

[0243] output[3]=input[0][3]+input[1][3]

[0244] After vectorization:

[0245] output_vector=input[0]_vector+input[1]_vector

[0246] Since the vector length is 4, there are 4 elements in each vector.

[0247] In different processor architectures, the length of a vector may be different, and the present disclosure does not limit the range of the vector length.

[0248] Corresponding to the aforementioned method embodiments, the present disclosure also provides device embodiments.

[0249] like Fig.17 As shown, Fig.17 It is a block diagram of an optimization device for a neural network model shown in the present disclosure according to an exemplary embodiment, the device includes: a selection module 610, used to select at least two target network layers from the neural network model, wherein the at least two target network layers include a first network layer and a second network layer connected to the input end of the first network layer, and the first network layer is an element-by-element operation layer; a fusion processing module 620, used to fuse the at least two target network layers to obtain the optimized neural network model.

[0250] In some optional embodiments, the fusion processing module includes: fusing the element-by-element operation of the first network layer into the second network layer.

[0251] In some optional embodiments, the selection module includes: selecting a network layer with the deepest network depth from at least one network layer connected to an input end of the element-by-element operation layer as the second network layer.

[0252] In some optional embodiments, the second network layer is used to implement the first operation, and the apparatus further includes: a second reading module, used to read the first input data of the second network layer and at least one second input data of the element-by-element operation layer from a first storage device; a second data processing module, used to perform the first operation on the first input data through a fusion layer to obtain a first processing result, wherein the fusion layer is obtained by fusing the at least two target network layers; a third data processing module, used to perform the element-by-element operation of the element-by-element operation layer on the first processing result and the at least one second input data through the fusion layer to obtain a second processing result; and a first data writing module, used to write the second processing result to a second storage device, wherein the output result of the optimized neural network model is obtained based on the second processing result.

[0253] In some optional embodiments, the device also includes: a first determination module, used to determine the target number of data processed at one time by the fusion layer obtained by the fusion processing, and the target number is an integer greater than 2; the third data processing module includes: based on the target number, synchronously reading the first processing result and the at least one second input data, performing the element-by-element operation of the element-by-element operation layer to obtain a second processing result, wherein the sum of the data quantity of the first processing result and the at least one second input data is equal to the target number.

[0254] In some optional embodiments, the first determination module includes: obtaining test performance parameters of the neural network model under multiple candidate data quantity conditions; based on the test performance parameters, determining the target number from the multiple candidate data quantities.

[0255] In some optional embodiments, the element-by-element operation of the first network layer includes a multiplication-addition operation based on a specific parameter; the third data processing module includes: writing the first processing result and the at least one second input data read synchronously into the target number of first storage devices; performing multiplication-addition operations on the specific parameters pre-written in at least one second storage device and the data in at least one first storage device, respectively, to obtain the second processing result.

[0256] In some optional embodiments, the element-by-element operation of the first network layer includes a multiplication-addition operation based on specific parameters; the device also includes: an optimization processing module, which is used to optimize the element-by-element operation of the first network layer if at least one specific parameter in the multiplication-addition operation includes a target value, so as to obtain the element-by-element operation after optimization.

[0257] In some optional embodiments, the element-by-element operation of the first network layer includes a multiplication-addition operation, and the device further includes: a first receiving module for receiving a multiplication-addition instruction; and an execution module for executing the multiplication-addition operation of the first network layer based on the multiplication-addition instruction.

[0258] like Fig.18 As shown, Fig.18It is a block diagram of a data processing device shown in accordance with an exemplary embodiment of the present disclosure, and the device includes: a third reading module 710, used to read first input data and at least one second input data from a first storage device; a fourth data processing module 720, used to perform a first operation on the first input data through a fusion layer to obtain a first processing result, wherein the fusion layer is obtained by fusing a first network layer and a second network layer, the first network layer is an element-by-element operation layer, and the output end of the second network layer is connected to the input end of the first network layer; a fifth data processing module 730, used to perform an element-by-element operation on the first processing result and the at least one second input data through the fusion layer to obtain a second processing result; a second data writing module 740, used to write the second processing result to a second storage device.

[0259] like Fig.19 As shown, Fig.19 It is a block diagram of another data processing device shown in the present disclosure according to an exemplary embodiment, the device includes: a first reading module 810, used to read a target number of first input data from a storage device, where the target number is an integer greater than 2; a first data processing module 820, used to perform an element-by-element operation of a first network layer in a neural network model on the read target number of first input data to obtain a first element-by-element processing result; a storage module 830, used to store the first element-by-element processing result to the storage device.

[0260] In some optional embodiments, when the number of multiple data to be processed by the element-by-element operation is greater than a preset value, the device further includes: a data acquisition module, used to acquire multiple second input data from the storage device, wherein the multiple second input data include the first element-by-element processing result; and a sixth data processing module, used to perform the element-by-element operation on the multiple second input data to obtain a second element-by-element processing result.

[0261] In some optional embodiments, the device also includes: a parameter acquisition module, used to obtain test performance parameters of the neural network model under multiple candidate data quantity conditions; a second determination module, used to determine the target number from the multiple candidate data quantities based on the test performance parameters.

[0262] In some optional embodiments, the element-by-element operation includes a multiplication-addition operation based on specific parameters; the first data processing module includes: writing the target number of first input data read into the target number of first storage devices; performing multiplication-addition operations on the specific parameters pre-written in multiple second storage devices and the data in the target number of first storage devices to obtain the first element-by-element processing result.

[0263] In some optional embodiments, it also includes: a second receiving module for receiving a target multiplication and addition instruction; the first data processing module includes: according to the target multiplication and addition instruction, performing element-by-element operation of the first network layer in the neural network model on the target number of first input data read.

[0264] In some optional embodiments, the target number of first input data includes at least one input data of the second network layer in the neural network model and at least one input data of the first network layer, wherein the output end of the second network layer is connected to the input end of the first network layer; the first data processing module includes: performing operation processing of the second network layer on at least one input data of the second network layer to obtain a first processing result; performing element-by-element operation processing on the first processing result and at least one input data of the first network layer to obtain a first element-by-element processing result.

[0265] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art may understand and implement it without creative work.

[0266] The embodiments of the present disclosure also provide a computer-readable storage medium, which stores a computer program for executing any of the above-mentioned methods for optimizing the neural network model.

[0267] The embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute any of the above-mentioned data processing methods.

[0268] In some optional embodiments, the embodiments of the present disclosure provide a computer program product, including computer-readable code. When the computer-readable code is run on a device, a processor in the device executes instructions for implementing the operation optimization method provided in any of the above embodiments.

[0269] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0270] An embodiment of the present disclosure also provides an optimization device for a neural network model, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to call the executable instructions stored in the memory to implement any of the above-mentioned methods for optimizing the neural network model.

[0271] Fig. 20 A schematic diagram of the hardware structure of a neural network model optimization device provided in an embodiment of the present disclosure. The neural network model optimization device 910 includes a processor 911, and may also include an input device 912, an output device 913, and a memory 914. The input device 912, the output device 913, the memory 914, and the processor 911 are interconnected via a bus.

[0272] The memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or portable read-only memory (CD-ROM), which is used for related instructions and data.

[0273] The input device is used to input data and / or signals, and the output device is used to output data and / or signals. The output device and the input device can be independent devices or an integrated device.

[0274] The processor may include one or more processors, for example, one or more central processing units (CPUs). When the processor is a CPU, the CPU may be a single-core CPU or a multi-core CPU.

[0275] The memory is used to store program codes and data of the network device.

[0276] The processor is used to call the program code and data in the memory to execute the steps in the above method embodiment. For details, please refer to the description in the method embodiment, which will not be repeated here.

[0277] Understandably, Fig. 20 Only a simplified design of an optimization device for a neural network model is shown. In practical applications, the operation optimization device may also include other necessary components, including but not limited to any number of input / output devices, processors, controllers, memories, etc., and all optimization devices for the neural network model that can implement the embodiments of the present disclosure are within the protection scope of the present disclosure.

[0278] An embodiment of the present disclosure also provides a data processing device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to call the executable instructions stored in the memory to implement any of the above-mentioned data processing methods.

[0279] Fig.21 The hardware structure diagram of a data processing device provided in an embodiment of the present disclosure is shown in FIG. The data processing device 1010 includes a processor 1011, and may also include an input device 1012, an output device 1013, and a memory 1014. The input device 1012, the output device 1013, the memory 1014, and the processor 1011 are interconnected via a bus.

[0280] The memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or portable read-only memory (CD-ROM), which is used for related instructions and data.

[0281] The input device is used to input data and / or signals, and the output device is used to output data and / or signals. The output device and the input device can be independent devices or an integrated device.

[0282] The processor may include one or more processors, for example, one or more central processing units (CPUs). When the processor is a CPU, the CPU may be a single-core CPU or a multi-core CPU.

[0283] The memory is used to store program codes and data of the network device.

[0284] The processor is used to call the program code and data in the memory to execute the steps in the above method embodiment. For details, please refer to the description in the method embodiment, which will not be repeated here.

[0285] Understandably, Fig.21 Only a simplified design of a data processing device is shown. In practical applications, the operation optimization device may also include other necessary components, including but not limited to any number of input / output devices, processors, controllers, memories, etc., and all data processing devices that can implement the embodiments of the present disclosure are within the scope of protection of the present disclosure.

[0286] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0287] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A method for optimizing a neural network model, characterized in that: The neural network model is used to perform various computer vision tasks; the method comprises: Selecting at least two target network layers from the neural network model, wherein the at least two target network layers include a first network layer and a second network layer connected to an input end of the first network layer, the first network layer is an element-by-element operation layer; and the second network layer is used to implement a first operation; Fusing the at least two target network layers to obtain a fused layer, thereby obtaining the optimized neural network model; The data processing process of the optimized neural network model includes: Reading first input data of the second network layer and at least one second input data of the element-by-element operation layer from a first storage device; the first input data and the second input data are determined according to image data of a computer vision task performed by the neural network model; Performing the first operation on the first input data through the fusion layer to obtain a first processing result; Determine a target number of data to be processed by the fusion layer at one time, where the target number is an integer greater than 2; Based on the target number, synchronously read the first processing result and the at least one second input data, perform the element-by-element operation of the element-by-element operation layer, and obtain a second processing result, wherein the sum of the data quantities of the first processing result and the at least one second input data is equal to the target number; The second processing result is written into a second storage device, wherein the output result of the optimized neural network model is obtained based on the second processing result.

2. The method according to claim 1, characterized in that The fusing the at least two target network layers comprises: The element-by-element operations of the first network layer are fused into the second network layer.

3. The method according to claim 1 or 2, characterized in that: The selecting at least two target network layers from the neural network model comprises: From at least one network layer connected to the input end of the element-by-element operation layer, select the network layer with the deepest network depth as the second network layer.

4. The method according to claim 1, characterized in that The determining of the target number of one-time processing of the fusion layer includes: Obtaining test performance parameters of the neural network model under conditions of multiple candidate data quantities; The target number is determined from the plurality of candidate data quantities based on the test performance parameter.

5. The method according to claim 1 or 4, characterized in that: The element-by-element operation of the first network layer includes a multiplication-addition operation based on a specific parameter; The step of synchronously reading the first processing result and the at least one second input data based on the target number, performing the element-by-element operation of the element-by-element operation layer, and obtaining the second processing result includes: Writing the first processing result and the at least one second input data read synchronously into the target number of first register devices; The specific parameters pre-written in at least one second storage device are respectively multiplied and added with the data in at least one first storage device to obtain the second processing result.

6. The method according to claim 1, characterized in that The element-by-element operation of the first network layer includes a multiplication-addition operation based on a specific parameter; and the method further includes: If at least one specific parameter in the multiplication-addition operation includes a target value, the element-by-element operation of the first network layer is optimized to obtain the optimized element-by-element operation.

7. The method according to claim 1, characterized in that The element-by-element operation of the first network layer includes a multiplication-addition operation, and the method further includes: Receive multiply-add instructions; Based on the multiply-add instruction, a multiply-add operation of the first network layer is performed.

8. A data processing method, characterized in that: The method comprises: Read a target number of first input data from a storage device, where the target number is an integer greater than 2; the target number of first input data includes at least one input data of a second network layer and at least one input data of a first network layer in a neural network model, wherein an output end of the second network layer is connected to an input end of the first network layer; the first network layer is an element-by-element operation layer; the first network layer and the second network layer are fused to obtain a fused layer; the neural network model is used to perform various computer vision tasks; the input data of the first network layer and the second network layer are determined according to image data under the computer vision task performed by the neural network model; The target number of first input data are processed by an element-by-element operation of a first network layer in a neural network model to obtain a first element-by-element processing result, including: Performing operation processing of the second network layer on at least one input data of the second network layer through the fusion layer to obtain a first processing result; Performing element-by-element processing on the first processing result and at least one input data of the first network layer to obtain a first element-by-element processing result; wherein the sum of the data quantities of the first processing result and at least one input data of the first network layer is equal to the target number; The first element-by-element processing result is stored in the storage device.

9. The method according to claim 8, characterized in that In a case where the number of the plurality of data to be processed in the element-by-element operation is greater than a preset value, the method further includes: Acquire a plurality of second input data from the storage device, wherein the plurality of second input data includes the first element-by-element processing result; The element-by-element operation is performed on the plurality of second input data to obtain a second element-by-element processing result.

10. The method according to claim 8 or 9, characterized in that: The method further comprises: Obtaining test performance parameters of the neural network model under conditions of multiple candidate data quantities; The target number is determined from the plurality of candidate data quantities based on the test performance parameter.

11. The method according to claim 8, characterized in that The element-by-element operation includes a multiplication-addition operation based on specific parameters; The step of performing an element-by-element operation on the target number of first input data to obtain a first element-by-element processing result includes: Writing the target number of first input data read into the target number of first register devices; A multiplication and addition operation is performed on the specific parameters pre-written in the plurality of second storage devices and the data in the target number of first storage devices to obtain the first element-by-element processing result.

12. The method according to claim 8, characterized in that Also includes: receiving a target multiply-add instruction; The processing of performing an element-by-element operation of a first network layer in a neural network model on the read target number of first input data includes: According to the target multiplication-addition instruction, the target number of first input data read are processed by element-by-element operation of the first network layer in the neural network model.

13. An optimization device for a neural network model, characterized in that: The neural network model is used to perform various computer vision tasks; the device comprises: A selection module, used to select at least two target network layers from the neural network model, wherein the at least two target network layers include a first network layer and a second network layer connected to an input end of the first network layer, the first network layer is an element-by-element operation layer; and the second network layer is used to implement a first operation; A fusion processing module, used for fusing the at least two target network layers to obtain a fusion layer, thereby obtaining the optimized neural network model; The device also includes: A second reading module is used to read the first input data of the second network layer and at least one second input data of the element-by-element operation layer from the first storage device; the first input data and the second input data are determined according to the image data of the computer vision task performed by the neural network model; A second data processing module, configured to perform the first operation on the first input data through a fusion layer to obtain a first processing result, wherein the fusion layer is obtained by fusion processing the at least two target network layers; A first determining module is used to determine a target number of data to be processed at one time by the fusion layer obtained by the fusion process, where the target number is an integer greater than 2; A third data processing module is used to perform the element-by-element operation of the element-by-element operation layer on the first processing result and the at least one second input data through the fusion layer to obtain a second processing result, including: based on the target number, synchronously reading the first processing result and the at least one second input data, performing the element-by-element operation of the element-by-element operation layer to obtain the second processing result, wherein the sum of the data quantities of the first processing result and the at least one second input data is equal to the target number; The first data writing module is used to write the second processing result into a second storage device, wherein the output result of the optimized neural network model is obtained based on the second processing result.

14. A data processing device, characterized in that: The device comprises: A first reading module is used to read a target number of first input data from a storage device, wherein the target number is an integer greater than 2; the target number of first input data includes at least one input data of a second network layer and at least one input data of a first network layer in a neural network model, wherein an output end of the second network layer is connected to an input end of the first network layer; the first network layer is an element-by-element operation layer; the first network layer and the second network layer are fused to obtain a fused layer; the neural network model is used to perform various computer vision tasks; the input data of the first network layer and the second network layer are determined according to image data under the computer vision task performed by the neural network model; The first data processing module is used to perform an element-by-element operation of a first network layer in a neural network model on the first input data of the target number read to obtain a first element-by-element processing result, including: Performing operation processing of the second network layer on at least one input data of the second network layer through the fusion layer to obtain a first processing result; Performing element-by-element processing on the first processing result and at least one input data of the first network layer to obtain a first element-by-element processing result; wherein the sum of the data quantities of the first processing result and at least one input data of the first network layer is equal to the target number; A storage module is used to store the first element-by-element processing result in the storage device.

15. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the optimization method of the neural network model described in any one of claims 1-7 or the data processing method described in any one of claims 8-12.

16. An optimization device for a neural network model, characterized in that: include: processor; a memory for storing instructions executable by the processor; Wherein, the processor is configured to call the executable instructions stored in the memory to implement the optimization method of the neural network model described in any one of claims 1-7 or the data processing method described in any one of claims 8-12.

Images