Method for operating a neural network model, readable medium and electronic device

Abstract

The application relates to the field of artificial intelligence, and discloses a neural network model running method, readable medium and electronic equipment. The method is applied to the electronic equipment, and the method comprises the following steps: the electronic equipment predicts the size of a calculation result according to the size of input data and model parameters of each calculation layer; the size of a plurality of sub results is determined according to the size of the predicted calculation result; the electronic equipment determines the range of a data block corresponding to each sub result in the input data based on the size of each sub result and the model parameters of each calculation layer, then calculates each data block to obtain a sub result corresponding to each data block; and the electronic equipment determines a calculation result corresponding to the input data according to the sub result corresponding to each data block. In this way, the electronic equipment only needs to calculate part of data in the input data each time, a large amount of intermediate data is not generated at one time, and the occupation of the memory in the process of running the neural network model is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to a neural network model running method, readable medium and electronic device. BACKGROUND

[0002] With the rapid development of artificial intelligence (AI), the application of convolutional neural network model in the field of artificial intelligence is more and more extensive. In the calculation process of convolutional neural network, the data to be calculated is input into the convolutional layer, and then the calculation is performed layer by layer. Since the number of channels of the existing convolutional neural network is increasing, if all data channels are input at the same time, it will result in a large amount of data, and the entire convolutional calculation process will consume a large amount of storage and computing resources.

[0003] Due to the limited computing power of hardware devices (such as computer devices), in the calculation process of neural network, if the input data to be calculated is too much and the number of weight channels of each convolutional layer is too much, a lot of intermediate results will be generated, which will result in too high requirements for hardware devices. For some hardware devices with insufficient performance, too much input data and generated intermediate data may cause data overflow, resulting in calculation errors. SUMMARY

[0004] Therefore, the embodiments of the present application provide a neural network model running method, readable medium and electronic device.

[0005] In a first aspect, the embodiments of the present application provide a neural network model running method applied to an electronic device, wherein the neural network model comprises a plurality of calculation layers, and the method comprises: predicting the size of a calculation result tensor corresponding to an input data tensor according to the size of the input data tensor and the model parameters of each calculation layer; determining the size of a plurality of sub-result tensors according to the size of the calculation result tensor; determining the range of a data block corresponding to each sub-result tensor in the input data tensor based on the size of each sub-result tensor and the model parameters of each calculation layer; calculating each data block according to the model parameters of each calculation layer to obtain a sub-result tensor corresponding to each data block; and determining the calculation result tensor corresponding to the input data tensor according to the sub-result tensor corresponding to each data block.

[0006] By the method provided in the embodiments of the present application, the electronic device only needs to calculate part of the input data each time, and a large amount of intermediate data is not generated at one time, thereby reducing the occupation of the memory in the process of running the neural network model. The electronic device does not need to send the model to other computing devices for running, thereby reducing the occupation of network resources and the time delay caused by sending the model to other computing devices. Moreover, the neural network model can be deployed on the local device for running, and after the calculation of all the convolution layers is completed, the final result is written into the external memory, thereby reducing the number of times of accessing the external memory. Splitting the input data into multiple data blocks can also enable different data blocks to be calculated by different computing engines of the processor, and the different computing engines can be run synchronously, thereby improving the running efficiency.

[0007] In a possible implementation of the first aspect, the calculation layer is a convolution layer or a deconvolution layer, the convolution layer is configured to perform a convolution operation on the input data tensor, and the deconvolution layer is configured to perform a deconvolution operation on the input data tensor; and the model parameters include a step size of the convolution operation or the deconvolution operation, a size of a convolution kernel, and a number of padding blocks.

[0008] In a possible implementation of the first aspect, determining the range of the data block corresponding to each sub-result tensor in the input data tensor based on the size of each sub-result tensor and the model parameters of each calculation layer includes: determining the size of each data block according to the size of each sub-result tensor and the model parameters of each calculation layer; determining the size of the overlapping region between adjacent data blocks according to the size of each sub-result tensor and the model parameters of each calculation layer; and determining the range of the data block corresponding to each sub-result tensor in the input data tensor according to the size of each data block and the size of the overlapping region between adjacent data blocks.

[0009] In a possible implementation of the first aspect, determining the size of the overlapping region between adjacent data blocks according to the size of each sub-result tensor and the model parameters of each calculation layer includes: determining the size of the overlapping region between the data block corresponding to the current sub-result tensor and the data block corresponding to the previous sub-result tensor in the different dimension direction according to the size of the previous sub-result tensor in the different dimension direction and the model parameters of each calculation layer.

[0010] In a possible implementation of the first aspect, the M layers of the calculation layers are determined, and the size of the overlapping region between adjacent data blocks is determined according to the size of each sub-result tensor and the model parameters of each layer of the calculation layers. The size of the overlapping region between adjacent data blocks in the input data tensor corresponding to the Mth layer of the calculation layers is determined according to the size of each sub-result tensor and the model parameters of the Mth layer of the calculation layers. The input data tensor of the Mth layer of the calculation layers is determined as the output data of the (M-1)th layer of the calculation layers, and the size of the overlapping region between adjacent data blocks in the input data tensor corresponding to the (M-1)th layer of the calculation layers is determined, until the size of the overlapping region between adjacent data blocks in the input data tensor corresponding to the first layer of the calculation layers is determined.

[0011] In a possible implementation of the first aspect, each data block is calculated according to the model parameters of each layer of the calculation layers to obtain a sub-result tensor corresponding to each data block, including: sending the plurality of data blocks to different calculation engines respectively, and different calculation engines respectively calculate the received data blocks according to the model parameters of each layer of the calculation layers to obtain a sub-result tensor corresponding to each data block.

[0012] In a possible implementation of the first aspect, the calculation engine is a calculation engine of an electronic device processor, or the calculation engine is a calculation engine of another electronic device processor.

[0013] In a possible implementation of the first aspect, the calculation engine is a calculation engine of an electronic device processor, or the calculation engine is a calculation engine of another electronic device processor.

[0014] In a possible implementation of the first aspect, the calculation engine is a calculation engine of an electronic device processor, or the calculation engine is a calculation engine of another electronic device processor.

[0015] In a possible implementation of the first aspect, the calculation engine is a calculation engine of an electronic device processor, or the calculation engine is a calculation engine of another electronic device processor. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 According to some embodiments of the present application, an architecture schematic diagram of a convolutional neural network is shown.

[0017] Figure 2According to some embodiments of the present application, a schematic diagram of a calculation process of a convolution operation is shown.

[0018] Figure 3 According to some embodiments of the present application, a schematic diagram of a calculation process of a deconvolution operation is shown.

[0019] Figure 4 According to some embodiments of the present application, a schematic diagram of deploying a convolutional neural network is shown.

[0020] Figure 5 According to some embodiments of the present application, a schematic diagram of an overlapping area in convolution calculation is shown.

[0021] Figure 6 According to some embodiments of the present application, a schematic diagram of a data block division method is shown.

[0022] Figure 7 According to some embodiments of the present application, a schematic diagram of a running method of a neural network model is shown.

[0023] Figures 8A-8C According to some embodiments of the present application, a schematic diagram of a calculation process of a three-layer convolutional layer is shown.

[0024] Figures 9A-9C According to some embodiments of the present application, a schematic diagram of calculating a data block start position is shown.

[0025] Figure 10 According to some embodiments of the present application, a schematic diagram of a running device of a neural network model is shown.

[0026] Figure 11 According to some embodiments of the present application, a schematic diagram of an electronic device is shown. DETAILED DESCRIPTION

[0027] The illustrative embodiments of the present application include, but are not limited to, a running method of a neural network model, a readable medium, and an electronic device.

[0028] For ease of understanding, first introduce the terms involved in the embodiments of the present application.

[0029] A Convolutional Neuron Network (CNN) is a deep neural network with convolutional structure, and is a deep learning architecture, which means that the algorithm of machine learning is used to learn at different levels of abstraction. As a deep learning architecture, CNN is a feed-forward artificial neural network, which can be applied to image processing, natural language processing and computer vision.

[0030] The structure of CNN is described below by taking the application of CNN in image processing as an example.

[0031] Figure 1 According to some embodiments of the present application, a structural diagram of a CNN model is shown. As shown in the figure, the CNN model can include an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer, wherein the pooling layer is optional. Figure 1

[0032] The input layer is used for pre-processing the input image, such as normalization operation, mean removal operation, principal component analysis (PCA) dimensionality reduction operation.

[0033] The convolutional layer can include multiple layers of convolutional calculation, as shown in the figure. Figure 1 The convolutional layer can include, for example, 3 layers of convolutional layer, such as conv1, conv2 and conv3. The convolutional layer can include many convolutional operators, also known as convolutional kernels, which are equivalent to a filter for extracting specific information from the input image. The convolutional kernel is essentially a weight matrix, and the depth dimension of the weight matrix is the same as that of the input image. The weight values in the weight matrix need to be obtained through a large number of training in practical application. The weight values obtained through training form each weight matrix, which can extract information from the input image, thereby helping the convolutional neural network to make correct prediction.

[0034] ​The calculation process of the convolution layer is that the convolution kernel is convolved with the data of the kernel size in the input image. After the data of one kernel size is calculated, the next kernel size data is calculated by moving a stride. The final result obtained is the feature map of the convolution layer. It should be understood that the feature map obtained by each layer of convolution layer in the convolution operation will become smaller and smaller. In order to ensure the size of the feature map and to increase the number of times that the pixel points at the edge of the image are used in the convolution layer, padding is added to the edge of the input image.

[0035] The pooling layer is used to reduce the calculation amount of the convolutional neural network. The pooling layer is not shown in Figure 1 . In the image processing process, the purpose of the pooling layer is to reduce the spatial size of the image. The pooling layer can be periodically introduced after the convolution layer, which can be one convolution layer followed by one pooling layer, or multiple convolution layers followed by one or more pooling layers. The pooling layer can include an average pooling operator and / or a maximum pooling operator for sampling the input image to obtain a smaller size image. The average pooling operator can calculate the average value of the pixel values in the image within a certain range. The maximum pooling operator can take the pixel with the maximum value in the range as the result of maximum pooling.

[0036] The fully connected layer is used to integrate the local information with class distinction in the convolution layer or the pooling layer to generate the final output information (the required class information or other related information). The fully connected layer can include multiple layers. The parameters of each layer of the fully connected layer can be pre-trained according to the related training data of a specific task type, for example, the task type can include image recognition, image classification, image super-resolution reconstruction, etc.

[0037] The output layer uses an activation function to map the input of the neuron to the output. At the same time, the output layer also has a loss function similar to the classification cross-entropy, which is specifically used to calculate the prediction error. Once the forward propagation of the entire convolutional neural network is completed, the back propagation starts to update the weight values and biases of the layers mentioned above to reduce the loss of the convolutional neural network and the error between the result output by the output layer of the convolutional neural network and the ideal result.

[0038] It should be noted that the convolutional neural network shown in Figure 1 is only an example of a convolutional neural network. In specific applications, the convolutional neural network can also exist in the form of other network models.

[0039] In some embodiments, the neural network model further comprises a deconvolution neural network, which comprises a de-pooling layer, a correction layer and a deconvolution layer. The de-pooling layer corresponds to the pooling layer in the CNN, the correction layer corresponds to the activation function in the CNN, and the deconvolution layer corresponds to the convolution layer in the CNN. The deconvolution can also be referred to as transpose convolution.

[0040] The convolution layer and the deconvolution layer can be collectively referred to as a calculation layer. The convolution layer performs convolution operation, and the deconvolution layer performs deconvolution operation. The calculation processes of the convolution operation and the deconvolution operation are illustrated below by taking an image as an example. In the example, Figure 2 and Figure 3 In the example, the kernel size in each of the above formulas is 3*3. Figure 2 In the example, the input image is a two-dimensional matrix of 4*4. Figure 3 In the example, the input image is a two-dimensional matrix of 2*2. It should be understood that the input image is two-dimensional, and the kernel should also be two-dimensional.

[0041] As shown in Figure 2 , in the convolution operation, the input image is 4*4, the kernel size is 3*3, and the stride is 1. As shown in Figure 2 , the kernel first performs convolution on the data in the first 3*3 units of the input image to obtain the calculation result 1. Then, as shown in Figure 2 , according to the stride equal to 1, the data in the last 3*3 units is calculated by moving one unit to the right to obtain the calculation result 2. In this way, as shown in Figure 2 , and Figure 2 , the calculation result 3 and the calculation result 4 are obtained. Finally, the calculation result 1 to the calculation result 4 are the feature map output by the convolution layer of the layer.

[0042] As shown in Figure 3 , in the deconvolution operation, the input image is 2*2, the kernel size is 3*3, the stride is 1, and two rows of padding are filled around the input image. Referring to the convolution process shown in Figure 2 , the final output feature map will have a size of 4*4.

[0043] In some embodiments, in the input layer, if the input image is a grayscale image, only one feature map will be obtained, and if the input image is a color image, three feature maps of red, green and blue channels will be obtained, i.e., a three-dimensional matrix.

[0044] That is to say, after the convolution operation, the output feature map is usually smaller than the input image, and after the deconvolution operation, the output feature map is usually larger than the input image. It should be understood that the above Figure 2 and Figure 3 The above is described by taking the convolution operation without padding and the deconvolution operation with padding as an example. Padding can also be performed on the input image in the convolution operation, which is not limited in the present application.

[0045] As can be seen, in the calculation process of the calculation layer, whether it is a convolution operation or a deconvolution operation, all the data to be calculated are input into the calculation layer and then calculated. Usually, the amount of data input into the calculation layer is large. For example, if the input data tensor is [1, 1024, 512, 3], the memory size occupied by the input data is at least 1*1024*512*3 bytes (byte), i.e. 1.5 megabytes (MB). A large amount of intermediate calculation results will be generated in the calculation of the input data in the calculation layer. If the input data is input into the terminal device for running, a large amount of data will be generated, and the convolution calculation process will require a large amount of memory of the terminal device. For some hardware devices with insufficient performance, too much input data and generated intermediate data can cause data overflow, resulting in calculation errors.

[0046] In some embodiments, when the electronic device 100 is a terminal device 101 with small memory, the terminal device 101 can also deploy the neural network model to a cloud server or an edge device for execution to reduce the occupation of the terminal device memory. As shown in Figure 4 The terminal device 101 can hand over the running of the neural network model to the cloud server 200 or send it to the edge computing device 300 for execution. After the cloud server 200 and / or the edge computing device 300 obtain the running result of the neural network model, the running result is returned to the terminal device 101.

[0047] The cloud server 200 is an entity that provides cloud services to users by utilizing basic resources in a cloud computing mode and has a large amount of basic resources (including computing resources, storage resources and network resources). The edge computing device 300 refers to an edge computing device in an edge environment or a software system running on one or more edge computing devices. The edge computing device 300 is geographically close to the terminal device 101, such as an edge computing all-in-one machine located on both sides of the road and a county-level edge server.

[0048] However, the terminal device 101 will take a long time to send the input data of the neural network model to the cloud server 200 for running, and will occupy a large amount of network resources. If the input data is sent to the edge computing device 300 for running, the memory of the edge server 300 is usually insufficient to support the convolution operation and the pooling operation of a large amount of data.

[0049] In some other embodiments, when the memory of the computing device running the neural network model is insufficient, the input data and intermediate calculation results can also be stored in an external memory. When running the neural network model, the above-mentioned data slices are read into the memory, for example, the red-green-blue 3-channel input of a color image is read, after the data of one channel is calculated and the corresponding feature map is obtained, the feature map is written to the external memory, and then the data of the next channel is read and calculated. However, if the number of layers of the computing layer is large, for example, there are multiple convolution layers, then the external memory needs to be frequently accessed, which will occupy a large amount of bus bandwidth and affect the normal operation of other applications of the device. Moreover, if the amount of data of one channel is large, the above-mentioned method will still occupy a large amount of memory when running the neural network model.

[0050] In order to solve the problem that the running of the neural network model occupies a large amount of memory, the present application provides a method for running a neural network model, which is applied to a neural network model with multiple computing layers. The multiple computing layers can be multiple convolution layers or multiple deconvolution layers. The method determines the size of the output result according to the size of the input data tensor and the model parameters of each computing layer, and determines the number of blocks into which the output result tensor needs to be divided in each dimension direction, i.e., the size of the multiple sub-result tensors. Then, the corresponding region (data block) of each sub-result tensor in the input data tensor is determined, and then the electronic device 100 performs the operation of the multiple computing layers on each data block to obtain each sub-result tensor, and finally combines each sub-result tensor to obtain the output result. The input data tensor and the output result tensor can be a first-order tensor, a second-order tensor or a multi-dimensional tensor. Hereinafter, the input data tensor and the output result tensor are referred to as input data and output result.

[0051] In this way, the electronic device 100 only needs to calculate part of the input data each time, and does not generate a large amount of intermediate data at one time, thereby reducing the memory occupation during the running of the neural network model. The electronic device 100 does not need to send the model to other computing devices for running, thereby reducing the occupation of network resources and the time delay caused by sending the model to other computing devices. Moreover, the neural network model can be deployed on the local device for running, and after the calculation of all the convolution layers is completed, the final result is written into the external memory, thereby reducing the number of times of accessing the external memory. Splitting the input data into multiple data blocks can also enable different data blocks to be calculated by different computing engines of the processor, and the different computing engines can be run synchronously, thereby improving the running efficiency.

[0052] It can be understood that, since the convolution kernel and the data in the input data are calculated each time, the data participating in the calculation each time and the data participating in the calculation before that time overlap, as shown in Figure 2 , the data participating in the calculation of (A) in Figure 2 and the data participating in the calculation of (B) in Figure 2 overlap when obtaining the calculation result 1 and the calculation result 2, respectively. Therefore, the running method of the neural network model provided in the present application can determine the data block corresponding to the first sub-result in the input data according to the model parameters of each calculation layer and the size of the first sub-result. Then, the overlapping region of the data block corresponding to each sub-result and the data block corresponding to the previous sub-result in the different dimension directions is determined to obtain the data block of each sub-result. Specifically, the overlapping region can be calculated according to the input size of the previous data block in the different dimension directions, the size of the output result of the previous data block in the last convolution layer, and the model parameters of each convolution layer. First, the overlapping region of the adjacent data blocks in the input of the last convolution layer is calculated, and then the size of the overlapping region of the adjacent data blocks in the input (i.e., the input data) of the first convolution layer is determined from the last convolution layer to the first convolution layer, thereby obtaining the starting position of the data block corresponding to the sub-computation result in the input data. Then, the size of the corresponding data block is determined according to the size of the sub-result, and the range of the data block corresponding to the sub-result is determined according to the starting position of the data block corresponding to the sub-result in the input data and the size of the data block. The determination method of the data block corresponding to each sub-result in the input data will be described below, and will not be described herein.

[0053] For example, as shown in Figure 5As shown, the calculation result is five data A30-A34, if A30-A32 is regarded as a sub-result 1 and A33-A34 is regarded as a sub-result 2. Then according to the sub-result 1 and the model parameters of the second layer convolution layer, it can be determined that the data A20-A23 in the second layer convolution layer participating in the calculation of the sub-result 1 should be, and according to the sub-result 2 and the model parameters of the second layer convolution layer, it can be determined that the data A23-A25 in the second layer convolution layer participating in the calculation of the sub-result 2 should be, that is, there is an overlapping area A23.

[0054] According to A20-A23 and the model parameters of the first layer convolution layer, it can be determined that the data A10-A15 in the input data participating in the calculation of A20-A23 should be, and according to A23-A25 and the model parameters of the first layer convolution layer, it can be determined that the data A14-A18 in the input data participating in the calculation of A23-A25 should be, that is, there is an overlapping area A14-A15.

[0055] That is, for splitting the calculation result into multiple sub-results, the input data corresponding to the sub-results will have an overlapping area. Therefore, in order to avoid the situation that some data is missed, the overlapping area between the data blocks needs to be calculated, and the starting position of the next data block is determined according to the overlapping area.

[0056] Similarly, for two-dimensional input data, in order to ensure that no data is missed when performing convolution operation on each data block, the overlapping area between adjacent data blocks in different dimensions needs to be determined according to the division of the calculation result. For example, as shown in FIG. 1B, Figure 6 As shown, the input data is 8*8 two-dimensional data, and the input data needs to be calculated by multiple layers of convolution. If according to the size of the calculation result, the electronic device 100 determines to divide the calculation result into two sub-results in the height direction and the width direction, that is, the electronic device 100 will obtain four sub-results respectively, then for the input data, the electronic device 100 calculates four data blocks respectively, according to the size of the sub-result 1, the range of the corresponding data block 1 can be determined according to the model parameters of each layer, for example, the range of the data block 1 is as shown in FIG. 1B(A). Figure 6

[0057] ​The electronic device 100 determines the region of the data block 2 corresponding to the sub-result 2, and calculates the size of the overlapping region between the data block 2 and the previous data block in both the height direction and the width direction, respectively. The previous data block of the data block 2 in the width direction is the data block 1, and the size of the overlapping region between the data block 2 and the data block 1 in the width direction can be determined according to the width of the data block 1, the width of the output result of the data block 1, and the values of the model parameters of each convolutional layer in the width direction. Since the data block 2 is also the first data block in the height direction, the range of the data block 2 in the height direction corresponds to that of the data block 1, and the size of the overlapping region between the data block 2 and the data block 1 in the height direction is 0. For example, when the size of the overlapping region between the data block 1 and the data block 2 in the width direction is 1, the region of the data block 2 is as shown in (B) of FIG. 6. Figure 6

[0058] The electronic device 100 determines the region of the data block 3 corresponding to the sub-result 3, and the previous data block of the data block 3 in the height direction is the data block 1, and the size of the overlapping region between the data block 3 and the data block 1 in the height direction can be determined according to the height of the data block 1, the height of the output result of the data block 1, and the values of the model parameters of each convolutional layer in the height direction. Since the data block 3 is also the first data block in the width direction, the range of the data block 3 in the width direction corresponds to that of the data block 1. For example, when the size of the overlapping region between the data block 3 and the data block 1 in the height direction is 1, the region of the data block 3 is as shown in (C) of FIG. 6. Figure 6

[0059] The electronic device 100 determines the region of the data block 4 corresponding to the sub-result 4, and the previous data block of the data block 4 in the height direction is the data block 2, and the size of the overlapping region between the data block 4 and the data block 1 in the height direction can be determined according to the height of the data block 2, the height of the output result of the data block 2, and the values of the model parameters of each convolutional layer in the height direction. The previous data block of the data block 4 in the width direction is the data block 3, and the size of the overlapping region between the data block 4 and the data block 3 in the width direction can be determined according to the width of the data block 3, the width of the output result of the data block 3, and the values of the model parameters of each convolutional layer in the width direction. For example, when the size of the overlapping region between the data block 4 and the data block 3 in the width direction is 1, and the size of the overlapping region between the data block 4 and the data block 2 in the height direction is 1, the region of the data block 4 is as shown in (D) of FIG. 6. Figure 6

[0060] ​​​The electronic device 100 in the embodiments of the present application can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, a notebook computer, a virtual reality device such as a VR / AR, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), or a dedicated camera (for example, a single-lens reflex camera or a card camera), and the like. The specific type of the terminal device is not limited in the present application.

[0061] The running method of the neural network model provided in the present application will be described in detail below. Figure 7 The running method of the neural network model provided in the present application will be described in detail below.

[0062] S710: Determine the size of the output result according to the size of the input data and the model parameters of each calculation layer.

[0063] The input data refers to the preprocessed data obtained by the electronic device 100 according to the CNN input layer, that is, the data input into the calculation layer. The calculation layer can be a convolution layer or a deconvolution layer, and the model parameters can be related parameters required for convolution operation or related parameters required for deconvolution operation, such as kernel size and stride. In some embodiments, the input data also performs padding operation.

[0064] The electronic device 100 can determine the size of the output result obtained after the input data is calculated by all calculation layers according to the size of the input data and the model parameters of each calculation layer.

[0065] The calculation process of the size of the obtained output result will be described in detail below. The calculation process of the output result in the width direction can refer to the following formula (1), where width(t) is the width size of the input data of the tth layer, out w (t) is the width size of the data after the convolution operation of the input data of the tth layer, kernel_size w (t) is the size of the convolution kernel in the width direction of the convolution operation of the tth layer, pad w (t) is the data size of the padding pad in the width direction of the tth layer, stride w (t) is the convolution step of the tth layer.

[0066]

[0067] It should be understood that in the embodiments of the present application, the calculation process is described by taking the width as an example, and the calculation process of the height or depth can refer to the calculation method of the width, which will not be described again.

[0068] S720: determining the size of the plurality of sub results according to the size of the output result.

[0069] According to the size of the output result, the electronic device 100 will determine how many sub results the output result will be divided into, and further determine the number of data blocks, so that the electronic device 100 can obtain the corresponding sub results by calculating the data blocks respectively. For example, the electronic device 100 obtains the size of the output result as a two-dimensional matrix of 16*16, and the electronic device 100 can divide the output result into 4 sub results according to the size, and then the electronic device 100 will calculate the data blocks corresponding to the 4 sub results in the input data respectively, and obtain the 4 sub results respectively.

[0070] It should be understood that when the electronic device 100 divides the sub results, the division in the same dimension direction should be the same, for example, for two-dimensional input data, the output result is divided into 2 blocks in the height direction and 3 blocks in the width direction, then the height of the sub results in the same row should be the same, and the width of the sub results in the same column should be the same.

[0071] S730: determining the area of each data block corresponding to each sub result in the input data according to the size of the sub result.

[0072] The electronic device 100 will determine the size of each data block according to the size of each sub result, and then determine the overlapping area of the data block and the adjacent data block in different dimension directions according to the size of each data block, the size of the corresponding sub result and the model parameters of each calculation layer, and then determine the starting position of the data block corresponding to the sub result in the input data according to the overlapping area between the data block and the previous data block in different dimension directions, and then determine the range of the second data block according to the starting position of the data block corresponding to the sub result in the input data and the size of the data block.

[0073] Specifically, the electronic device 100 determines the size of each data block based on the size of each sub-result. Then, it determines the preceding data block in different dimensions. Based on the size of the preceding data block in different dimensions, the size of the corresponding sub-result, and the model parameters of each convolutional layer, it calculates the overlapping area between the data block and the preceding data block in the input of the last convolutional layer in different dimensions. Then, starting from the last convolutional layer upwards, it determines the size of the overlapping area between adjacent data blocks in the input of the first convolutional layer (i.e., the input data). By identifying the overlapping area between the data block and the preceding data block in different dimensions, and based on this overlapping area, it determines the starting position of the data block corresponding to the sub-result in the input data, thereby determining the range of each data block.

[0074] The following describes in detail the calculation process of the overlapping area between the data block and the previous data block in different dimensions, as well as the starting position of the data block, when the electronic device 100 performs a convolution operation at the computing layer.

[0075] The following describes the overlap in the width direction. w (t) An example is given to illustrate the calculation of the overlapping region. It should be understood that the calculation process for the height and depth directions can be referenced from the calculation process for the width direction. The size of the overlapping region of adjacent data blocks in the width direction is shown in the diagram. w The specific calculation process of (t) can be referred to the following formula (2), where 1≤t≤T and t is a positive integer, and T is the number of convolutional layers. w (t) represents the width of the input data corresponding to the previous data block in the t-th convolutional layer, w w The calculation method for (t) can be found in formula (3) below. pos w (t+1) represents the size of the non-overlapping portion between the computation result of the previous data block after the t-th convolutional layer and the computation result of the next data block after the t-th convolutional layer. w (t) represents the number of pads in the width direction of the previous data block in the t-th convolutional layer.

[0076] overlap w (t)=w w (t)-pos w (t+1)×stride w (t)+pad w (t) (2)

[0077] w w (t)=(out w (t)-1)×stride w(t) + kernel_size w (t) - pad w (t) (3)

[0078] According to overlap w (t) and the position l of the last data of the previous data block w (t) can determine the corresponding starting position p of the next data block in the width direction in the t-th layer of the convolutional layer w (t) can be determined according to the following formula (4).

[0079] p w (t) = l w (t) - overlap w (t) (4)

[0080] According to out w (t-1) and overlap w (t) can determine pos w (t), wherein out w (t-1) is equal to w w (t), and then pos w (t) is brought into the above formula (2), and so on, so that the starting position of each data block in the first layer of the convolutional layer can be finally determined.

[0081] In some embodiments, when the electronic device 100 performs the convolution operation on the data block, the electronic device 100 also performs the pooling operation on the data block, but in the pooling operation, there is a case that the size of the data input into the pooling layer is not an integer multiple of the kernel_size, and the pooling layer also performs the pooling operation on the data less than kernel_size, that is, the ceil operation is performed on the input data. In the above case, the result calculated according to formula (3) also needs to be correspondingly reduced by the extra data caused by the ceil operation.

[0082] The following describes in detail the calculation process of the overlapping area of the data block with the previous data block in different dimension directions and the starting position of the data block when the electronic device 100 performs the deconvolution operation.

[0083] The calculation process of the overlapping area of the data block with the previous data block in the width direction is taken as an example. According to the size of the output result of the current data block and the size of the convolution kernel kernel_size w ’(t) determines the size of the data input into the deconvolution operation, wherein the calculation process in the width direction can be specifically referred to the following formula (5).

[0084]

[0085] w_int w (t) is the intermediate value in the width direction in the process of calculating the size of the input data of the deconvolution operation of the layer (t), stride_int w newpad w ’(t) is determined according to the kernel_size and the actual padding number in the width direction of the deconvolution operation. newpad w The calculation process of newpad

[0086] newpad w ’(t) = kernel_size w ’(t) - pad w ’(t) - 1 (6)

[0087] Then, the intermediate value overlap_int w (t) in the width direction in the process of calculating the size of the repeated region is determined according to formula (7), and the final size of the repeated region also needs to be calculated according to overlap_int w (t) to be determined.

[0088] overlap_int w (t) = w_int w (t) - pos’(t) x stride_int w + newpad’(t) (7)

[0089] w_int w (t) and overlap_int w (t) obtained by the above calculation are adjusted to obtain the size w w ’(t) and overlap w ’(t) of the data actually input into the deconvolution operation in the width direction. The process of adjusting w_int w (t) to obtain w w ’(t) can be referred to in formula (8) below, and the process of adjusting overlap_int w (t) to obtain overlap w ’(t) can be referred to in formula (9) below, where left is initially equal to 0, stride w ’(t) is the step length of the deconvolution operation of the t-th layer.

[0090]

[0091]

[0092] where (w_intw (t) - left) / stride w (t) represents w_int w the value of (t) - left divided by stride w (t) and rounded down, stride w (t)! = 0 represents stride w (t) is not equal to 0, (w_int w (t) - left) % stride w (t) represents w_int w the value of (t) - left divided by stride w (t) is taken modulo.

[0093] Then, the intermediate parameter remain in the width direction is adjusted w , the calculation process of remain w may refer to the following formula (10).

[0094]

[0095] According to the intermediate parameter remain, a new left is obtained, the adjusted left can refer to the following formula (11), and the adjusted left is used for calculation of the repeated area of the next layer of deconvolution.

[0096] left (t-1) = remain w (t) - 1 + stride w (t) x (remain w (t) == 0) (11)

[0097] Finally, according to formula (4), the starting position of the data block can be determined according to the size of the overlap w (t) between the data block and the previous data block in the width direction.

[0098] In addition, according to the running method of the neural network model provided in the present application, the calculation of the padding of the intermediate block is also adjusted according to the input data to determine a plurality of data blocks, wherein, taking the width direction as an example, the padding on the left side of the data block is recorded as pad_l, and the padding on the right side of the data is recorded as pad_r. The calculation process of pad_r can refer to the following formula (12), and the calculation process of pad_l can refer to the following formula (13), wherein, w out (t) is the width size of the calculation result of the data block output at the t layer, w pos (t) is the width size of the input data corresponding to the data block at the t layer of convolution layer.

[0099] pad_r = kernel_size w '(t) - (w out '(t) + pad_l - (w pos '(t) - 1) x stride w '(t) (12)

[0100]

[0101] S740: Calculate the corresponding sub-results for each data block respectively, and combine all the sub-results to obtain the output result.

[0102] The electronic device 100 calculates each data block according to the model parameters of each calculation layer. Specifically, the electronic device 100 can perform a convolution operation or an inverse convolution operation on the first data block according to the model parameters, and then obtain the corresponding sub-result. After the electronic device 100 combines the sub-results, the final calculation result can be obtained.

[0103] In some embodiments, the electronic device 100 can also separately send each data block to different calculation engines of the processor for calculation. The different calculation engines can run synchronously, improving the running efficiency.

[0104] In some embodiments, after the electronic device 100 calculates each data block to obtain the final calculation result, the electronic device 100 further performs a pooling operation on the first data block.

[0105] In summary, according to the running method of the neural network model provided in the present application, the electronic device 100 only needs to calculate part of the input data each time, and does not generate a large amount of intermediate data at one time, thereby reducing the occupation of the memory during the running of the neural network model. The electronic device 100 does not need to send the model to other computing devices for running, thereby reducing the occupation of network resources and the time delay caused by sending the model to other computing devices. Moreover, the neural network model can be deployed on the local device for running, and after the calculation of all the convolution layers is completed, the final result is written into the external memory, thereby reducing the number of times of accessing the external memory. Splitting the input data into multiple data blocks can also send different data blocks to different calculation engines of the processor for calculation. The different calculation engines can run synchronously, thereby improving the running efficiency.

[0106] For ease of understanding, the following describes the calculation process of the starting position of the data block in the width direction when the electronic device 100 performs a convolution operation. The CNN has three convolution layers (conv1, conv2, and conv3) as an example.

[0107] First, the input data is data 0-data 32, i.e., the input data width is equal to 33, and the model parameters of conv1 are stride w (1) = 2, kernel_size w (1) = 3, and the model parameters of conv2 are stride w (2) = 1, kernel_size w (2) = 3, and the model parameters of conv3 are stride w (3) = 2, kernel_size w (3) = 3, and the left and right sides of padding are pad_l = pad_r = 1. According to formula (1), the final output result of the input data is 9 units S30-S39. It should be understood that S30-S39 are only for illustration and do not represent specific values. At this time, the electronic device 100 does not obtain the specific value of the output result.

[0108] The electronic device 100 is preconfigured to divide the output result into two sub-results, wherein the sub-result A is 4 units S30-S33, and the sub-result B is 5 units S34-S39.

[0109] Next, the calculation process of the first data block corresponding to the sub-result A is described.

[0110] As shown in Figure 8A , the model parameters of conv3 are stride w (3) = 2, kernel_size w (3) = 3, and the left and right sides of padding are pad_l = pad_r = 1. Because the output result of conv3 is S30-S33, i.e., the output result size is 4 units, the first data block corresponds to S20-S27 in the input data of conv3 through formula (3) above, and the size of the input data of conv3 is 8 units.

[0111] As shown in Figure 8B , the input data of conv3 is the output data of conv2, and the model parameters of conv2 are stride w (2) = 1, kernel_size w (2) = 3, and the left and right sides of padding are pad_l = pad_r = 1. Because the output result of conv2 is S20-S27, the first data block corresponds to S10-S18 in the input data of conv2 through formula (3) above, and the size of the input data is 8 units.

[0112] As shown in Figure 8CAs shown, the input data for conv2 is the output data for conv1, and the model parameter for conv1 is stride. w (1) = 2, kernel_size w (1) = 3, the padding on the left and right sides is pad_l = pad_r = 1. Since the output result of conv1 is S10-S18, it can be determined by the above formula (3) that the size of the first data block is 18 units, i.e. data 0-data 17.

[0113] Based on the above calculation results, the electronic device 100 will determine the starting position of the second data block corresponding to the calculation sub-result B.

[0114] like Figure 9A As shown, based on the output result of the first data block conv3 being S30-S33, the size of the output result pos(4) = out(3) = 4 can be determined. According to formula (3) and out(3) = 4, stride(3) = 2, kernel_size(3) = 3, pad(3) = 1, the size of the input data of conv3 can be determined as w(3) = 8. Then, according to formula (2) and substituting the above values, overlap(3) = 1 can be determined. That is to say, the input data of the second data block corresponding to conv3 overlaps with the input data of the first data block corresponding to conv3 by 1 unit. According to formula (4), the starting position of the input data of the second data block corresponding to conv3 can be determined to be one unit forward compared to the input data of the first data block corresponding to conv3. Therefore, the starting position of the input data of the second data block corresponding to conv3 is S27.

[0115] like Figure 9B As shown, based on the output of the first data block conv2 being S20-S27, and the starting position of the second data block in conv3 being S27, the output size pos(3) = 7 and out(2) = 8 can be determined. According to formula (3) and out(2) = 8, stride(2) = 1, kernel_size(2) = 3, pad(2) = 1, the size of the input data of conv2 can be determined as w(2) = 9. Then, according to formula (2) and substituting the above values, overlap(2) = 3 can be determined. That is to say, the input data of the second data block corresponding to conv2 overlaps with the input data of the first data block corresponding to conv2 by 3 units. According to formula (4), the starting position of the input data of the second data block corresponding to conv2 can be determined to be 3 units forward compared to the input data of the first data block corresponding to conv2. Therefore, the starting position of the second data block in conv2 is S16.

[0116] like Figure 9CAs shown, the output result of the first data block conv1 is S10-S18, the starting position of the second data block in conv2 is S16, the size pos(2) of the output result can be determined as 6, out(1) is 9, according to formula (3) and out(1) is 9, stride(2) is 2, kernel_size(2) is 3, and pad(2) is 1, the size w(1) of the input data of conv3 can be determined as 18. Then according to formula (2) and substituting the above values, it can be determined that overlap(1) is 7. That is to say, the input data of the second data block corresponding to conv1 has an overlap of 3 units with the input data of the first data block corresponding to conv1, and according to formula (4), it can be determined that the starting position of the input data of the second data block corresponding to conv2 is moved forward by 3 units from the input data of the first data block corresponding to conv1, and then the starting position of the second data block in conv1 is data 11.

[0117] In this way, the electronic device 100 can inversely deduce the starting position of the second data block in the input data according to the size of the first data block calculation result, so that when the electronic device 100 calculates the second data block after calculating the first data block, the data in the overlapping part between the first data block and the second data block will not be missed. Further, the electronic device 100 only needs to calculate part of the input data each time, and will not generate a large amount of intermediate data at one time, reducing the occupation of the memory in the running of the CNN. The electronic device 100 does not need to send the CNN to other computing devices for running, reducing the occupation of network resources, and also reducing the time delay caused by sending the CNN to other computing devices.

[0118] Further, the CNN can be deployed on a local device for running, and after the calculation of all the convolution layers is completed, the final result is written into the external memory again, reducing the number of times of accessing the external memory. Splitting the input data into multiple data blocks can also enable different data blocks to be calculated by different computing engines of the processor, and different computing engines can be run synchronously, improving the running efficiency.

[0119] In order to solve the problem that the running of the neural network model occupies a large amount of memory, the present application provides a neural network model running device 1000, which comprises a prediction unit 1010, a determination unit 1020 and a sending unit 1030.

[0120] The prediction unit 1010 predicts the size of the calculation result tensor corresponding to the input data tensor according to the size of the input data tensor and the model parameters of each calculation layer;

[0121] The determining unit 1020 is configured to determine sizes of a plurality of sub-result tensors according to a size of the calculation result tensor; the determining unit 1020 is further configured to determine ranges of data blocks corresponding to each sub-result tensor in the input data tensor based on the size of each sub-result tensor and model parameters of each calculation layer; the determining unit 1020 is further configured to calculate each data block according to the model parameters of each calculation layer to obtain a sub-result tensor corresponding to each data block; and the determining unit 1020 is further configured to determine the calculation result tensor corresponding to the input data tensor according to the sub-result tensor corresponding to each data block.

[0122] In some embodiments, the calculation layer is a convolution layer or a deconvolution layer, the convolution layer is configured to perform a convolution operation on the input data tensor, and the deconvolution layer is configured to perform a deconvolution operation on the input data tensor; and the model parameters include a step size, a kernel size, and a padding block number of the convolution operation or the deconvolution operation.

[0123] In some embodiments, the determining unit 1020 is further configured to determine a size of each data block according to the size of each sub-result tensor and the model parameters of each calculation layer; determine a size of an overlapping region between adjacent data blocks according to the size of each sub-result tensor and the model parameters of each calculation layer; and determine the range of the data block corresponding to each sub-result tensor in the input data tensor according to the size of each data block and the size of the overlapping region between adjacent data blocks.

[0124] In some embodiments, the determining unit 1020 is further configured to determine a size of an overlapping region between a data block corresponding to a current sub-result tensor and a data block corresponding to a previous sub-result tensor in a different dimension direction according to the size of the previous sub-result tensor in the different dimension direction and the model parameters of each calculation layer.

[0125] In some embodiments, the calculation layers include M layers, the determining unit 1020 is further configured to determine a size of an overlapping region between adjacent data blocks in the input data tensor corresponding to an Mth calculation layer according to the size of each sub-result tensor and the model parameters of the Mth calculation layer; take the input data tensor of the Mth calculation layer as the output data of an (M-1)th calculation layer, and determine a size of an overlapping region between adjacent data blocks in the input data tensor corresponding to the (M-1)th calculation layer, until a size of an overlapping region between adjacent data blocks in the input data tensor corresponding to a first calculation layer is determined.

[0126] In some embodiments, the sending unit 1030 sends the plurality of data blocks to different calculation engines respectively, and the different calculation engines respectively perform calculation on the received data blocks according to the model parameters of each calculation layer to obtain the sub-result tensor corresponding to each data block.

[0127] In some embodiments, the computing engine is the computing engine of an electronic device processor, or the computing engine is the computing engine of another electronic device processor.

[0128] In this way, the neural network model's execution mechanism can deduce the starting position of the second data block in the input data based on the size of the calculation result of the first data block. When calculating the second data block after calculating the first data block, the neural network model's execution mechanism will not miss the data in the overlapping area between the first and second data blocks. Furthermore, the neural network model's execution mechanism only needs to calculate a portion of the input data each time, avoiding the generation of a large amount of intermediate data at once, thus reducing memory usage during CNN execution. The neural network model's execution mechanism does not need to send the CNN to other computing devices for execution, reducing network resource consumption and the latency caused by sending the CNN to other computing devices. Moreover, the CNN can be deployed and run on a local device; after all convolutional layer calculations are completed, the final result is written to external memory, reducing the number of times external memory is accessed. Dividing the input data into multiple data blocks also allows different data blocks to be calculated by different computing engines of the processor, and different computing engines can run synchronously, improving operating efficiency.

[0129] further, Figure 11 According to some embodiments of this application, a schematic diagram of the structure of an electronic device 100 is shown. For example... Figure 11 As shown, the electronic device 100 includes one or more processors 101A, an NPU 101B, system memory 102, non-volatile memory (NVM) 103, a communication interface 104, input / output (I / O) devices 105, and system control logic 106 for coupling the processors 101A, system memory 102, NVM 103, communication interface 104, and input / output (I / O) devices 105. Wherein:

[0130] The processor 101A can include one or more processing units, for example, can include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro-programmed control unit (MCU), an artificial intelligence (AI) processor, or a processing module or processing circuit of a field programmable gate array (FPGA). The processing module or processing circuit can include one or more single-core or multi-core processors.

[0131] The neural network processor 101B can be used to implement the inference of the neural network model and run the instructions corresponding to the running method of the neural network model provided in the embodiments of the present application. The neural network processor 101B can be a separate processor or can be integrated into the processor 101A.

[0132] The system memory 102 is a volatile memory, for example, a random-access memory (RAM), a double data rate synchronous dynamic random-access memory (DDR SDRAM), etc. The system memory is used to temporarily store data and / or instructions, for example, in some embodiments, the system memory 102 can be used to store the related instructions of the neural network model, the calculation results of the data block, etc.

[0133] The non-volatile memory 103 can include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 103 can include any suitable non-volatile memory and / or any suitable non-volatile storage device, for example, a flash memory, a hard disk drive (HDD), a compact disc (CD), a digital versatile disc (DVD), a solid-state drive (SSD), etc. In some embodiments, the non-volatile memory 103 can also be a removable storage medium, for example, a secure digital (SD) memory card, etc.

[0134] In particular, system memory 102 and non-volatile memory 103 can include, respectively, a temporary copy and a permanent copy of instructions 107. Instructions 107 can include instructions that, when executed by at least one of processor 101 A and / or neural network processor 101 B, cause electronic device 100 to implement the method of operating a neural network model provided by embodiments of the present application.

[0135] Communication interface 104 can include a transceiver to provide electronic device 100 with a wired or wireless communication interface to communicate with any other suitable device over one or more networks. In some embodiments, communication interface 104 can be integrated with other components of electronic device 100, e.g., communication interface 104 can be integrated with processor 101 A. In some embodiments, electronic device 100 can communicate with other devices through communication interface 104, e.g., electronic device 100 can obtain a neural network model to be operated from another electronic device through communication interface 104.

[0136] Input / output (I / O) device 105 can include input devices such as a keyboard, a mouse, etc., and output devices such as a display, etc., through which a user can interact with electronic device 100.

[0137] System control logic 106 can include any suitable interface controllers to provide any suitable interfaces to other modules of electronic device 100. For example, in some embodiments, system control logic 106 can include one or more memory controllers to provide an interface to connect to system memory 102 and non-volatile memory 103.

[0138] In some embodiments, at least one of processor 101 A can be packaged together with logic for one or more controllers of system control logic 106 to form a system in package (SiP). In other embodiments, at least one of processor 101 A can also be integrated on the same chip with logic for one or more controllers of system control logic 106 to form a system-on-chip (SoC).

[0139] It can be understood that the electronic device 100 can be any electronic device capable of running a neural network model, including but not limited to a mobile phone, a wearable device (such as a smart watch, etc.), a tablet computer, a desktop, a laptop, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, and a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) / virtual reality (VR) device, etc., and the embodiments of the present application are not limited thereto.

[0140] It can be understood that Figure 11 The structure of the electronic device 100 shown is only an example, and in other embodiments, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0141] Embodiments of the mechanisms disclosed herein can be implemented in hardware, software, firmware, or any combination thereof. Embodiments of the application can be implemented as computer programs or program code executing on programmable systems comprising at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0142] Program code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices, in known fashion. For purposes of this application, a processing system includes any system that has a processor, such as for example a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0143] The program code can be implemented in a high level procedural or object oriented programming language to communicate with a processing system. The program code can be implemented in assembly or machine language, if desired. In fact, the mechanisms described herein are not limited in scope to any particular programming language. In any case, the language can be a compiled or interpreted language.

[0144] In some cases, the disclosed embodiments can be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments can also be implemented as instructions carried by or stored on a transitory or non-transitory machine-readable (e.g., computer-readable) storage medium, which can be read and executed by one or more processors. For example, the instructions can be distributed over the network or by other computer readable media.

[0145] Thus, a machine-readable medium can include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including without limitation a floppy disk, an optical disc (e.g., a compact disc or a Blu-ray® disc), a read only memory (ROM), a random access memory (RAM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a magnetic media, an optical media, a flash memory device, or a solid state memory device.

[0146] Thus, a machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including without limitation a floppy disk, an optical disc (e.g., a compact disc or a Blu-ray® disc), a read only memory (ROM), a random access memory (RAM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a magnetic media, an optical media, a flash memory device, or a solid state memory device.

[0147] In the drawings, some of the structural or methodological features can be shown in particular arrangements and / or orders. However, it should be appreciated that such particular arrangements and / or orders can not be required. Instead, in some embodiments, the features can be arranged in a different manner and / or order than those shown in the illustrative figures. Additionally, inclusion of structural or methodological features in particular figures does not imply that such features are required in all embodiments, and in some embodiments, these features can not be included or can be combined with other features.

[0148] It should be noted that each unit / module mentioned in each device embodiment of the present application is a logical unit / module, and in reality, one logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented by a combination of multiple physical units / modules. The physical implementation of these logical units / modules is not the most important point, and the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed in the present application, which does not mean that the above-mentioned device embodiments do not have other units / modules.

[0149] It should be noted that in the examples and descriptions of the present patent, the terms "comprise", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "comprises one" does not exclude the existence of other identical elements in the process, method, article or device comprising the element.

[0150] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it should be understood by those skilled in the art that various changes in form and details can be made therein without departing from the spirit and scope of the present application.

Claims

1. A method for operating a neural network model, applied to electronic devices, characterized in that, The neural network model comprises a plurality of calculation layers, and the method comprises: predicting the size of a calculation result tensor corresponding to an input data tensor according to the size of the input data tensor and model parameters of each calculation layer; determining the size of a plurality of sub-result tensors according to the size of the calculation result tensor; determining the range of a data block corresponding to each sub-result tensor in the input data tensor based on the size of each sub-result tensor and the model parameters of each calculation layer, to obtain a plurality of data blocks corresponding to the input data tensor; sending the plurality of data blocks to different calculation engines respectively, and performing calculation on the received data blocks by the different calculation engines according to the model parameters of each calculation layer, to obtain a sub-result tensor corresponding to each data block; determining the calculation result tensor corresponding to the input data tensor according to the sub-result tensor corresponding to each data block.

2. The method of claim 1, wherein: the calculation layer is a convolution layer or a deconvolution layer, the convolution layer is configured to perform convolution operation on the input data tensor, and the deconvolution layer is configured to perform deconvolution operation on the input data tensor; the model parameters comprise a step size, a kernel size, and a padding block number of the convolution operation or the deconvolution operation.

3. The method of claim 2, wherein, determining the range of the data block corresponding to each sub-result tensor in the input data tensor based on the size of each sub-result tensor and the model parameters of each calculation layer comprises: determining the size of each data block according to the size of each sub-result tensor and the model parameters of each calculation layer; determining the size of an overlapping region between adjacent data blocks according to the size of each sub-result tensor and the model parameters of each calculation layer; determining the range of the data block corresponding to each sub-result tensor in the input data tensor according to the size of each data block and the size of the overlapping region between adjacent data blocks.

4. The method of claim 3, wherein, determining the size of the overlapping region between adjacent data blocks according to the size of each sub-result tensor and the model parameters of each calculation layer comprises: determining the size of the overlapping region between the data block corresponding to the current sub-result tensor and the data block corresponding to a previous sub-result tensor in a different dimension direction according to the size of the previous sub-result tensor in the different dimension direction and the model parameters of each calculation layer.

5. The method of claim 4, wherein, the calculation layers are M layers in total, and determining the size of the overlapping region between adjacent data blocks according to the size of each sub-result tensor and the model parameters of each calculation layer further comprises: determining the size of the overlapping region between the adjacent data blocks in the input data tensor corresponding to the Mth calculation layer according to the size of each sub-result tensor and the model parameters of the Mth calculation layer; taking the input data tensor of the Mth calculation layer as the output data of the M-1th calculation layer, and determining the size of the overlapping region between the adjacent data blocks in the input data tensor corresponding to the M-1th calculation layer, until the size of the overlapping region between the adjacent data blocks in the input data tensor corresponding to the first calculation layer is determined.

6. The method of claim 5, wherein: The computing engine is a computing engine of the electronic device processor, or the computing engine is a computing engine of another electronic device processor.

7. A readable medium characterized by The readable medium contains instructions, which, when executed by a processor of an electronic device, cause the electronic device to implement the running method of the neural network model of any one of claims 1 to 6.

8. An electronic device, comprising: Comprise: a memory for storing instructions executed by one or more processors of an electronic device; and a processor, one of the processors of the electronic device, for running the instructions to cause the electronic device to implement the running method of the neural network model of any one of claims 1 to 6.

9. A computer program product, characterised in that, The computer program product comprises instructions which, when executed, cause a computer to perform the running method of the neural network model of any one of claims 1 to 6.

Images