Operator generation method and device, equipment, medium and program product

Abstract

The invention relates to an operator generation method and device, equipment, a storage medium and a program product. The method comprises the following steps: constructing a deep convolutional neural network; determining a configuration item of the deep convolutional neural network and a plurality of kernel instances corresponding to the configuration item; running the deep convolutional neural network based on the kernel instance to obtain a speed measurement result corresponding to the kernel instance; based on the speed measurement result, selecting an optimal kernel instance from the multiple kernel instances; and generating a target operator based on the optimal kernel instance and the configuration item. According to the embodiment of the invention, the optimal kernel instance corresponding to each configuration item is determined by adopting an offline search mode for the multiple kernel instances corresponding to the common configuration items, and the corresponding optimal kernel instance can be directly called in a deep convolutional neural network reasoning process without online search, so that the reasoning time of a network model is shortened.

Description

Technical Field

[0001] The present disclosure relates to the field of computer processing technology, and in particular to an operator generation method, apparatus, device, storage medium and program product. Background Art

[0002] With the rapid development of artificial intelligence technology, deep learning neural networks have achieved great success in pattern recognition such as image classification, target detection, image segmentation, speech recognition, and machine translation. However, a key indicator of whether deep convolutional neural networks can be successfully applied in terminals and meet product requirements is the reasoning performance of the neural network model. The good reasoning performance of the Bible network requires that the model's operators have the performance to fully utilize the computing power of the terminal hardware. Therefore, in order to achieve the high performance requirements of the operator, it is necessary to tune the operator in a specific scenario. The tuned operator has extreme performance in a specific scenario.

[0003] At present, the commonly used operator tuning method is to store the kernel instances under all configuration items corresponding to the operator in the model data package. When the network model is inferred and judged, the most suitable kernel instance is selected by online search to generate the target operator.

[0004] The above operator tuning method is limited by the model data package and cannot configure the kernel instances under all configuration items in the model data package. In addition, the online search method will increase the inference time of the network model. Summary of the invention

[0005] In order to solve the above technical problems, the embodiments of the present disclosure provide an operator generation method, apparatus, device, medium and program product. For multiple kernel instances corresponding to common configuration items, an offline search method is used to determine the optimal kernel instance corresponding to each configuration item. During the deep convolutional neural network inference process, the corresponding optimal kernel instance can be directly called without online search, thereby reducing the inference time of the network model.

[0006] In a first aspect, an embodiment of the present disclosure provides an operator generation method, including:

[0007] Constructing a deep convolutional neural network, wherein the number of group convolutions, the number of input channels, and the number of output channels included in the deep convolutional neural network are the same, the number of filters in the deep convolutional neural network is the same as the number of output channels, and each filter only includes a kernel function with an input channel number of 1;

[0008] Determine configuration items of the deep convolutional neural network and multiple kernel instances corresponding to the configuration items, the configuration items including operator size and operator step size, and the kernel instance including the size of the element block in the input feature map and the size of the element block in the output feature map;

[0009] Running the deep convolutional neural network based on the kernel instance to obtain a speed measurement result corresponding to the kernel instance;

[0010] Based on the speed measurement result, selecting an optimal kernel instance from the multiple kernel instances;

[0011] A target operator is generated based on the optimal kernel instance and the configuration item.

[0012] In a second aspect, an embodiment of the present disclosure provides an operator generation device, including:

[0013] A neural network construction module, used to construct a deep convolutional neural network, wherein the number of group convolutions, the number of input channels, and the number of output channels included in the deep convolutional neural network are the same, the number of filters in the deep convolutional neural network is the same as the number of output channels, and each filter only includes a kernel function with an input channel number of 1;

[0014] A kernel instance configuration module, used to determine configuration items of the deep convolutional neural network and multiple kernel instances corresponding to the configuration items, wherein the configuration items include operator size and operator step size, and the kernel instance includes the size of the element block in the input feature map and the size of the element block in the output feature map;

[0015] A speed measurement result determination module, configured to run the deep convolutional neural network based on the kernel instance to obtain a speed measurement result corresponding to the kernel instance;

[0016] An optimal kernel instance selection module, configured to select an optimal kernel instance from the multiple kernel instances based on the speed measurement result;

[0017] A target operator generation module is used to generate a target operator based on the optimal kernel instance and the configuration item.

[0018] In a third aspect, an embodiment of the present disclosure provides an electronic device, the electronic device comprising:

[0019] one or more processors;

[0020] A storage device for storing one or more programs;

[0021] When the one or more programs are executed by the one or more processors, the one or more processors implement the operator generation method as described in any one of the first aspects above.

[0022] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the operator generation method as described in any one of the above-mentioned first aspects.

[0023] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, an operator generation method as described in any one of the above-mentioned first aspects is implemented.

[0024] The embodiment of the present disclosure provides an operator generation method, device, equipment, storage medium and program product, the method comprising: constructing a deep convolutional neural network, wherein the number of group convolutions, the number of input channels and the number of output channels included in the deep convolutional neural network are the same, the number of filters in the deep convolutional neural network is the same as the number of output channels, and each filter only includes a kernel function with an input channel number of 1; determining the configuration items of the deep convolutional neural network and multiple kernel instances corresponding to the configuration items, the configuration items including operator size and operator step size, the kernel instance including the size of the element block in the input feature map and the size of the element block in the output feature map; running the deep convolutional neural network based on the kernel instance to obtain the speed measurement result corresponding to the kernel instance; based on the speed measurement result, selecting the optimal kernel instance from the multiple kernel instances; generating a target operator based on the optimal kernel instance and the configuration item. The embodiment of the present disclosure determines the optimal kernel instance corresponding to each configuration item by using an offline search method for multiple kernel instances corresponding to common configuration items, and directly calls the corresponding optimal kernel instance in the deep convolutional neural network reasoning process without online search, thereby reducing the reasoning time of the network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0026] Figure 1 A schematic diagram of a depth-separable convolution structure provided by an embodiment of the present disclosure is shown;

[0027] Figure 2 A flowchart of an operator generation method in an embodiment of the present disclosure;

[0028] Figure 3a It is a schematic diagram of the speed measurement results in the float case when kernel size=1 and stride=1 in the embodiment of the present disclosure;

[0029] Figure 3b It is a schematic diagram of the speed measurement result in the half case when kernel size=1 and stride=1 in the embodiment of the present disclosure;

[0030] Figure 4a It is a schematic diagram of the speed measurement results in the float case when kernel size=3 and stride=1 in the embodiment of the present disclosure;

[0031] Figure 4b It is a schematic diagram of the speed measurement result in the half case when kernel size=3 and stride=1 in the embodiment of the present disclosure;

[0032] Figure 5a It is a schematic diagram of the speed measurement results in the float case when kernel size=3 and stride=2 in the embodiment of the present disclosure;

[0033] Figure 5b It is a schematic diagram of the speed measurement result in the half case when kernel size=3 and stride=2 in the embodiment of the present disclosure;

[0034] Figure 6a It is a schematic diagram of the speed measurement results in the float case when kernel size=5 and stride=1 in the embodiment of the present disclosure;

[0035] Figure 6b It is a schematic diagram of the speed measurement result in the half case when kernel size=5 and stride=1 in the embodiment of the present disclosure;

[0036] Figure 7a It is a schematic diagram of the speed measurement results in the float case when kernel size=5 and stride=2 in the embodiment of the present disclosure;

[0037] Figure 7b It is a schematic diagram of the speed measurement result in the half case when kernel size=5 and stride=2 in the embodiment of the present disclosure;

[0038] Figure 8 It is a schematic diagram of the speed measurement results of different input sizes when kernel ID=5 in the embodiment of the present disclosure;

[0039] Fig. 9 In the embodiment of the present disclosure Schematic diagram of comparison with TRT speed measurement results;

[0040] Fig.10 is a structural schematic diagram of an operator generating device in an embodiment of the present disclosure;

[0041] Fig.11 It is a structural schematic diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0043] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0044] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0045] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0046] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0047] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0048] The operator generation method proposed in the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0049] Figure 1 This is a flowchart of an operator generation method in an embodiment of the present disclosure. The method can be executed by an operator generation device. The operator generation device can be implemented in software and / or hardware. The operator generation method can be configured in an electronic device.

[0050] Among them, the electronic device can be any other type of electronic device capable of performing data processing, which may include but is not limited to: mobile phones, stations, units, devices, multimedia computers, multimedia tablets, Internet nodes, communicators, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices or any combination thereof, including accessories and peripherals of these devices or any combination thereof.

[0051] like Figure 2 As shown, the operator generation method provided by the embodiment of the present disclosure mainly includes steps S101-S105.

[0052] S101. Construct a deep convolutional neural network, wherein the number of group convolutions, the number of input channels, and the number of output channels included in the deep convolutional neural network are the same, the number of filters in the deep convolutional neural network is the same as the number of output channels, and each filter only includes a kernel function with an input channel number of 1.

[0053] Among them, deep convolutional neural network refers to depthwise convolution, such as Figure 2 As shown, in Depthwise convolution, one convolution kernel convolves one channel. In other words, Depthwise convolution is a special convolution with Group = IC = OC. Essentially, the weight contains O_C filters, and each filter has only one IC / group = 1 kernel. Therefore, when calculating, each kernel essentially performs a planar convolution with the corresponding input channel to obtain an output channel. Therefore, the three-dimensional multiplication and accumulation degenerates into a two-dimensional multiplication and accumulation.

[0054] When implementing Depthwise convolution, the simplest implementation is to calculate one output element per thread. The mapping relationship between the convolution operation input size and output size is:

[0055] O_H=(I_H+2*Pad H )-((KH-1)*hole H +1) / Stride H +1;

[0056] O_W=(I_W+2*Pad W )-((KW-1)*hole W +1) / Stride W +1;

[0057] Where I_H is the height of the input element, I_W is the width of the input element, O_H is the height of the output element, and O_W is the width of the output element. hole H and hole W is the expansion coefficient of the convolution kernel, Pad H It is the padding of the convolution kernel in the height direction, Pad W is the width-wise padding of the convolution kernel, Stride H Stride is the moving step length of the convolution kernel in the height direction. W is the moving step of the convolution kernel in the width direction. The input element can be a pixel in the input feature map, and the output element can be a pixel corresponding to the input element in the output feature map.

[0058] Traversing the output elements, if the coordinates of the output element to be calculated are (n, oh, ow, oc), then according to the mapping relationship between the convolution coordinates, it can be known that the coordinate area of ​​the input element block involved in the output element is ( Figure 2 The mapping relationship between the starting coordinates (ih, iw) of the area in the black thick line frame in the input feature map and the starting coordinates (oh, ow) of the area in the black thick line frame in the output feature map is:

[0059] ih=oh*Stride H -Pad H ;

[0060] iw=ow*Stride W -Pad W .

[0061] Furthermore, block optimization improves parallelism. Assuming that each thread calculates TILE_H and TILE_W, the size of the element block in the corresponding input feature map is:

[0062] SRC_TILE_H=(TILE_H-1)*Stride H +K H

[0063] SRC_TILE_W=(TILE_W-1)*Stride W +K W .

[0064] According to the above calculation results, after setting the size of the element block in the output feature map of the single-thread calculation (TILE_H, TILE_W), the input block size (SRC_TILE_H, SRC_TILE_W) can be obtained. The weight block size is the entire weight (KERNEL_H, KERNEL_W). All three parts are loaded into the thread register, and then the padding output is directly calculated.

[0065] S102. Determine configuration items of the deep convolutional neural network and multiple kernel instances corresponding to the configuration items, wherein the configuration items include operator size and operator step size, and the kernel instance includes the size of the element block in the input feature map and the size of the element block in the output feature map.

[0066] The size of the element block in the input feature map is (SRC_TILE_H, SRC_TILE_W), and the size of the element block in the output feature map is (TILE_H, TILE_W). The size of the element block in the input feature map can be represented by the width TILE_W of the element block in the input feature map and the height TILE_H of the element block in the input feature map. The size of the element block in the output feature map can be represented by the width SRC_TILE_W of the element block in the output feature map and the height SRC_TILE_H of the element block in the output feature map.

[0067] In the embodiment of the present disclosure, multiple kernel instances under the same configuration differ only in input and output configuration information, so the input and output configuration information is used to represent the corresponding kernel instance.<TILE_H,TILE_W,SRC_TILE_H,SRC_TILE_W> Represents an instantiated kernel instance.

[0068] In one embodiment of the present disclosure, the configuration items can be set according to the commonly used convolution configuration. The configuration items may include operator size and operator step size. Among them, the configuration items include an operator size of 1 and an operator step size of 1, that is, kernel size = 1, stride = 1; the configuration items include an operator size of 3 and an operator step size of 1, that is, kernel size = 3, stride = 1; the configuration items include an operator size of 3 and an operator step size of 2, that is, kernel size = 3, stride = 2; the configuration items include an operator size of 5 and an operator step size of 1, that is, kernel size = 5, stride = 1; the configuration items include an operator size of 5 and an operator step size of 2, that is, kernel size = 5, stride = 2.

[0069] It should be noted that the above is only an example of setting operator parameters, and other operator parameters can also be set according to actual conditions.

[0070] In the embodiment of the present disclosure, for each of the above configuration items, multiple corresponding kernel instances are configured.

[0071] In a possible implementation of the present disclosure, when kernel size=1 and stride=1, commonly used input and output configuration information includes the following cases: <1,1,1,1>, <2,2,2,2>, <3,3,3,3>, <4,4,4,4>, etc. In other words, when kernel size=1 and stride=1, use<TILE_H,TILE_W,SRC_TILE_H,SRC_TILE_W> <1,1,1,1>, <2,2,2,2>, <3,3,3,3>, <4,4,4,4> are configuration template parameters, and their corresponding kernel instances are obtained.<TILE_H,TILE_W,SRC_TILE_H,SRC_TILE_W> Configure the template parameters for <1,1,1,1> to obtain the kernel instance corresponding to <1,1,1,1>.<TILE_H,TILE_W,SRC_TILE_H,SRC_TILE_W> Configure the template parameters for <2,2,2,2> to obtain the kernel instance corresponding to <2,2,2,2>.<TILE_H,TILE_W,SRC_TILE_H,SRC_TILE_W> Configure the template parameters for <3,3,3,3> to obtain the kernel instance corresponding to <3,3,3,3>.<TILE_H,TILE_W,SRC_TILE_H,SRC_TILE_W> Configure template parameters for <4,4,4,4> and obtain the kernel instance corresponding to <4,4,4,4>.

[0072] In a possible implementation of the present disclosure, when kernel size=3 and stride=1, commonly used input and output configuration information includes the following cases: <2,2,4,4>, <3,3,5,5>, <4,4,6,6>, etc. In other words, when kernel size=3 and stride=1, use<TILE_H,TILE_W,SRC_TILE_H,SRC_TILE_W> <2,2,4,4>, <3,3,5,5>, <4,4,6,6> are configuration template parameters, and their corresponding kernel instances are obtained.<TILE_H,TILE_W,SRC_TILE_H,SRC_TILE_W> Configure the template parameters for <2,2,4,4> to obtain the kernel instance corresponding to <2,2,4,4>.<TILE_H,TILE_W,SRC_TILE_H,SRC_TILE_W> Configure the template parameters for <3,3,5,5> to obtain the kernel instance corresponding to <3,3,5,5>.<TILE_H,TILE_W,SRC_TILE_H,SRC_TILE_W> Configure template parameters for <4,4,6,6> and obtain the kernel instance corresponding to <4,4,6,6>.

[0073] In a possible implementation of the present disclosure, when kernel size=3 and stride=2, commonly used input and output configuration information includes the following cases: <2,2,5,5>, <3,3,7,7>, etc. In other words, when kernelsize=3 and stride=2, use<TILE_H,TILE_W,SRC_TILE_H,SRC_TILE_W> is <2,2,5,5>,

[0074] <3,3,7,7> are configuration template parameters, and their corresponding kernel instances are obtained.<TILE_H,TILE_W,SRC_TILE_H,SRC_TILE_W> Configure the template parameters for <2,2,5,5> to obtain the kernel instance corresponding to <2,2,5,5>.<TILE_H,TILE_W,SRC_TILE_H,SRC_TILE_W> Configure template parameters for <3,3,7,7> and obtain the kernel instance corresponding to <3,3,7,7>.

[0075] In a possible implementation of the present disclosure, when kernel size=5 and stride=1, commonly used input and output configuration information includes the following cases: <2,2,6,6>, etc. In other words, when kernel size=5 and stride=1, use<TILE_H,TILE_W,SRC_TILE_H,SRC_TILE_W> Configure template parameters for <2,2,6,6> and obtain its corresponding kernel instance.

[0076] In a possible implementation of the present disclosure, when kernel size=5 and stride=2, commonly used input and output configuration information includes the following: <2,2,7,7>, etc. In other words, when kernel size=5 and stride=2, use<TILE_H,TILE_W,SRC_TILE_H,SRC_TILE_W> Configure template parameters for <2,2,7,7> and obtain its corresponding kernel instance.

[0077] In one embodiment of the present disclosure, the kernel instance corresponding to each configuration item includes a preset kernel instance, wherein the height of the element block in the input feature map in the preset kernel instance is 1, the width of the element block in the input feature map is 1, the height of the element block in the output feature map is -1, and the width of the element block in the output feature map is -1.

[0078] In the disclosed embodiment, <1,1,-1,-1> represents a naive implementation method in which one thread calculates one output, which is common to all configuration items. Therefore, under each of the above-mentioned setting operator parameters, a kernel instance corresponding to <1,1,-1,-1> is included.

[0079] In one embodiment of the present disclosure, for convenience of representation, each kernel instance is set with its corresponding kernel identifier.

[0080] Specifically, the kernel instance corresponding to <1,1,-1,-1> has a kernel ID of Kernel ID = 0. In the case of kernelsize = 1 and stride = 1, the kernel instance corresponding to <1,1,1,1> has a kernel ID of Kernel ID = 1, the kernel instance corresponding to <2,2,2,2> has a kernel ID of Kernel ID = 2, the kernel instance corresponding to <3,3,3,3> has a kernel ID of Kernel ID = 3, and the kernel instance corresponding to <4,4,4,4> has a kernel ID of Kernel ID = 4.

[0081] When kernel size=3 and stride=1, the kernel ID corresponding to the kernel instance corresponding to <2,2,4,4> is Kernel ID=5, the kernel ID corresponding to the kernel instance corresponding to <3,3,5,5> is Kernel ID=6, and the kernel ID corresponding to the kernel instance corresponding to <4,4,6,6> is Kernel ID=7.

[0082] When kernel size=3 and stride=2, the kernel ID corresponding to the kernel instance corresponding to <2,2,5,5> is Kernel ID=8, and the kernel ID corresponding to the kernel instance corresponding to <3,3,7,7> is Kernel ID=9.

[0083] When kernel size=5 and stride=1, the kernel identifier corresponding to the kernel instance corresponding to <2,2,6,6> is Kernel ID=10.

[0084] When kernel size=5 and stride=2, the kernel identifier corresponding to the kernel instance corresponding to <2,2,7,7> is Kernel ID=11.

[0085] It should be noted that the embodiments of the present disclosure only provide an exemplary description of the correspondence between the kernel identifier and the kernel instance, and other settings may be made according to actual conditions.

[0086] S102: Run the deep convolutional neural network based on the kernel instance to obtain a speed measurement result corresponding to the kernel instance.

[0087] In the embodiment of the present disclosure, as described above, multiple conventional configuration items can be configured, and for each configuration item, a deep convolutional neural network is run based on multiple kernel instances corresponding to the configuration item to obtain a speed measurement result. The speed measurement result includes the running speed of each kernel instance in each configuration item when running the deep convolutional neural network.

[0088] In the disclosed embodiment, the deep convolutional neural network is run based on the kernel instance to obtain the speed measurement result corresponding to the kernel instance, including: for multiple input data states, the deep convolutional neural network is run based on different input sizes and the kernel instance to obtain the speed measurement result corresponding to each kernel instance under each input size, wherein the input data state refers to the data type of the input data corresponding to the deep convolutional neural network and the data format of the input data.

[0089] Wherein, the input data state refers to the type of input data and the format of the input data that the deep convolutional neural network needs to process. The data format includes NCHW format and NHWC format. Wherein, the data type includes half-precision half format and floating-point float format. Wherein, N: Batch number, such as the number of images; H: Height, feature map height, that is, the number of pixels in the vertical height direction; W: Width, feature map width, that is, the number of pixels in the horizontal width direction; C: Channels, feature map channels, for example, the Channels of a color RGB image is 3; since data can only be stored linearly, these four dimensions have a corresponding order. Different deep learning frameworks will store feature map data in different orders. The NCHW format means that the arrangement order of feature map data is [number of Batch, feature map channels Channels, feature map height Height, feature map width Width]; the NHWC format means that the arrangement order of feature map data is [number of Batch, feature map height Height, feature map width Width, feature map channels Channels].

[0090] In one implementation of the present disclosure, speed measurement is performed for each kernel instance under each configuration item.

[0091] In the disclosed embodiment, for the case of half-precision half type and NCHW format, the speed of each kernel instance under each configuration item is measured separately. For the case of half-precision half type and NHWC format, the speed of each kernel instance under each configuration item is measured separately. For the case of floating-point float type and NCHW format, the speed of each kernel instance under each configuration item is measured separately. For the case of floating-point float type and NHWC format, the speed of each kernel instance under each configuration item is measured separately.

[0092] Specifically, for kernel size = 1, stride = 1, for half-precision half type and NCHW format, each Kernel ID = 0, Kernel ID = 1, Kernel ID = 2, Kernel ID = 3, Kernel ID = 4 corresponds to a kernel instance, and a deep convolutional neural network is run to obtain the speed test results. For half-precision half type and NHWC format, each Kernel ID = 0, Kernel ID = 1, Kernel ID = 2, Kernel ID = 3, Kernel ID = 4 corresponds to a kernel instance, and a deep convolutional neural network is run to obtain the speed test results. For kernel size = 1, stride = 1, for floating-point float type and NCHW format, each Kernel ID = 0, Kernel ID = 1, Kernel ID = 2, Kernel ID = 3, Kernel ID = 4 corresponds to a kernel instance, and a deep convolutional neural network is run to obtain the speed test results. For the floating point type and NHWC format, each Kernel ID = 0, Kernel ID = 1, Kernel ID = 2, Kernel ID = 3, and Kernel ID = 4 corresponds to a kernel instance, running the deep convolutional neural network to obtain the speed test results.

[0093] In the disclosed embodiment, for the case of kernel size = 3, stride = 1, for the half-precision half type and NCHW format, each Kernel ID = 0, Kernel ID = 5, Kernel ID = 6, Kernel ID = 7 corresponds to running a deep convolutional neural network respectively to obtain a speed measurement result. For the half-precision half type and NHWC format, each Kernel ID = 0, Kernel ID = 5, Kernel ID = 6, Kernel ID = 7 corresponds to running a deep convolutional neural network respectively to obtain a speed measurement result. For the floating-point float type and NCHW format, each Kernel ID = 0, Kernel ID = 5, Kernel ID = 6, Kernel ID = 7 corresponds to running a deep convolutional neural network respectively to obtain a speed measurement result. For the floating-point float type and NHWC format, each Kernel ID = 0, Kernel ID = 5, Kernel ID = 6, Kernel ID = 7 corresponds to running a deep convolutional neural network respectively to obtain a speed measurement result.

[0094] In the disclosed embodiment, for the case of kernel size = 3, stride = 2, for the half-precision half type and NCHW format, each Kernel ID = 0, Kernel ID = 8, Kernel ID = 9 corresponds to a kernel instance, and a deep convolutional neural network is run respectively to obtain a speed measurement result. For the half-precision half type and NHWC format, each Kernel ID = 0, Kernel ID = 8, Kernel ID = 9 corresponds to a kernel instance, and a deep convolutional neural network is run respectively to obtain a speed measurement result. For the floating-point float type and NCHW format, each Kernel ID = 0, Kernel ID = 8, Kernel ID = 9 corresponds to a kernel instance, and a deep convolutional neural network is run respectively to obtain a speed measurement result. For the floating-point float type and NHWC format, each Kernel ID = 0, Kernel ID = 8, Kernel ID = 9 corresponds to a kernel instance, and a deep convolutional neural network is run respectively to obtain a speed measurement result.

[0095] In the disclosed embodiment, for the case of kernel size = 5, stride = 1, for the half-precision half type and NCHW format, each Kernel ID = 0, Kernel ID = 10 corresponds to a kernel instance, and a deep convolutional neural network is run respectively to obtain a speed measurement result. For the half-precision half type and NHWC format, each Kernel ID = 0, Kernel ID = 10 corresponds to a kernel instance, and a deep convolutional neural network is run respectively to obtain a speed measurement result. For the floating-point float type and NCHW format, each Kernel ID = 0, Kernel ID = 10 corresponds to a kernel instance, and a deep convolutional neural network is run respectively to obtain a speed measurement result. For the floating-point float type and NHWC format, each Kernel ID = 0, Kernel ID = 10 corresponds to a kernel instance, and a deep convolutional neural network is run respectively to obtain a speed measurement result.

[0096] In the embodiment of the present disclosure, when kernel size=5 and stride=2,

[0097] For half-precision half type and NCHW format, each Kernel ID=0, KernelID=11 corresponds to a kernel instance, and a deep convolutional neural network is run to obtain the speed test results. For half-precision half type and NHWC format, each Kernel ID=0, KernelID=11 corresponds to a kernel instance, and a deep convolutional neural network is run to obtain the speed test results. For floating-point float type and NCHW format, each Kernel ID=0, KernelID=11 corresponds to a kernel instance, and a deep convolutional neural network is run to obtain the speed test results. For floating-point float type and NHWC format, each Kernel ID=0, KernelID=11 corresponds to a kernel instance, and a deep convolutional neural network is run to obtain the speed test results.

[0098] On the basis of the above embodiments, for the case of half-precision half type and NCHW format, the speed of each kernel instance under each configuration item is measured at multiple input sizes. For the case of half-precision half type and NHWC format, the speed of each kernel instance under each configuration item is measured at multiple input sizes. For the case of floating-point float type and NCHW format, the speed of each kernel instance under each configuration item is measured at multiple input sizes. For the case of floating-point float type and NHWC format, the speed of each kernel instance under each configuration item is measured at multiple input sizes.

[0099] S104: Based on the speed measurement result, select an optimal kernel instance from the multiple kernel instances.

[0100] In one embodiment of the present disclosure, the selecting the optimal kernel instance from the multiple kernel instances based on the speed measurement results includes: for each input data state, filtering out the kernel instance with the fastest running speed under each input size from the speed measurement results corresponding to the input data state; obtaining the kernel identifier corresponding to each kernel instance with the fastest running speed; and selecting the optimal kernel instance from the multiple kernel instances based on the distribution pattern of the kernel identifiers.

[0101] In the disclosed embodiment, based on the above speed test results, for the half-precision half type and NCHW format, the speed of each kernel instance under each configuration item is measured under multiple input sizes, and the running speed of each kernel instance under multiple input sizes is obtained, and the kernel identifier corresponding to the kernel instance with the fastest kernel running speed under each input size is selected. For the half-precision half type and NHWC format, the speed of each kernel instance under each configuration item is measured under multiple input sizes, and the running speed of each kernel instance under multiple input sizes is obtained, and the kernel identifier corresponding to the kernel instance with the fastest kernel running speed under each input size is selected. For the floating-point float type and NCHW format, the speed of each kernel instance under each configuration item is measured under multiple input sizes, and the running speed of each kernel instance under multiple input sizes is obtained, and the kernel identifier corresponding to the kernel instance with the fastest kernel running speed under each input size is selected. For the floating-point type and NHWC format, the speed of each kernel instance under each configuration item is measured under multiple input sizes, and the running speed of each kernel instance under multiple input sizes is obtained. The kernel identifier corresponding to the kernel instance with the fastest running speed under each input size is selected.

[0102] For example, for floating point type and NHWC format, each kernel instance under each configuration item is tested at multiple input sizes to obtain the running speed of each kernel instance under multiple input sizes. That is, each kernel instance corresponding to each input size has a corresponding running speed, and the kernel instance with the fastest running speed is selected.

[0103] In one embodiment of the present disclosure, selecting an optimal kernel instance from the multiple kernel instances based on the distribution law of the kernel identifiers includes: if the distribution law of the kernel identifiers corresponding to at least one input data state satisfies a preset condition, determining that at least one input data state that satisfies the preset condition corresponds to an optimal kernel instance, wherein the optimal kernel instance is the kernel identifier with the largest number among all the kernel identifiers corresponding to at least one input data state.

[0104] In one embodiment of the present disclosure, the optimal kernel instance is selected from the multiple kernel instances based on the distribution law of the kernel identifier, including: if the distribution law of the kernel identifier corresponding to at least one input data state does not meet a preset condition, then based on the distribution law of the kernel identifier, the multiple input sizes are segmented; each segment of the input size corresponds to an optimal kernel instance.

[0105] In one embodiment of the present disclosure, the distribution pattern of the kernel identifier meets a preset condition, including: a variation range of a speed measurement curve constructed by the kernel identifier is within a set range.

[0106] In the embodiment of the present disclosure, for the case of kernel size = 1, stride = 1, in the case of floating point float type, for the NCHW format and the NHWC format, each Kernel ID = 0, Kernel ID = 1, Kernel ID = 2, Kernel ID = 3, Kernel ID = 4 corresponds to a kernel instance, and for each kernel instance under each configuration item, the speed is measured under multiple input sizes, and the running speed of each kernel instance under multiple input sizes is obtained. The kernel identifier corresponding to the kernel instance with the fastest kernel running speed under each input size is selected, and the regular distribution corresponding to the kernel identifier corresponding to the kernel instance with the fastest running speed is as follows: Figure 3a As shown in the figure, in the case of floating point float data type, and in NCHW format and NHWC format, the running speed of each input size is as follows Figure 3a As shown. For kernel size = 1, stride = 1, in the case of half-precision half data type, for NCHW format and NHWC format, each kernel ID = 0, kernel ID = 1, kernel ID = 2, kernel ID = 3, kernel ID = 4, each kernel instance under each configuration item is tested at multiple input sizes, and the running speed of each kernel instance under multiple input sizes is obtained. The kernel ID corresponding to the kernel instance with the fastest running speed under each input size is selected. The regular distribution of the kernel ID corresponding to the kernel instance with the fastest running speed is as follows: Figure 3b As shown. In the case of half-precision data type, and in NCHW format and NHWC format, the running speed of each input size is as follows, Figure 3b shown.

[0107] like Figure 3a and Figure 3bAs shown in the figure, the optimal speeds of NCHW format and NHWC format are close under each input size, and the speed curves corresponding to the distribution law of kernel identifiers are basically overlapped. When the size and channel are small, the speed curves corresponding to NCHW format and NHWC format under the two data types of float type and half precision half type have regular jumps near the similar speed curves. The optimal kernel instances of float and half data types are different. When the packet size is weighed, only one kernel instance is selected. When the data type is float type and the data format is NCHW format or NHWC format, the kernel instance corresponding to Kernel ID=1 is selected, that is, the kernel instance corresponding to (<1,1,1,1>) is selected as the optimal kernel instance. When the data type is half type and the data format is NCHW format or NHWC format, the kernel instance corresponding to Kernel ID=2 is selected, that is, the kernel instance corresponding to (<2,2,2,2>) is selected as the optimal kernel instance.

[0108] In the embodiment of the present disclosure, for kernel size = 3, stride = 1, in the case of floating point float type, for NCHW format and NHWC format, each kernel instance corresponding to Kernel ID = 0, Kernel ID = 5, Kernel ID = 6, Kernel ID = 7, for each kernel instance under each configuration item, the speed is measured under multiple input sizes, and the running speed of each kernel instance under multiple input sizes is obtained. The kernel identifier corresponding to the kernel instance with the fastest kernel running speed under each input size is selected, and the regular distribution corresponding to the kernel identifier corresponding to the kernel instance with the fastest running speed is as follows: Figure 4a As shown in the figure, in the case of floating point float data type, and in NCHW format and NHWC format, the running speed of each input size is as follows Figure 3a As shown. For kernel size = 3, stride = 1, in half-precision half type, for NCHW format and NHWC format, each kernel instance corresponding to KernelID = 0, Kernel ID = 5, Kernel ID = 6, Kernel ID = 7, after speed testing for each kernel instance under each configuration item under multiple input sizes, the running speed of each kernel instance under multiple input sizes is obtained, and the kernel identifier corresponding to the kernel instance with the fastest running speed under each input size is selected. The regular distribution of the kernel identifier corresponding to the kernel instance with the fastest running speed is shown as follows: Figure 4b As shown in the figure, in the case of half-precision half type, as well as NCHW format and NHWC format, the running speed of each input size is as follows Figure 4b shown.

[0109] like Figure 4a and Figure 4b As shown in the figure, since kernel size is not equal to 1, the impact of memory arrangement on reading efficiency is reflected in speed. The memory arrangement in NCHW format has better reading continuity for kernel instances and higher cache hit rate. Therefore, in large channels and large sizes, the speed of NCHW format is significantly better than that of NCHW format. When the data type is float type or half type, and the data format is NCHW format or NHWC format, the speed test curves corresponding to the kernel instance are basically the same, and the speed test curves of NCHW format and NHWC format of the same type are also basically the same. The kernel identifier corresponding to the optimal kernel instance of the four configurations is KernelID=5, that is, the kernel instance corresponding to (<2,2,4,4>) is the optimal kernel instance.

[0110] In the embodiment of the present disclosure, for kernel size = 3, stride = 2, in the case of floating point float type, for NCHW format and NHWC format, each kernel instance corresponding to Kernel ID = 0, Kernel ID = 8, and Kernel ID = 9, after speed measurement is performed for each kernel instance under each configuration item under multiple input sizes, the running speed of each kernel instance under multiple input sizes is obtained, and the kernel identifier corresponding to the kernel instance with the fastest kernel running speed under each input size is selected. The regular distribution of the kernel identifier corresponding to the kernel instance with the fastest running speed is as follows: Figure 5a As shown in the figure, in the case of floating point type, and in NCHW format and NHWC format, the running speed of each input size is as follows Figure 5a As shown. For kernel size = 3, stride = 2, for half-precision half data types, for NCHW format and NHWC format, each Kernel ID = 0, Kernel ID = 8, Kernel ID = 9 corresponds to a kernel instance. For each kernel instance under each configuration item, the speed is measured under multiple input sizes, and the running speed of each kernel instance under multiple input sizes is obtained. The kernel ID corresponding to the kernel instance with the fastest running speed under each input size is selected. The regular distribution of the kernel ID corresponding to the kernel instance with the fastest running speed is shown as follows: Figure 5b As shown in the figure, in the case of half-precision half type, as well as NCHW format and NHWC format, the running speed of each input size is as follows Figure 5b shown.

[0111] like Figure 5a and Figure 5bAs shown in the figure, stride=2 causes discontinuous reading of input element blocks, especially for float data type, the speed test curves corresponding to kernel instances in NCHW format and NHWC format are obviously differentiated. Therefore, when the data type is float type and the data format is NHWC format, the kernel instance corresponding to Kernel ID=8 is selected as the optimal kernel instance. When the data type is float type and the data format is NCHW format, the kernel instance corresponding to Kernel ID=0 is selected as the optimal kernel instance. When the data type is half type and the data format is NHWC format, the kernel instance corresponding to Kernel ID=8 is selected as the optimal kernel instance. When the data type is half type and the data format is NCHW format, when the input size is less than 36, the kernel instance corresponding to Kernel ID=0 is selected as the optimal kernel instance. When the input size is greater than or equal to 36, the kernel instance corresponding to Kernel ID=8 is selected as the optimal kernel instance.

[0112] In the embodiment of the present disclosure, for kernel size = 5, stride = 1, in the case of floating point float type, for NCHW format and NHWC format, each kernel instance corresponding to KernelID = 0, Kernel ID = 10, for each kernel instance under each configuration item, the speed is measured under multiple input sizes, and the running speed of each kernel instance under multiple input sizes is obtained. The kernel identifier corresponding to the kernel instance with the fastest kernel running speed under each input size is selected, and the regular distribution corresponding to the kernel identifier corresponding to the kernel instance with the fastest running speed is as follows: Figure 6a As shown in the figure, in the case of floating point type, and in NCHW format and NHWC format, the running speed of each input size is as follows Figure 6a As shown. For kernel size = 5, stride = 1, for half-precision half data types, for NCHW format and NHWC format, each Kernel ID = 0, Kernel ID = 10 corresponds to a kernel instance. For each kernel instance under each configuration item, the speed is measured under multiple input sizes, and the running speed of each kernel instance under multiple input sizes is obtained. The kernel ID corresponding to the kernel instance with the fastest running speed under each channel is selected. The regular distribution of the kernel ID corresponding to the kernel instance with the fastest running speed is as follows Figure 6b As shown in the figure, in the case of half-precision half type, as well as NCHW format and NHWC format, the running speed of each input size is as follows Figure 6b shown.

[0113] like Figure 6a and Figure 6bAs shown in the figure, as the kernel size is further increased, the performance advantage of the NCHW format over the NHWC format is further expanded. Only the kernel instance corresponding to <2,2,6,6> and the kernel instance corresponding to naive are compared. It can be seen that the kernel instance corresponding to <2,2,6,6> can be approximately considered to have advantages in both the NCHW format and the NHWC format under both float and half types. The optimal kernel instance of the four configurations is the kernel instance corresponding to Kernel ID=10.

[0114] In the embodiment of the present disclosure, for kernel size = 5, stride = 2, in the case of half-precision half data type, for NCHW format and NHWC format, each kernel instance corresponding to Kernel ID = 0, Kernel ID = 11, for each kernel instance under each configuration item, the speed is measured under multiple input sizes, and the running speed of each kernel instance under multiple input sizes is obtained. The kernel identifier corresponding to the kernel instance with the fastest kernel running speed under each input size is selected, and the regular distribution corresponding to the kernel identifier corresponding to the kernel instance with the fastest running speed is as follows: Figure 7a As shown in the figure, in the case of floating point type, and in NCHW format and NHWC format, the running speed of each input size is as follows Figure 7a As shown. For kernel size = 5, stride = 2, for half-precision half data types, for NCHW format and NHWC format, each Kernel ID = 0, Kernel ID = 11 corresponds to a kernel instance. For each kernel instance under each configuration item, the speed is measured under multiple input sizes, and the running speed of each kernel instance under multiple input sizes is obtained. The kernel ID corresponding to the kernel instance with the fastest running speed under each input size is selected. The regular distribution of the kernel ID corresponding to the kernel instance with the fastest running speed is shown as follows: Figure 7b As shown in the figure, in the case of half-precision half type, as well as NCHW format and NHWC format, the running speed of each input size is as follows Figure 7b shown.

[0115] like Figure 7a and Figure 7bAs shown in the figure, in the NHWC format, when the data type is float type or half type, the corresponding optimal kernel instance is the kernel instance corresponding to Kernel ID = 11. In the NCHW format, when the data type is float type or half type, the optimal kernel instance is segmented. When the input size is less than 36, the kernel instance corresponding to Kernel ID = 0 is selected as the optimal kernel instance. When the input size is greater than or equal to 36, the kernel instance corresponding to Kernel ID = 11 is selected as the optimal kernel instance.

[0116] In one embodiment of the present disclosure, based on the Lenovo PR model, the speed is compared with TRT based on the above configuration strategy. From the above OP analysis, it can be seen that the configuration of DWConv contained in the model is k3s1, and the inference data type is set to Half. The speed under all input sizes is tested as follows: Figure 8 shown.

[0117] like Fig. 9 As shown in the figure, under the half data type, the naive implementation is basically the same as the TRT implementation under large channel and small size. When the kernel ID corresponding to the optimal kernel instance is Kernel ID=5, the implementation is better than the TRT (8.2) implementation under full size, with a speedup gain of 30% to 90%, especially under small channel and large size.

[0118] S105: Generate a target operator based on the optimal kernel instance and the configuration item.

[0119] Based on the above embodiment, when the configuration item is kernel size=1, stride=1, and the data type is float type, regardless of whether the data format is NCHW or NHWC, the kernel instance corresponding to <1,1,1,1> is directly called to implement the underlying logic of the target operator. When the data type is half type, regardless of whether the data format is NCHW or NHWC, the kernel instance corresponding to <2,2,2,2> is directly called to implement the underlying logic of the target operator.

[0120] When the configuration items are kernel size = 3, stride = 1, regardless of whether the data type is float or half, and regardless of whether the data format is NCHW or NHWC, the kernel instance corresponding to <2,2,4,4> is directly called to implement the underlying logic of the target operator.

[0121] When the configuration items are kernel size = 3, stride = 2, no matter the data type is float or half, no matter the data format is NCHW or NHWC, the kernel instance corresponding to <2,2,4,4> is directly called to implement the underlying logic of the target operator.

[0122] When the configuration item is kernel size=3, stride=2, when the data format is NHWC and the data type is Float type or half type, directly call the kernel instance corresponding to <2,2,5,5> to implement the underlying logic of the target operator. When the data type is Float type and the data format is NCHW, directly call the kernel instance corresponding to <1,1,-1,-1> to implement the underlying logic of the target operator. When the data type is half type and the data format is NCHW, if the input size is less than 36, directly call the kernel instance corresponding to <1,1,-1,-1> to implement the underlying logic of the target operator. If the input size is greater than or equal to 36, directly call the kernel instance corresponding to <2,2,5,5> to implement the underlying logic of the target operator.

[0123] When the configuration items are kernel size = 5, stride = 1, regardless of whether the data type is float or half, and regardless of whether the data format is NCHW or NHWC, the kernel instance corresponding to <2,2,6,6> is directly called to implement the underlying logic of the target operator.

[0124] When the configuration item is kernel size=5, stride=2, and the data format is NHWC, regardless of whether the data type is float or half, directly call the kernel instance corresponding to <2,2,7,7> to implement the underlying logic of the target operator. When the data format is NCHW, the data type is float or half, and the input size is less than 36, directly call the kernel instance corresponding to <1,1,-1,-1> to implement the underlying logic of the target operator. When the input size is greater than or equal to 36, directly call the kernel instance corresponding to <2,2,7,7> to implement the underlying logic of the target operator.

[0125] In the disclosed embodiment, the optimal kernel instance determined by the target operator parameters and data information can be directly called, thus avoiding the problem of instantiating each convolution configuration.

[0126] Fig.10 is a schematic diagram of the structure of an operator generating device in an embodiment of the present disclosure. The operator generating device can be implemented in software and / or hardware. The operator generating device can be configured in Figure 1 In the electronic device described in.

[0127] like Fig.10 As shown, the operator generation device 100 provided in the embodiment of the present disclosure mainly includes: a neural network construction module 1001, a kernel instance configuration module 1002, a speed measurement result determination module 1003, an optimal kernel instance selection module 1004 and a target operator generation module 1005.

[0128] Among them, the neural network construction module 1001 is used to construct a deep convolutional neural network, wherein the number of group convolutions, the number of input channels and the number of output channels included in the deep convolutional neural network are the same, the number of filters in the deep convolutional neural network is the same as the number of output channels, and each filter only includes a kernel function with an input channel number of 1; the kernel instance configuration module 1002 is used to determine the configuration items of the deep convolutional neural network and multiple kernel instances corresponding to the configuration items, the configuration items include operator size and operator step size, and the kernel instance includes the size of the element block in the input feature map and the size of the element block in the output feature map; the speed measurement result determination module 1003 is used to run the deep convolutional neural network based on the kernel instance to obtain the speed measurement result corresponding to the kernel instance; the optimal kernel instance selection module 1004 is used to select the optimal kernel instance from the multiple kernel instances based on the speed measurement result; the target operator generation module 1005 is used to generate the target operator based on the optimal kernel instance and the configuration item.

[0129] In one embodiment of the present disclosure, the speed measurement result determination module 1003 is specifically used to run the deep convolutional neural network for multiple input data states based on different input sizes and the kernel instances to obtain the speed measurement results corresponding to each kernel instance under each input size, wherein the input data state refers to the data type of the input data corresponding to the deep convolutional neural network and the data format of the input data.

[0130] In one embodiment of the present disclosure, the optimal kernel instance selection module 1004 includes a fastest kernel instance screening unit, which is used to screen out the kernel instance with the fastest running speed under each input size from the speed measurement results corresponding to the input data state for each input data state; a kernel identifier acquisition unit, which is used to obtain the kernel identifier corresponding to each kernel instance with the fastest running speed; and an optimal kernel instance selection unit, which is used to select the optimal kernel instance from the multiple kernel instances based on the distribution pattern of the kernel identifier.

[0131] In one embodiment of the present disclosure, an optimal kernel instance selection unit is specifically used to determine that at least one input data state that meets the preset conditions corresponds to an optimal kernel instance if the distribution pattern of the kernel identifiers corresponding to at least one input data state meets the preset conditions, wherein the optimal kernel instance is the kernel identifier with the largest number among all the kernel identifiers corresponding to at least one input data state.

[0132] In one embodiment of the present disclosure, the optimal kernel instance selection unit is specifically used to segment the multiple input sizes based on the distribution law of the kernel identifier if the distribution law of the kernel identifier corresponding to at least one input data state does not meet the preset condition; each segment of the input size corresponds to an optimal kernel instance.

[0133] In one embodiment of the present disclosure, the distribution pattern of the kernel identifier meets a preset condition, including: a variation range of a speed measurement curve constructed by the kernel identifier is within a set range.

[0134] In one embodiment of the present disclosure, the kernel instance corresponding to each configuration item includes a preset kernel instance, wherein the height of the element block in the input feature map in the preset kernel instance is 1, the width of the element block in the input feature map is 1, the height of the element block in the output feature map is -1, and the width of the element block in the output feature map is -1.

[0135] The operator generation device provided in the embodiment of the present disclosure can execute the steps executed in the operator generation method provided in the embodiment of the method of the present disclosure, and the execution steps and beneficial effects are no longer repeated here.

[0136] Fig.11 Schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Fig.11 , which shows a schematic diagram of the structure of an electronic device 1100 suitable for implementing the embodiment of the present disclosure. The electronic device 1100 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), wearable terminal devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Fig.11 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0137] like Fig.11As shown, the electronic device 1100 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage device 1108 to a random access memory (RAM) 1103 to implement the operator generation method of the embodiment described in the present disclosure. In the RAM 1103, various programs and data required for the operation of the terminal device 1100 are also stored. The processing device 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0138] Typically, the following devices may be connected to the I / O interface 1105: an input device 1106 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1107 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1108 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1109. The communication device 1109 may allow the terminal device 1100 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.11 The terminal device 1100 is shown as having various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0139] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart, thereby implementing the operator generation method as described above. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1109, or installed from the storage device 1108, or installed from the ROM 1102. When the computer program is executed by the processing device 1101, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0140] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0141] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0142] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0143] The computer-readable medium carries one or more programs. When the one or more programs are executed by the terminal device, the terminal device: constructs a deep convolutional neural network, wherein the number of group convolutions, the number of input channels and the number of output channels included in the deep convolutional neural network are the same, the number of filters in the deep convolutional neural network is the same as the number of output channels, and each filter only includes a kernel function with an input channel number of 1; determines a configuration item of the deep convolutional neural network and multiple kernel instances corresponding to the configuration item, the configuration item includes an operator size and an operator step size, and the kernel instance includes the size of an element block in an input feature map and the size of an element block in an output feature map; runs the deep convolutional neural network based on the kernel instance to obtain a speed measurement result corresponding to the kernel instance; selects an optimal kernel instance from the multiple kernel instances based on the speed measurement result; and generates a target operator based on the optimal kernel instance and the configuration item.

[0144] Optionally, when the above one or more programs are executed by the terminal device, the terminal device may also execute other steps described in the above embodiments.

[0145] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0146] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0147] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, constitute a limitation on the unit itself.

[0148] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0149] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0150] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0151] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0152] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An operator generation method, characterized in that: include: Constructing a deep convolutional neural network, wherein the number of group convolutions, the number of input channels, and the number of output channels included in the deep convolutional neural network are the same, the number of filters in the deep convolutional neural network is the same as the number of output channels, and each filter only includes a kernel function with an input channel number of 1; Determine configuration items of the deep convolutional neural network and multiple kernel instances corresponding to the configuration items, the configuration items including operator size and operator step size, and the kernel instance including the size of the element block in the input feature map and the size of the element block in the output feature map; Running the deep convolutional neural network based on the kernel instance to obtain a speed measurement result corresponding to the kernel instance; Based on the speed measurement result, selecting an optimal kernel instance from the multiple kernel instances; A target operator is generated based on the optimal kernel instance and the configuration item.

2. The operator generation method according to claim 1, characterized in that: The step of running the deep convolutional neural network based on the kernel instance to obtain a speed measurement result corresponding to the kernel instance includes: For multiple input data states, the deep convolutional neural network is run based on different input sizes and the kernel instances to obtain speed measurement results corresponding to each kernel instance under each input size, wherein the input data state refers to the data type and data format of the input data corresponding to the deep convolutional neural network.

3. The operator generation method according to claim 3, characterized in that: The selecting the optimal kernel instance from the multiple kernel instances based on the speed measurement result includes: For each input data state, filter out the kernel instance with the fastest running speed under each input size from the speed test results corresponding to the input data state; Get the kernel ID corresponding to each fastest kernel instance; An optimal kernel instance is selected from the multiple kernel instances based on the distribution rule of the kernel identifiers.

4. The operator generation method according to claim 3, characterized in that: Selecting an optimal kernel instance from the multiple kernel instances based on the distribution rule of the kernel identifiers includes: If the distribution pattern of kernel identifiers corresponding to at least one input data state meets a preset condition, it is determined that at least one input data state that meets the preset condition corresponds to an optimal kernel instance, wherein the optimal kernel instance is the kernel identifier with the largest number among all kernel identifiers corresponding to at least one input data state.

5. The operator generation method according to claim 3, characterized in that: Selecting an optimal kernel instance from the multiple kernel instances based on the distribution rule of the kernel identifiers includes: If the distribution law of the kernel identifier corresponding to at least one input data state does not meet the preset condition, segmenting the plurality of input sizes based on the distribution law of the kernel identifier; There is one optimal kernel instance for each input size.

6. The operator generation method according to claim 4, characterized in that: The distribution rule of the kernel identification meets a preset condition, including: the variation range of the speed measurement curve constructed by the kernel identification is within a set range.

7. The operator generation method according to claim 1, characterized in that: The kernel instance corresponding to each configuration item includes a preset kernel instance, wherein the height of the element block in the input feature map in the preset kernel instance is 1, the width of the element block in the input feature map is 1, the height of the element block in the output feature map is -1, and the width of the element block in the output feature map is -1.

8. An operator generating device, characterized in that: include: A neural network construction module, used to construct a deep convolutional neural network, wherein the number of group convolutions, the number of input channels, and the number of output channels included in the deep convolutional neural network are the same, the number of filters in the deep convolutional neural network is the same as the number of output channels, and each filter only includes a kernel function with an input channel number of 1; A kernel instance configuration module, used to determine configuration items of the deep convolutional neural network and multiple kernel instances corresponding to the configuration items, wherein the configuration items include operator size and operator step size, and the kernel instance includes the size of the element block in the input feature map and the size of the element block in the output feature map; A speed measurement result determination module, configured to run the deep convolutional neural network based on the kernel instance to obtain a speed measurement result corresponding to the kernel instance; An optimal kernel instance selection module, configured to select an optimal kernel instance from the multiple kernel instances based on the speed measurement result; A target operator generation module is used to generate a target operator based on the optimal kernel instance and the configuration item.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program or instructions, which implements the method according to any one of claims 1 to 7 when executed by a processor.

Images