Operator compilation method, device and electronic device

Abstract

An embodiment of the present invention provides an operator compilation method, device, and electronic device. The method includes: the electronic device obtains a lookup table corresponding to the operator according to the type of the operator, and queries the optimal splitting configuration corresponding to the tensor shape from the lookup table according to the tensor shape of the operator, wherein the lookup table includes the tensor shape and the optimal splitting configuration corresponding to the tensor shape, the electronic device compiles the operator according to the tensor shape and the optimal splitting configuration corresponding to the tensor shape, and generates a first data file. During the operator compilation process, the optimal splitting configuration is obtained by obtaining the lookup table corresponding to the operator and the operator is compiled using the optimal splitting configuration, thereby improving flexibility, improving computing unit utilization, reducing bandwidth, and accelerating computing speed when implementing operator compilation.

Description

Technical field

[0001] Embodiments of the present invention relate to the field of artificial intelligence (AI) technology, and in particular to an operator compilation method, device, and electronic device. [Background Technology]

[0002] With the development of artificial intelligence, neural network models are becoming increasingly complex. During the neural network operation process, a large number of operator compilations are required. As the number of neural network model parameters increases, the speed of operator compilation significantly affects the running speed of the neural network model. Therefore, improving the efficiency of operator compilation is essential.

[0003] Currently, when compiling operators, a theoretically optimal compilation process is usually achieved based on a fixed hardware algorithm and energy consumption model. The hardware and algorithm are bound together. When the hardware is changed to compile the operator, the algorithm needs to be iterated, resulting in poor flexibility. A complete energy consumption model is very difficult to design and requires multiple iterations to mature. If the energy consumption model is not designed properly, it may cause low computing unit utilization, high bandwidth, and slow computing speed during operator compilation. [Summary of the invention]

[0004] In view of this, embodiments of the present invention provide an operator compilation method, apparatus, and electronic device to improve flexibility, increase computing unit utilization, reduce bandwidth, and speed up computing when implementing operator compilation.

[0005] In a first aspect, an embodiment of the present invention provides an operator compilation method, the method comprising:

[0006] Obtaining a lookup table corresponding to the operator according to the type of the operator;

[0007] According to the tensor shape of the operator, querying the optimal segmentation configuration corresponding to the tensor shape from the lookup table, wherein the lookup table includes the tensor shape and the optimal segmentation configuration corresponding to the tensor shape;

[0008] The operator is compiled according to the tensor shape and the optimal split configuration corresponding to the tensor shape to generate a first data file.

[0009] Optionally, the operator corresponds to a plurality of set tensor shapes; before obtaining the lookup table corresponding to the operator according to the type of the operator, the method further includes:

[0010] The lookup table corresponding to the operator is updated according to each set tensor shape and the generated optimal segmentation configuration corresponding to each set tensor shape.

[0011] Optionally, updating the lookup table corresponding to the operator according to each set tensor shape and the generated optimal segmentation configuration corresponding to each set tensor shape includes:

[0012] According to the i-th set tensor shape, generate the optimal segmentation configuration corresponding to the i-th set tensor shape;

[0013] Updating the lookup table corresponding to the operator according to the i-th set tensor shape and the optimal segmentation configuration corresponding to the i-th set tensor shape;

[0014] Determining whether the sequence number of the i-th set tensor shape is less than or equal to the total number of the plurality of set tensor shapes;

[0015] If it is determined that the serial number of the i-th set tensor shape is less than the total number of multiple set tensor shapes, the i+1-th set tensor shape is used as the i-th set tensor shape, and the step of generating the optimal segmentation configuration corresponding to the i-th set tensor shape based on the i-th set tensor shape is executed, where i is a positive integer.

[0016] Optionally, generating an optimal segmentation configuration corresponding to the i-th set tensor shape according to the i-th set tensor shape includes:

[0017] Generating a plurality of segmentation configurations corresponding to the i-th set tensor shape according to the i-th set tensor shape;

[0018] According to the i-th set tensor shape and the j-th segmentation configuration corresponding to the i-th set tensor shape, an optimal segmentation configuration corresponding to the i-th set tensor shape is determined, where j is a positive integer.

[0019] Optionally, before updating the lookup table corresponding to the operator according to each set tensor shape and the generated optimal segmentation configuration corresponding to each set tensor shape, the method further includes:

[0020] On multiple dimensions of the tensor of the operator, multiple set tensor shapes are generated according to the set upper limit of each dimension and the set dimension step corresponding to each dimension.

[0021] Optionally, generating a plurality of segmentation configurations corresponding to the i-th set tensor shape according to the i-th set tensor shape includes:

[0022] On different dimensions of the i-th set tensor shape, the i-th set tensor shape is segmented according to the set dimension segmentation thresholds corresponding to the different dimensions, and multiple segmentation configurations corresponding to the i-th set tensor shape are generated.

[0023] Optionally, determining the optimal segmentation configuration corresponding to the i-th set tensor shape according to the i-th set tensor shape and the j-th segmentation configuration corresponding to the i-th set tensor shape includes:

[0024] Determining an optimal split configuration according to an i-th set tensor shape and a j-th split configuration corresponding to the i-th set tensor shape;

[0025] Determine whether the sequence number of the j-th split configuration is less than or equal to the total number of the multiple split configurations corresponding to the i-th set tensor shape;

[0026] If it is determined that the serial number of the j-th split configuration is less than the total number of multiple split configurations corresponding to the i-th set tensor shape, the j+1-th split configuration is used as the j-th split configuration, and the step of determining the optimal split configuration based on the i-th set tensor shape and the j-th split configuration corresponding to the i-th set tensor shape is executed.

[0027] Optionally, determining the optimal segmentation configuration according to the i-th set tensor shape and the j-th segmentation configuration corresponding to the i-th set tensor shape includes:

[0028] According to the i-th set tensor shape and the j-th slicing configuration corresponding to the i-th set tensor shape, obtaining a comprehensive performance value corresponding to the j-th slicing configuration;

[0029] Determine whether the comprehensive performance value corresponding to the j-th slicing configuration is better than the comprehensive performance value corresponding to the j-1-th slicing configuration;

[0030] If it is determined that the comprehensive performance value corresponding to the j-th slicing configuration is better than the comprehensive performance value corresponding to the j-1-th slicing configuration, the j-th slicing configuration is set as the optimal slicing configuration; if it is determined that the comprehensive performance value corresponding to the j-th slicing configuration is not better than the comprehensive performance value corresponding to the j-1-th slicing configuration, the j-1-th slicing configuration is set as the optimal slicing configuration.

[0031] Optionally, obtaining a comprehensive performance value corresponding to the j-th slicing configuration according to the i-th set tensor shape and the j-th slicing configuration corresponding to the i-th set tensor shape includes:

[0032] Compiling the operator according to the i-th set tensor shape and the j-th slicing configuration corresponding to the i-th set tensor shape to generate a second data file;

[0033] sending the second data file to the mobile terminal;

[0034] receiving a plurality of performance values obtained and sent by the mobile terminal when the mobile terminal runs the second data file;

[0035] A weighted calculation is performed on the multiple performance values to generate a comprehensive performance value corresponding to the j-th segmentation configuration.

[0036] Optionally, obtaining a comprehensive performance value corresponding to the j-th slicing configuration according to the i-th set tensor shape and the j-th slicing configuration corresponding to the i-th set tensor shape includes:

[0037] Compiling the operator according to the i-th set tensor shape and the j-th slicing configuration corresponding to the i-th set tensor shape to generate a second data file;

[0038] running the second data file to obtain a plurality of performance values;

[0039] A weighted calculation is performed on the multiple performance values to generate a comprehensive performance value corresponding to the j-th segmentation configuration.

[0040] Optionally, it also includes:

[0041] If it is determined that the serial number of the j-th segmentation configuration is equal to the total number of multiple segmentation configurations corresponding to the i-th set tensor shape, the step of updating the lookup table corresponding to the operator according to the i-th set tensor shape and the optimal segmentation configuration corresponding to the i-th set tensor shape is executed.

[0042] In a second aspect, an embodiment of the present invention provides an operator compilation device, the device comprising:

[0043] An acquisition module, configured to acquire a lookup table corresponding to an operator according to the type of the operator;

[0044] A query module, configured to query the lookup table for an optimal segmentation configuration corresponding to the tensor shape according to the tensor shape of the operator, wherein the lookup table includes the tensor shape and the optimal segmentation configuration corresponding to the tensor shape;

[0045] A generation module is used to compile the operator according to the tensor shape and the optimal segmentation configuration corresponding to the tensor shape to generate a first data file.

[0046] Optionally, it further includes: an update module;

[0047] The updating module is used to update the lookup table corresponding to the operator according to each set tensor shape and the generated optimal segmentation configuration corresponding to each set tensor shape.

[0048] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the operator compilation method in the first aspect or any possible implementation of the first aspect.

[0049] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the device, enable the device to execute the operator compilation method in the first aspect or any possible implementation of the first aspect.

[0050] In the technical solution provided by the embodiment of the present invention, the electronic device obtains the lookup table corresponding to the operator according to the type of the operator, and queries the optimal splitting configuration corresponding to the tensor shape from the lookup table according to the tensor shape of the operator, wherein the lookup table includes the tensor shape and the optimal splitting configuration corresponding to the tensor shape. The electronic device compiles the operator according to the tensor shape and the optimal splitting configuration corresponding to the tensor shape to generate a first data file. During the operator compilation process, the optimal splitting configuration is obtained by obtaining the lookup table corresponding to the operator and the operator is compiled according to the optimal splitting configuration, thereby improving flexibility, improving computing unit utilization, reducing bandwidth and accelerating computing speed when implementing operator compilation.

Brief Description of the Drawings

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 A flowchart of an operator compilation method provided by an embodiment of the present invention;

[0053] Figure 2 A flowchart of a lookup table updating method provided by an embodiment of the present invention;

[0054] Figure 3 A flowchart of a method for determining an optimal partitioning configuration provided by an embodiment of the present invention;

[0055] Figure 4 A flowchart of another method for determining an optimal partitioning configuration provided by an embodiment of the present invention;

[0056] Figure 5A flowchart of another method for determining an optimal partitioning configuration provided by an embodiment of the present invention;

[0057] Figure 6 A flowchart of a method for obtaining comprehensive performance values provided by an embodiment of the present invention;

[0058] Figure 7 A flowchart of another method for obtaining comprehensive performance values provided by an embodiment of the present invention;

[0059] Figure 8 A schematic diagram of the structure of an operator compilation device provided by an embodiment of the present invention;

[0060] Figure 9 A schematic diagram of an electronic device provided by an embodiment of the present invention. [Specific implementation method]

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0062] It should be understood that the embodiments described are only a portion of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0063] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "an", "the" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0064] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the associated objects.

[0065] The word "if," as used herein, may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0066] Figure 1 A flowchart of an operator compilation method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes:

[0067] Step 11: Obtain the lookup table corresponding to the operator according to the operator type.

[0068] Each step of the embodiment of the present invention may be performed by an electronic device, including but not limited to a mobile phone, a tablet computer, a portable PC, a desktop computer, a wearable device, etc.

[0069] The embodiments of the present invention can be implemented through various neural network back-end compilers, wherein the neural network back-end compiler is software installed in an electronic device.

[0070] In the embodiments of the present invention, operators are the basic units of neural network calculations. For example, operators may include convolution, pooling, activation, or normalization. Taking the convolution operator as an example, operators may be classified by type, such as two-dimensional convolution or three-dimensional convolution. Different types of operators correspond to different lookup tables. For example, if the operator type includes two-dimensional convolution, the electronic device obtains the lookup table corresponding to the two-dimensional convolution based on the two-dimensional convolution.

[0071] Step 12: According to the tensor shape of the operator, an optimal segmentation configuration corresponding to the tensor shape is searched from a lookup table, wherein the lookup table includes the tensor shape and the optimal segmentation configuration corresponding to the tensor shape.

[0072] Each tensor shape and its corresponding optimal split configuration can be stored in a lookup table as a key-value pair. For example, when an operator's tensor shape is [100, 100, 100, 100], the electronic device retrieves the optimal split configuration corresponding to that tensor shape from the lookup table as "split into four tensor shapes along the H dimension, each with a shape of [100, 25, 100, 100]."

[0073] Step 13: Compile the operator according to the tensor shape and the optimal split configuration corresponding to the tensor shape to generate a first data file.

[0074] The electronic device compiles the parameters of the operator according to the tensor shape and the optimal split configuration corresponding to the tensor shape to generate an operator parameter structure; the electronic device serializes the operator parameter structure to generate a first data file, wherein the type of the first data file is a binary file; the first data file includes but is not limited to binary weight data.

[0075] In the embodiment of the present invention, before step 11, the following steps are further included:

[0076] Step 10: Update the lookup table corresponding to the operator according to each set tensor shape and the generated optimal segmentation configuration corresponding to each set tensor shape.

[0077] In the embodiment of the present invention, before step 10, the following steps are further included:

[0078] In the multiple dimensions of the operator's tensor, the electronic device generates multiple set tensor shapes according to the upper limit of each dimension set and the dimensional step size corresponding to each dimension set. For example, the operator's tensor consists of four dimensions: batch (Batch, N), height (Height, H), width (Width, W) and channel (C). The electronic device sets the upper limit of the four dimensions N, H, W and C to 100, and sets the corresponding dimensional step size of each dimension to 10. Then, the electronic device can generate multiple set tensor shapes according to the upper limit of each dimension of N, H, W and C set to 100 and the dimensional step size corresponding to each dimension set to 10. For example, the generated multiple set tensor shapes include [10, 10, 10, 10], [20, 10, 10, 10], [10, 20, 10, 10], [10, 10, 20, 10], ... [100, 100, 100, 100].

[0079] Each operator can correspond to multiple set tensor shapes, and the electronic device can generate multiple slicing configurations corresponding to each set tensor shape based on the set tensor shape. The electronic device obtains the optimal slicing configuration from the multiple slicing configurations corresponding to each set tensor shape, and saves each set tensor shape and the optimal slicing configuration corresponding to each set tensor shape in the lookup table corresponding to the operator, thereby updating the lookup table.

[0080] For example, the multiple set tensor shapes of the operator include [10, 10, 10, 10], [20, 10, 10, 10], [10, 20, 10, 10], [10, 10, 20, 10], ... [100, 100, 100, 100]. For example, when the tensor shape is set to [100, 100, 100, 100], the multiple split configurations corresponding to [100, 100, 100, 100] include: "Split into 2 tensor shapes from the H dimension, each tensor shape is [100, 50, 100, 100]", "Split into 4 tensor shapes from the H dimension, each tensor shape is [100, 25, 100, 100]", "Split into 2 tensor shapes from the W ... Tensor shape, each tensor shape is [100, 100, 50, 100]", "split from the W dimension into 4 tensor shapes, each tensor shape is [100, 100, 25, 100]", split from the C dimension into 2 tensor shapes, each tensor shape is [100, 100, 100, 50] and "split from the C dimension into 4 tensor shapes, each tensor shape is [100, 100, 100, 25]".

[0081] For example, in the H dimension of [100, 100, 100, 100], the H dimension of [100, 100, 100, 100] is split according to the dimension splitting threshold corresponding to the H dimension, generating multiple split configurations corresponding to [100, 100, 100, 100]. For example, when the dimension splitting threshold is 2, the split configuration includes "split from the H dimension into 2 tensor shapes, each tensor shape is [100, 50, 100, 100]"; for another example, when the dimension splitting threshold is 4, the split configuration includes "split from the H dimension into 4 tensor shapes, each tensor shape is [100, 25, 100, 100]".

[0082] For another example, on the W dimension of [100, 100, 100, 100], the W dimension of [100, 100, 100, 100] is split according to the dimension splitting threshold corresponding to the W dimension, generating multiple split configurations corresponding to [100, 100, 100, 100]. For example, when the dimension splitting threshold is 2, the split configuration includes "split from the W dimension into 2 tensor shapes, each tensor shape is [100, 100, 50, 100]"; for another example, when the dimension splitting threshold is 4, the split configuration includes "split from the W dimension into 4 tensor shapes, each tensor shape is [100, 100, 25, 100]".

[0083] For another example, on the C dimension of [100, 100, 100, 100], the C dimension of [100, 100, 100, 100] is split according to the dimension splitting threshold corresponding to the C dimension, generating multiple split configurations corresponding to [100, 100, 100, 100]. For example, when the dimension splitting threshold is 2, the split configuration includes "split from the C dimension into 2 tensor shapes, each tensor shape is [100, 100, 100, 50]"; for another example, when the dimension splitting threshold is 4, the split configuration includes "split from the C dimension into 4 tensor shapes, each tensor shape is [100, 100, 100, 25]".

[0084] In summary, the generated multiple split configurations may include "split into 2 tensor shapes from the H dimension, each tensor shape is [100, 50, 100, 100]", "split into 4 tensor shapes from the H dimension, each tensor shape is [100, 25, 100, 100]", "split into 2 tensor shapes from the W dimension, each tensor shape is [100, 100, 50, 100]", "split into 4 tensor shapes from the W dimension, each tensor shape is [100, 100, 25, 100]", "split into 2 tensor shapes from the C dimension, each tensor shape is [100, 100, 100, 50]", and "split into 4 tensor shapes from the C dimension, each tensor shape is [100, 100, 100, 25]".

[0085] For example, when the optimal split configuration corresponding to the tensor shape [100, 100, 100, 100] is "split into 4 tensor shapes from H dimensions, each with a shape of [100, 25, 100, 100]", [100, 100, 100, 100] and the corresponding "split into 4 tensor shapes from H dimensions, each with a shape of [100, 25, 100, 100]" are saved to the lookup table corresponding to the operator.

[0086] In an embodiment of the present invention, the electronic device obtains a lookup table corresponding to the operator according to the type of operator, queries the optimal splitting configuration corresponding to the tensor shape from the lookup table according to the tensor shape of the operator, and compiles the operator according to the tensor shape and the optimal splitting configuration corresponding to the tensor shape to generate a first data file. During the operator compilation process, the optimal splitting configuration is obtained by obtaining the lookup table corresponding to the operator and the operator is compiled according to the optimal splitting configuration, thereby improving flexibility, improving computing unit utilization, reducing bandwidth, and accelerating computing speed when implementing operator compilation. When the operator is compiled, the operator is compiled by obtaining the lookup table, which realizes automatic adaptation to changes in the back-end hardware. When the back-end hardware changes, it will not affect the operator compilation process.

[0087] Figure 2A flowchart of a lookup table updating method provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, step 10 specifically includes:

[0088] Step 101: Generate an optimal segmentation configuration corresponding to the i-th set tensor shape according to the i-th set tensor shape.

[0089] For example, when i is 1 and the first set tensor shape is [10, 10, 10, 10], the electronic device generates an optimal split configuration corresponding to [10, 10, 10, 10] based on [10, 10, 10, 10]. The optimal split configuration is: "Split from the C dimension into 2 tensor shapes, each with a shape of [10, 10, 10, 5]. For another example, when i is 100 and the 100th set tensor shape is [100, 100, 100, 100], the electronic device generates an optimal split configuration corresponding to [100, 100, 100, 100] based on [100, 100, 100]. The optimal split configuration is: "Split from the H dimension into 4 tensor shapes, each with a shape of [100, 25, 100, 100]. i is a positive integer, for example, i=1, 2, ..., N.

[0090] Figure 3 A flow chart of a method for determining an optimal partitioning configuration provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, step 101 specifically includes:

[0091] Step 1011: Generate multiple segmentation configurations corresponding to the i-th set tensor shape according to the i-th set tensor shape.

[0092] At different dimensions of the i-th set tensor shape, the electronic device segments the i-th set tensor shape according to the dimension segmentation thresholds corresponding to the different dimensions, generating multiple segmentation configurations corresponding to the i-th set tensor shape. During the operator compilation process, in order to achieve efficient inference operations in a multi-processor architecture, the operator's tensor shape will be segmented data-independently. The segmented tensor shapes can participate in independent operations, allowing multiple processors to perform concurrent operations on the operator at the same time.

[0093] During operator compilation, the set tensor shape is typically split into several equal-sized blocks along the three dimensions H, W, and C. These equal-sized blocks can be split configurations. The electronic device splits the set tensor shape based on the dimensional split thresholds corresponding to the different dimensions. The same set tensor shape can generate dozens or even hundreds of split configurations, and different split configurations can affect the efficiency of operator compilation.

[0094] For example, when i is equal to 100 and the 100th set tensor shape is [100, 100, 100, 100], the upper limits of the dimension splitting thresholds of H, W and C dimensions of [100, 100, 100, 100] are all set to 4, and the lower limits of the dimension splitting thresholds are all set to 1. The electronic device splits [100, 100, 100, 100] according to the dimension splitting thresholds of H, W or C, and generates multiple splitting configurations corresponding to [100, 100, 100, 100]. Among them, multiple split configurations include: "Split into 2 tensor shapes from the H dimension, each tensor shape is [100, 50, 100, 100]", "Split into 4 tensor shapes from the H dimension, each tensor shape is [100, 25, 100, 100]", "Split into 2 tensor shapes from the W dimension, each tensor shape is [100, 100, 50, 100]", "Split into 4 tensor shapes from the W dimension, each tensor shape is [100, 100, 25, 100]", split into 2 tensor shapes from the C dimension, each tensor shape is [100, 100, 100, 50] and "Split into 4 tensor shapes from the C dimension, each tensor shape is [100, 100, 100, 25]".

[0095] For example, in the H dimension of [100, 100, 100, 100], the electronic device sets the upper limit of the dimension splitting threshold of the H dimension to 4, and sets the lower limit of the dimension splitting threshold to 1. Then the electronic device splits the H dimension of [100, 100, 100, 100] according to the set upper limit 4 of the dimension splitting threshold and the lower limit 1 of the dimension splitting threshold of the H dimension, and generates multiple splitting configurations corresponding to [100, 100, 100, 100]. The multiple splitting configurations include "split from the H dimension into 2 tensor shapes, each tensor shape is [100, 50, 100, 100]", "Split from the H dimension into 4 tensor shapes, each tensor shape is [100, 25, 100, 100]". For another example, in the C dimension of [100, 100, 100, 100], the electronic device sets the upper limit of the dimension splitting threshold of the C dimension to 4, and sets the lower limit of the dimension splitting threshold to 1. Then the electronic device splits the C dimension of [100, 100, 100, 100] according to the set upper limit 4 of the dimension splitting threshold and the lower limit 1 of the dimension splitting threshold of the C dimension, and generates multiple splitting configurations corresponding to [100, 100, 100, 100]. The multiple splitting configurations include "split from the C dimension into 2 tensor shapes, each tensor shape is [100, 100, 100, 50]" and "split from the C dimension into 4 tensor shapes, each tensor shape is [100, 100, 100, 25]".

[0096] Step 1012: Determine the optimal segmentation configuration corresponding to the i-th set tensor shape based on the i-th set tensor shape and the j-th segmentation configuration corresponding to the i-th set tensor shape, where j is a positive integer.

[0097] The electronic device obtains the comprehensive performance value corresponding to the jth slicing configuration based on the i-th set tensor shape and the j-th slicing configuration corresponding to the i-th set tensor shape, compares the comprehensive performance value corresponding to the j-th slicing configuration with the comprehensive performance value corresponding to the j-1-th slicing configuration, and determines the optimal slicing configuration corresponding to the i-th set tensor shape based on the comparison result. For example, when i is 100 and j is 6, the electronic device obtains the comprehensive performance value corresponding to the 6th slicing configuration based on the 100th set tensor shape and the 6th slicing configuration corresponding to the 100th set tensor shape, compares the comprehensive performance value corresponding to the 6th slicing configuration with the comprehensive performance value corresponding to the 5th slicing configuration, and determines the optimal slicing configuration corresponding to the 100th set tensor shape based on the comparison result.

[0098] Step 102: Update the lookup table corresponding to the operator according to the i-th set tensor shape and the optimal segmentation configuration corresponding to the i-th set tensor shape.

[0099] The electronic device saves the i-th set tensor shape and the optimal split configuration corresponding to the i-th set tensor shape to the lookup table corresponding to the operator to update the lookup table. For example, when i is 1 and the first set tensor shape is [10, 10, 10, 10], the optimal split configuration corresponding to [10, 10, 10, 10] is "split into 2 tensor shapes from the C dimension, each tensor shape is [10, 10, 10, 5]", and [10, 10, 10, 10] and the corresponding "split into 2 tensor shapes from the C dimension, each tensor shape is [10, 10, 10, 5]" are saved to the lookup table corresponding to the operator. For another example, when i is equal to 100 and the 100th set tensor shape is [100, 100, 100, 100], the optimal split configuration corresponding to [100, 100, 100, 100] is "split into 4 tensor shapes from H dimension, each tensor shape is [100, 25, 100, 100]", and [100, 100, 100, 100] and the corresponding "split into 4 tensor shapes from H dimension, each tensor shape is [100, 25, 100, 100]" are saved to the lookup table corresponding to the operator.

[0100] Step 103: Determine whether the sequence number of the i-th set tensor shape is less than or equal to the total number of the multiple set tensor shapes. If so, execute step 104; if so, the process ends.

[0101] If the electronic device determines that the serial number of the i-th set tensor shape is less than the total number of multiple set tensor shapes, it indicates that it is necessary to determine the optimal segmentation configuration corresponding to the remaining set tensor shapes, and then execute step 104; if the electronic device determines that the serial number of the i-th set tensor shape is equal to the total number of multiple set tensor shapes, it indicates that all set tensor shapes have been traversed and the optimal segmentation configuration corresponding to each set tensor shape has been determined, the update of the lookup table is completed, and the process ends. For example, the total number of multiple set tensor shapes is 100. When i is equal to 66 and the serial number of the 66th set tensor shape is 66, the serial number of the 66th set tensor shape is less than the total number of multiple set tensor shapes, indicating that the optimal segmentation configuration corresponding to the remaining set tensor shapes needs to be determined, and step 104 is executed; when i is equal to 100 and the serial number of the 100th set tensor shape is 100, the serial number of the 100th set tensor shape is equal to the total number of multiple set tensor shapes, indicating that all set tensor shapes have been traversed and the optimal segmentation configuration corresponding to each set tensor shape has been determined, the update of the lookup table is completed, and the process ends.

[0102] Step 104 : Set the (i+1)th set tensor shape as the (i)th set tensor shape, and execute step 101 .

[0103] When the electronic device determines that the sequence number of the i-th set tensor shape is less than the total number of the multiple set tensor shapes, indicating that it is necessary to determine the optimal segmentation configuration corresponding to the remaining set tensor shapes, the i+1-th set tensor shape is used as the i-th set tensor shape, and the optimal segmentation configuration corresponding to the i-th set tensor shape is generated based on the i-th set tensor shape. For example, when i is equal to 2 and the sequence number of the second set tensor shape is 2, at this time, the sequence number of the second set tensor shape is less than the total number of the multiple set tensor shapes, the third set tensor shape is used as the i-th set tensor shape, that is, i=3, and the step of generating the optimal segmentation configuration corresponding to the third set tensor shape based on the third set tensor shape in step 101 is executed.

[0104] In an embodiment of the present invention, the electronic device generates an optimal splitting configuration corresponding to the i-th set tensor shape based on the i-th set tensor shape, and updates the lookup table corresponding to the operator based on the i-th set tensor shape and the optimal splitting configuration corresponding to the i-th set tensor shape. In the process of updating the lookup table, the electronic device performs data-independent splitting on the tensor shape of the operator. The split tensor shape can participate in the calculation independently, and by determining the optimal splitting configuration, efficient concurrent calculation is achieved.

[0105] Figure 4 A flowchart of another method for determining an optimal partitioning configuration provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown, step 1112 specifically includes:

[0106] Step S1: Determine an optimal segmentation configuration based on the i-th set tensor shape and the j-th segmentation configuration corresponding to the i-th set tensor shape.

[0107] The electronic device obtains the comprehensive performance value corresponding to the jth slicing configuration based on the i-th set tensor shape and the j-th slicing configuration corresponding to the i-th set tensor shape, compares the comprehensive performance value corresponding to the j-th slicing configuration with the comprehensive performance value corresponding to the j-1-th slicing configuration, and determines the optimal slicing configuration based on the comparison result. For example, when i is 8 and j is 6, the electronic device obtains the comprehensive performance value corresponding to the 6th slicing configuration based on the 8th set tensor shape and the 6th slicing configuration corresponding to the 8th set tensor shape, compares the comprehensive performance value corresponding to the 6th slicing configuration with the comprehensive performance value corresponding to the 5th slicing configuration, and determines the optimal slicing configuration corresponding to the 8th set tensor shape based on the comparison result.

[0108] Step S2: Determine whether the sequence number of the j-th segmentation configuration is less than or equal to the total number of multiple segmentation configurations corresponding to the i-th set tensor shape. If so, execute step S3; if so, execute step 102.

[0109] If the electronic device determines that the serial number of the j-th split configuration is less than the total number of multiple split configurations corresponding to the ith set tensor shape, it indicates that it is necessary to determine whether the remaining split configurations corresponding to the ith set tensor shape are the optimal split configurations of the tensor shape, then step S3 is executed; if the electronic device determines that the serial number of the j-th split configuration is equal to the total number of multiple split configurations corresponding to the ith set tensor shape, it indicates that all split configurations corresponding to the ith set tensor shape have been traversed, and the optimal split configuration corresponding to the set tensor shape has been determined, then step 102 is executed. For example, when i is equal to 3 and j is equal to 6, the total number of multiple split configurations corresponding to the third set tensor shape is 9, and the serial number of the sixth split configuration is 6. At this time, the serial number of the sixth split configuration is less than the total number of multiple split configurations, indicating that it is necessary to determine whether the remaining split configurations corresponding to the third set tensor shape are the optimal split configurations of the tensor shape, then execute step S3; when j is equal to 9, the serial number of the ninth split configuration is 9. At this time, the serial number of the ninth split configuration is equal to the total number of multiple split configurations, indicating that all split configurations corresponding to the third set tensor shape have been traversed, and the optimal split configuration corresponding to the tensor shape has been determined, then execute step 102.

[0110] Step S3: Use the j+1th segmentation configuration as the jth segmentation configuration, and execute step S1.

[0111] When the electronic device determines that the sequence number of the jth split configuration is less than the total number of the multiple split configurations corresponding to the i-th set tensor shape, the j+1th split configuration is used as the j-th split configuration, and the optimal split configuration is determined based on the i-th set tensor shape and the j-th split configuration corresponding to the i-th set tensor shape. For example, when i is equal to 3 and j is equal to 2, the sequence number of the second split configuration is 2, and the total number of the multiple split configurations corresponding to the third set tensor shape is 9. At this time, the sequence number of the second split configuration is less than the total number of the multiple split configurations, and the third split configuration is used as the j-th split configuration, that is, j=3, and the optimal split configuration is determined based on the third set tensor shape and the third split configuration corresponding to the third set tensor shape in step S1.

[0112] Figure 5 A flowchart of another method for determining an optimal partitioning configuration provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, step S1 specifically includes:

[0113] Step S11: According to the i-th set tensor shape and the j-th segmentation configuration corresponding to the i-th set tensor shape, obtain a comprehensive performance value corresponding to the j-th segmentation configuration.

[0114] For example, when i is 100 and j is 6, the electronic device obtains the comprehensive performance value corresponding to the 6th slicing configuration based on the 100th set tensor shape and the 6th slicing configuration corresponding to the 100th set tensor shape. When the 100th set tensor shape is [100, 100, 100, 100] and the 6th slicing configuration is "sliced from the C dimension into 4 tensor shapes, each tensor shape is [100, 100, 100, 25]", the electronic device obtains the comprehensive performance value corresponding to the 6th slicing configuration based on [100, 100, 100, 100] and "sliced from the C dimension into 4 tensor shapes, each tensor shape is [100, 100, 100, 25]". i is a positive integer, for example, i = 1, 2, ... N). j is a positive integer, for example, j = 1, 2, ... M.

[0115] Step S12: Determine whether the comprehensive performance value corresponding to the j-th segmentation configuration is better than the comprehensive performance value corresponding to the j-1-th segmentation configuration. If so, execute step S13; if not, execute step S14.

[0116] If the electronic device determines that the comprehensive performance value corresponding to the j-th split configuration is better than the comprehensive performance value corresponding to the j-1-th split configuration, indicating that the j-th split configuration is better than the j-1-th split configuration, then step S13 is executed; if the electronic device determines that the comprehensive performance value corresponding to the j-th split configuration is not better than the comprehensive performance value corresponding to the j-1-th split configuration, indicating that the j-1-th split configuration is better than the j-th split configuration, then step S14 is executed. For example, when j is equal to 6, the comprehensive performance value corresponding to the 6th slicing configuration is 3, and the comprehensive performance value corresponding to the 5th slicing configuration is 7. At this time, the comprehensive performance value corresponding to the 6th slicing configuration is better than the comprehensive performance value corresponding to the 5th slicing configuration, indicating that the 6th slicing configuration is better than the 5th slicing configuration, and step S13 is executed; when the comprehensive performance value corresponding to the 6th slicing configuration is 9, and the comprehensive performance value corresponding to the 5th slicing configuration is 7, at this time, the comprehensive performance value corresponding to the 6th slicing configuration is not better than the comprehensive performance value corresponding to the 5th slicing configuration, indicating that the 5th slicing configuration is better than the 6th slicing configuration, and step S14 is executed.

[0117] Step S13: Set the j-th segmentation configuration as the optimal segmentation configuration.

[0118] When the electronic device determines that the comprehensive performance value corresponding to the jth slicing configuration is better than the comprehensive performance value corresponding to the j-1th slicing configuration, the jth slicing configuration is set as the optimal slicing configuration. For example, when j is 6, the comprehensive performance value corresponding to the 6th slicing configuration is 3, and the comprehensive performance value corresponding to the 5th slicing configuration is 7. In this case, the electronic device determines that the comprehensive performance value corresponding to the 6th slicing configuration is better than the comprehensive performance value corresponding to the 5th slicing configuration, and sets the 6th slicing configuration as the optimal slicing configuration.

[0119] Step S14: taking the j-1th segmentation configuration as the optimal segmentation configuration.

[0120] When the electronic device determines that the comprehensive performance value corresponding to the j-th split configuration is not better than the comprehensive performance value corresponding to the j-1-th split configuration, the electronic device sets the j-1-th split configuration as the optimal split configuration. For example, when j is 6, the comprehensive performance value corresponding to the 6th split configuration is 9, and the comprehensive performance value corresponding to the 5th split configuration is 7. In this case, the electronic device determines that the comprehensive performance value corresponding to the 6th split configuration is not better than the comprehensive performance value corresponding to the 5th split configuration, and sets the 5th split configuration as the optimal split configuration.

[0121] Figure 6 This is a flow chart of a method for obtaining comprehensive performance values provided in an embodiment of the present invention. Figure 6 As shown, as an optional solution, step S11 may specifically include:

[0122] Step S111 : Compile an operator according to the i-th set tensor shape and the j-th split configuration corresponding to the i-th set tensor shape to generate a second data file.

[0123] The electronic device compiles the parameters of the operator according to the i-th set tensor shape and the j-th split configuration corresponding to the i-th set tensor shape to generate an operator parameter structure; the electronic device serializes the operator parameter structure to generate a second data file, wherein the type of the second data file is a binary file; the second data file includes but is not limited to binary weight data.

[0124] Step S112: Send the second data file to the mobile terminal.

[0125] In the embodiment of the present invention, mobile terminals include but are not limited to mobile phones, wearable devices, etc.

[0126] When the electronic device is a tablet computer, a portable PC or a desktop computer, the electronic device sends the second data file to the mobile terminal through a Universal Serial Bus (USB) interface.

[0127] Step S113: Receive multiple performance values acquired and sent by the mobile terminal when running the second data file.

[0128] The plurality of performance values include but are not limited to computing time and bandwidth data. The electronic device receives the computing time and bandwidth data acquired and sent by the mobile terminal via the USB interface.

[0129] In an embodiment of the present invention, after receiving the second data file sent by the electronic device through the USB interface, the mobile terminal saves the second data file locally, and the processor of the mobile terminal runs the second data file. The mobile terminal obtains multiple performance values through the operation log.

[0130] Step S114: Perform weighted calculation on the multiple performance values to generate a comprehensive performance value corresponding to the j-th segmentation configuration.

[0131] In the embodiment of the present invention, a weight value corresponding to each performance value can be set according to user needs. The electronic device performs weighted calculation on each performance value according to the set weight value corresponding to each performance value to generate a comprehensive performance value corresponding to the jth segmentation configuration.

[0132] The electronic device performs weighted calculation based on the calculation time and the weight value corresponding to the calculation time, as well as the bandwidth data and the weight value corresponding to the bandwidth data, to generate a comprehensive performance value corresponding to the jth slicing configuration. For example, when the user requires a slicing configuration with long calculation time and low bandwidth, the weight value of the calculation time is set to 0.5 and the weight value of the bandwidth data is set to 0.1. When j is equal to 2, the calculation time of the second slicing configuration is 10 seconds and the bandwidth data is 2 megabits per second. In this case, the comprehensive performance value corresponding to the second slicing configuration is 0.5*10+0.1*2=5.2. For another example, when the user requires a slicing configuration with short calculation time and high bandwidth, the weight value of the calculation time is set to 0.3 and the weight value of the bandwidth data is set to 0.6. When j is equal to 2, the calculation time of the second slicing configuration is 10 seconds and the bandwidth data is 2 megabits per second. In this case, the comprehensive performance value corresponding to the second slicing configuration is 0.3*10+0.6*2=4.2.

[0133] Figure 7 This is a flow chart of another method for obtaining comprehensive performance values provided in an embodiment of the present invention. Figure 7 As shown, as another optional solution, step S11 may specifically include:

[0134] Step S115 : Compile the operator according to the i-th set tensor shape and the j-th split configuration corresponding to the i-th set tensor shape to generate a second data file.

[0135] The electronic device compiles the parameters of the operator according to the i-th set tensor shape and the j-th split configuration corresponding to the i-th set tensor shape to generate an operator parameter structure; the electronic device serializes the operator parameter structure to generate a second data file, wherein the type of the second data file is a binary file; the second data file includes but is not limited to binary weight data.

[0136] Step S116: Execute the second data file to obtain a plurality of performance values.

[0137] The multiple performance values include but are not limited to computing time and bandwidth data. When the electronic device is a mobile phone or a wearable device, the electronic device saves the second data file locally, and the processor of the electronic device runs the second data file, and the electronic device obtains the multiple performance values through the operation log.

[0138] Step S117: Perform weighted calculation on the multiple performance values to generate a comprehensive performance value corresponding to the j-th segmentation configuration.

[0139] The electronic device performs weighted calculation on each performance value according to the weight value corresponding to each performance value set, and generates a comprehensive performance value corresponding to the j-th split configuration. The electronic device performs weighted calculation based on the calculation time and the weight value corresponding to the calculation time, as well as the bandwidth data and the weight value corresponding to the bandwidth data, to generate a comprehensive performance value corresponding to the j-th split configuration. For example, when the user requires a split configuration with long calculation time and low bandwidth, the weight value of the calculation time is set to 0.5 and the weight value of the bandwidth data is set to 0.1. When j is equal to 2, the calculation time of the second split configuration is 10 seconds and the bandwidth data is 2 megabits / second. At this time, the comprehensive performance value corresponding to the second split configuration is 0.5*10+0.1*2=5.2. For another example, when the user requires a sharding configuration with short computing time and high bandwidth, the weight value of the computing time is set to 0.3 and the weight value of the bandwidth data is set to 0.6. When j is equal to 2, the computing time of the second sharding configuration is 10 seconds and the bandwidth data is 2 megabits per second. At this time, the comprehensive performance value corresponding to the second sharding configuration is 0.3*10+0.6*2=4.2.

[0140] It should be noted that for the same operator, the weight value corresponding to each performance value is fixed.

[0141] As an optional scheme, when i is equal to 1 and j is equal to 1, the serial number of the first set tensor shape is 1 and the serial number of the first split configuration corresponding to the first set tensor shape is 1. At this time, a lookup table is generated according to the first set tensor shape and the first split configuration corresponding to the first set tensor shape. At this time, the lookup table saves the first set tensor shape and the first split configuration corresponding to the first set tensor shape.

[0142] In an embodiment of the present invention, when the electronic device performs operator compilation, if there are multiple types of operators, the electronic device may generate a lookup table corresponding to each type of operator. The electronic device may update the lookup tables corresponding to different types of operators according to step S10.

[0143] In the technical solution of the operator compilation method provided by the embodiment of the present invention, the electronic device updates the lookup table corresponding to the operator according to each set tensor shape and the optimal splitting configuration corresponding to each set tensor shape generated, obtains the lookup table corresponding to the operator according to the type of the operator, and queries the optimal splitting configuration corresponding to the tensor shape from the lookup table according to the tensor shape of the operator, wherein the lookup table includes the tensor shape and the optimal splitting configuration corresponding to the tensor shape, and the electronic device compiles the operator according to the tensor shape and the optimal splitting configuration corresponding to the tensor shape to generate a first data file. Before the operator is compiled, when the lookup table is updated, the electronic device performs data-independent splitting on the tensor shape of the operator, and the tensor shape after splitting can participate in the operation independently, and the optimal splitting configuration is determined; in the process of operator compilation, the optimal splitting configuration is obtained by obtaining the lookup table corresponding to the operator and the operator is compiled according to the optimal splitting configuration, thereby improving flexibility, improving computing unit utilization, reducing bandwidth, and efficient concurrent operations to speed up the operation speed. When compiling an operator, the operator is compiled by obtaining a lookup table, which automatically adapts to changes in the backend hardware. When the backend hardware changes, it will not affect the operator compilation process.

[0144] Figure 8 A schematic diagram of the structure of an operator compilation device provided by an embodiment of the present invention is shown in FIG. Figure 8 As shown, the device includes: an acquisition module 11, a query module 12 and a generation module 13.

[0145] The acquisition module 11 is connected to the query module 12 , and the query module 12 is connected to the generation module 13 .

[0146] The acquisition module 11 is used to obtain the lookup table corresponding to the operator according to the type of the operator; the query module 12 is used to query the optimal splitting configuration corresponding to the tensor shape from the lookup table according to the tensor shape of the operator, wherein the lookup table includes the tensor shape and the optimal splitting configuration corresponding to the tensor shape; the generation module 13 is used to compile the operator according to the tensor shape and the optimal splitting configuration corresponding to the tensor shape to generate a first data file.

[0147] In the embodiment of the present invention, the device further includes: an updating module 14 .

[0148] The updating module 14 is used to update the lookup table corresponding to the operator according to each set tensor shape and the generated optimal segmentation configuration corresponding to each set tensor shape.

[0149] In the embodiment of the present invention, the updating module 14 includes: a generating submodule 141 , an updating submodule 142 , a judging submodule 143 , and a setting submodule 144 .

[0150] The generation submodule 141 is used to generate the optimal splitting configuration corresponding to the i-th set tensor shape based on the i-th set tensor shape; the update submodule 142 is used to update the lookup table corresponding to the operator based on the i-th set tensor shape and the optimal splitting configuration corresponding to the i-th set tensor shape; the judgment submodule 143 is used to judge whether the serial number of the i-th set tensor shape is less than or equal to the total number of multiple set tensor shapes; the setting submodule 144 is used to use the i+1-th set tensor shape as the i-th set tensor shape if the judgment submodule 143 judges that the serial number of the i-th set tensor shape is less than the total number of multiple set tensor shapes, and trigger the generation submodule 141 to execute the step of generating the optimal splitting configuration corresponding to the i-th set tensor shape based on the i-th set tensor shape, where i is a positive integer.

[0151] In an embodiment of the present invention, the generation submodule 141 is specifically used to generate multiple segmentation configurations corresponding to the i-th set tensor shape based on the i-th set tensor shape; determine the optimal segmentation configuration corresponding to the i-th set tensor shape based on the i-th set tensor shape and the j-th segmentation configuration corresponding to the i-th set tensor shape, where j is a positive integer.

[0152] In an embodiment of the present invention, the generation module 13 is further configured to generate a plurality of set tensor shapes on multiple dimensions of the operator tensor according to a set upper limit of each dimension and a set dimensional step corresponding to each dimension.

[0153] In an embodiment of the present invention, the generation submodule 141 is specifically used to split the i-th set tensor shape at different dimensions of the i-th set tensor shape according to the dimension splitting thresholds corresponding to the set different dimensions, and generate multiple splitting configurations corresponding to the i-th set tensor shape.

[0154] In an embodiment of the present invention, the generation submodule 141 is specifically used to determine the optimal splitting configuration based on the i-th set tensor shape and the j-th splitting configuration corresponding to the i-th set tensor shape; judge whether the serial number of the j-th splitting configuration is less than or equal to the total number of multiple splitting configurations corresponding to the i-th set tensor shape; if it is judged that the serial number of the j-th splitting configuration is less than the total number of multiple splitting configurations corresponding to the i-th set tensor shape, then take the j+1-th splitting configuration as the j-th splitting configuration, and execute the steps of determining the optimal splitting configuration based on the i-th set tensor shape and the j-th splitting configuration corresponding to the i-th set tensor shape.

[0155] In an embodiment of the present invention, the generation submodule 141 is specifically used to obtain the comprehensive performance value corresponding to the jth split configuration based on the i-th set tensor shape and the j-th split configuration corresponding to the i-th set tensor shape; determine whether the comprehensive performance value corresponding to the j-th split configuration is better than the comprehensive performance value corresponding to the j-1-th split configuration; if it is determined that the comprehensive performance value corresponding to the j-th split configuration is better than the comprehensive performance value corresponding to the j-1-th split configuration, then the j-th split configuration is set as the optimal split configuration; if it is determined that the comprehensive performance value corresponding to the j-th split configuration is not better than the comprehensive performance value corresponding to the j-1-th split configuration, then the j-1-th split configuration is used as the optimal split configuration.

[0156] In an embodiment of the present invention, the generation submodule 141 is specifically used to compile an operator based on the i-th set tensor shape and the j-th split configuration corresponding to the i-th set tensor shape to generate a second data file; send the second data file to the mobile terminal; receive multiple performance values obtained and sent by the mobile terminal when running the second data file; perform weighted calculation on the multiple performance values to generate a comprehensive performance value corresponding to the j-th split configuration.

[0157] In an embodiment of the present invention, the generation submodule 141 is specifically used to compile an operator based on the i-th set tensor shape and the j-th split configuration corresponding to the i-th set tensor shape to generate a second data file; run the second data file to obtain multiple performance values; perform weighted calculation on the multiple performance values to generate a comprehensive performance value corresponding to the j-th split configuration.

[0158] In an embodiment of the present invention, the generation submodule 141 is specifically used to trigger the update submodule 142 to execute the step of updating the lookup table corresponding to the operator according to the i-th set tensor shape and the optimal splitting configuration corresponding to the i-th set tensor shape if it is determined that the serial number of the j-th splitting configuration is equal to the total number of multiple splitting configurations corresponding to the i-th set tensor shape.

[0159] In the technical solution of the operator compilation device provided by an embodiment of the present invention, the electronic device obtains a lookup table corresponding to the operator according to the type of the operator, and queries the optimal splitting configuration corresponding to the tensor shape from the lookup table according to the tensor shape of the operator, wherein the lookup table includes the tensor shape and the optimal splitting configuration corresponding to the tensor shape. The electronic device compiles the operator according to the tensor shape and the optimal splitting configuration corresponding to the tensor shape to generate a first data file. During the operator compilation process, the optimal splitting configuration is obtained by obtaining the lookup table corresponding to the operator and the operator is compiled according to the optimal splitting configuration, thereby improving flexibility, improving computing unit utilization, reducing bandwidth and accelerating computing speed when implementing operator compilation.

[0160] Figure 9A schematic diagram of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 9 As shown, the electronic device 21 includes: a processor 211, a memory 212, and a computer program 213 stored in the memory 212 and executable on the processor 211. When the computer program 213 is executed by the processor 211, the operator compilation method in the embodiment is implemented. To avoid repetition, they are not described here one by one.

[0161] The electronic device 21 includes, but is not limited to, a processor 211 and a memory 212. Those skilled in the art will understand that Figure 9 It is only an example of the electronic device 21 and does not constitute a limitation of the electronic device 21. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0162] The processor 211 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0163] The memory 212 can be an internal storage unit of the electronic device 21, such as a hard disk or memory of the electronic device 21. The memory 212 can also be an external storage device of the electronic device 21, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash memory card, etc. equipped on the electronic device 21. Furthermore, the memory 212 can also include both an internal storage unit of the electronic device 21 and an external storage device. The memory 212 is used to store computer programs and other programs and data required by the network device. The memory 212 can also be used to temporarily store data that has been output or is about to be output.

[0164] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0165] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An operator compilation method, characterized in that: The method comprises: Obtaining a lookup table corresponding to the operator according to the type of the operator; According to the tensor shape of the operator, querying the optimal segmentation configuration corresponding to the tensor shape from the lookup table, wherein the lookup table includes the tensor shape and the optimal segmentation configuration corresponding to the tensor shape; The operator is compiled according to the tensor shape and the optimal split configuration corresponding to the tensor shape to generate a first data file.

2. The method according to claim 1, characterized in that The operator corresponds to a plurality of set tensor shapes; before obtaining the lookup table corresponding to the operator according to the type of the operator, the method further includes: The lookup table corresponding to the operator is updated according to each set tensor shape and the generated optimal segmentation configuration corresponding to each set tensor shape.

3. The method according to claim 2, characterized in that The updating of the lookup table corresponding to the operator according to each set tensor shape and the generated optimal segmentation configuration corresponding to each set tensor shape includes: According to the i-th set tensor shape, generate the optimal segmentation configuration corresponding to the i-th set tensor shape; Updating the lookup table corresponding to the operator according to the i-th set tensor shape and the optimal segmentation configuration corresponding to the i-th set tensor shape; Determining whether the sequence number of the i-th set tensor shape is less than or equal to the total number of the plurality of set tensor shapes; If it is determined that the serial number of the i-th set tensor shape is less than the total number of multiple set tensor shapes, the i+1-th set tensor shape is used as the i-th set tensor shape, and the step of generating the optimal segmentation configuration corresponding to the i-th set tensor shape based on the i-th set tensor shape is executed, where i is a positive integer.

4. The method according to claim 3, characterized in that The step of generating an optimal segmentation configuration corresponding to the i-th set tensor shape according to the i-th set tensor shape includes: Generating a plurality of segmentation configurations corresponding to the i-th set tensor shape according to the i-th set tensor shape; According to the i-th set tensor shape and the j-th segmentation configuration corresponding to the i-th set tensor shape, an optimal segmentation configuration corresponding to the i-th set tensor shape is determined, where j is a positive integer.

5. The method according to any one of claims 2 to 4, characterized in that: Before updating the lookup table corresponding to the operator according to each set tensor shape and the generated optimal segmentation configuration corresponding to each set tensor shape, the method further includes: On multiple dimensions of the tensor of the operator, multiple set tensor shapes are generated according to the set upper limit of each dimension and the set dimension step corresponding to each dimension.

6. The method according to claim 4, characterized in that Generating a plurality of segmentation configurations corresponding to the i-th set tensor shape according to the i-th set tensor shape includes: On different dimensions of the i-th set tensor shape, the i-th set tensor shape is segmented according to the set dimension segmentation thresholds corresponding to the different dimensions, and multiple segmentation configurations corresponding to the i-th set tensor shape are generated.

7. The method according to claim 4, characterized in that The determining, based on the i-th set tensor shape and the j-th segmentation configuration corresponding to the i-th set tensor shape, an optimal segmentation configuration corresponding to the i-th set tensor shape includes: Determining an optimal split configuration according to an i-th set tensor shape and a j-th split configuration corresponding to the i-th set tensor shape; Determine whether the sequence number of the j-th split configuration is less than or equal to the total number of the multiple split configurations corresponding to the i-th set tensor shape; If it is determined that the serial number of the j-th split configuration is less than the total number of multiple split configurations corresponding to the i-th set tensor shape, the j+1-th split configuration is used as the j-th split configuration, and the step of determining the optimal split configuration based on the i-th set tensor shape and the j-th split configuration corresponding to the i-th set tensor shape is executed.

8. The method according to claim 7, characterized in that The determining of the optimal segmentation configuration according to the i-th set tensor shape and the j-th segmentation configuration corresponding to the i-th set tensor shape includes: According to the i-th set tensor shape and the j-th slicing configuration corresponding to the i-th set tensor shape, obtaining a comprehensive performance value corresponding to the j-th slicing configuration; Determine whether the comprehensive performance value corresponding to the j-th slicing configuration is better than the comprehensive performance value corresponding to the j-1-th slicing configuration; If it is determined that the comprehensive performance value corresponding to the j-th slicing configuration is better than the comprehensive performance value corresponding to the j-1-th slicing configuration, the j-th slicing configuration is set as the optimal slicing configuration; if it is determined that the comprehensive performance value corresponding to the j-th slicing configuration is not better than the comprehensive performance value corresponding to the j-1-th slicing configuration, the j-1-th slicing configuration is set as the optimal slicing configuration.

9. The method according to claim 8, characterized in that The obtaining, according to the i-th set tensor shape and the j-th slicing configuration corresponding to the i-th set tensor shape, a comprehensive performance value corresponding to the j-th slicing configuration includes: Compiling the operator according to the i-th set tensor shape and the j-th slicing configuration corresponding to the i-th set tensor shape to generate a second data file; sending the second data file to the mobile terminal; receiving a plurality of performance values obtained and sent by the mobile terminal when the mobile terminal runs the second data file; A weighted calculation is performed on the multiple performance values to generate a comprehensive performance value corresponding to the j-th segmentation configuration.

10. The method according to claim 8, characterized in that The obtaining, according to the i-th set tensor shape and the j-th slicing configuration corresponding to the i-th set tensor shape, a comprehensive performance value corresponding to the j-th slicing configuration includes: Compiling the operator according to the i-th set tensor shape and the j-th slicing configuration corresponding to the i-th set tensor shape to generate a second data file; running the second data file to obtain a plurality of performance values; A weighted calculation is performed on the multiple performance values to generate a comprehensive performance value corresponding to the j-th segmentation configuration.

11. The method according to claim 8, characterized in that Also includes: If it is determined that the serial number of the j-th segmentation configuration is equal to the total number of multiple segmentation configurations corresponding to the i-th set tensor shape, the step of updating the lookup table corresponding to the operator according to the i-th set tensor shape and the optimal segmentation configuration corresponding to the i-th set tensor shape is executed.

12. An operator compilation device, characterized in that: The device comprises: An acquisition module, configured to acquire a lookup table corresponding to an operator according to the type of the operator; A query module, configured to query the lookup table for an optimal segmentation configuration corresponding to the tensor shape according to the tensor shape of the operator, wherein the lookup table includes the tensor shape and the optimal segmentation configuration corresponding to the tensor shape; A generation module is used to compile the operator according to the tensor shape and the optimal segmentation configuration corresponding to the tensor shape to generate a first data file.

13. The device according to claim 12, characterized in that Also includes: Update module; The updating module is used to update the lookup table corresponding to the operator according to each set tensor shape and the generated optimal segmentation configuration corresponding to each set tensor shape.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the operator compilation method according to any one of claims 1 to 11.

15. An electronic device, characterized in that: include: one or more processors; Memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to perform the operator compilation method according to any one of claims 1 to 11.

Images