Operator combination determination method and determination device

Abstract

The invention discloses an operator combination determination method and determination device. The method comprises the following steps: acquiring an operator to be replaced; inputting first input data into the operator to be replaced to obtain first output data; a target operator combination is determined according to the first input data, the first output data and an operator set, the operator set comprises a plurality of alternative operators, the target operator combination comprises at least one alternative operator, and under the condition that the same data is input, the target operator combination comprises at least one alternative operator. The output of the to-be-replaced operator and the target operator combination meets a similar relationship.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to an operator combination determination method, an operator combination determination method and a determination device. Background Art

[0002] With the widespread application of deep learning and high-performance computing technologies, hardware accelerators such as GPU (Graphics Processing Unit), TPU (Tensor Processing Unit) and NPU (Neural Network Processing Unit) have become the key to improving model training and reasoning efficiency. However, hardware accelerators from different manufacturers are optimized for specific operators during design, resulting in different sets of supported operators, which brings challenges to model deployment. Summary of the invention

[0003] The present application provides a method for determining an operator combination, including:

[0004] Get the operator to be replaced;

[0005] Inputting the first input data into the operator to be replaced to obtain first output data;

[0006] A target operator combination is determined according to the first input data, the first output data and an operator set, the operator set includes a plurality of candidate operators, the target operator combination includes at least one of the candidate operators, and when the same data is input, the output of the operator to be replaced and the target operator combination satisfy a similarity relationship.

[0007] Optionally, the operation of the operator to be replaced cannot be supported by the target operation module, and the operation of the candidate operator can be supported by the target operation module.

[0008] Optionally, determining a target operator combination according to the first input data, the first output data, and an operator set includes:

[0009] Inputting the first input data and the first output data into a feature extraction model to obtain first feature information output by the feature extraction model;

[0010] The target operator combination is determined according to the first feature information and the operator set.

[0011] Optionally, determining the target operator combination according to the first feature information and the operator set includes:

[0012] Inputting the first feature information into an operator combination calculation model to obtain target operator combination information output by the operator combination calculation model, wherein the operator combination calculation model is obtained by training an initial model with target sample data, and the target sample data includes a plurality of sample operator combination information and sample feature information corresponding to each of the sample operator combination information;

[0013] The target operator combination is determined according to the target operator combination information and the operator set.

[0014] Optionally, determining the target operator combination according to the first feature information and the operator set includes:

[0015] Determining a plurality of candidate operator combinations from the operator set;

[0016] Inputting the first input data into a plurality of the candidate operator combinations to obtain candidate output data corresponding to each of the candidate operator combinations;

[0017] Inputting each of the candidate output data and the first input data into the feature extraction model respectively, to obtain second feature information corresponding to each of the candidate operator combinations output by the feature extraction model;

[0018] The target operator combination is determined according to a matching relationship between each of the second feature information and the first feature information and a plurality of the candidate operator combinations.

[0019] Optionally, determining a target operator combination according to the first input data, the first output data, and an operator set includes:

[0020] Determining a plurality of candidate operator combinations from the operator set;

[0021] Inputting the first input data into a plurality of the candidate operator combinations to obtain candidate output data corresponding to each of the candidate operator combinations;

[0022] The target operator combination is determined according to a matching relationship between each candidate output data and the first output data and a plurality of candidate operator combinations.

[0023] Optionally, determining a target operator combination according to the first input data, the first output data, and an operator set includes:

[0024] Determine a plurality of matching operator combinations according to the first input data, the first output data, and the operator set;

[0025] Obtaining operation step information corresponding to each matching operator combination;

[0026] Performing a screening operation on a plurality of matching operator combinations according to the operation step information;

[0027] The target operator combination is determined according to the screening result corresponding to the screening operation.

[0028] Optionally, determining a target operator combination according to the first input data, the first output data, and an operator set includes:

[0029] Determine a preliminary operator combination according to the first input data, the first output data and the operator set;

[0030] Inputting the first input data into the primary selection operator combination to obtain primary selection output data;

[0031] performing an adjustment operation on the preliminary selection operator combination according to difference information between the first output data and the preliminary selection output data;

[0032] The target operator combination is obtained according to the adjustment result corresponding to the adjustment operation.

[0033] Optionally, determining a target operator combination according to the first input data, the first output data, and an operator set includes:

[0034] Determining an operator combination to be verified according to the first input data, the first output data and an operator set;

[0035] Inputting the second input data into the operator to be replaced to obtain second output data;

[0036] Inputting the second input data into the operator combination to be verified to obtain output data to be verified;

[0037] When the second output data and the output data to be verified satisfy a verification condition, the operator combination to be verified is determined as the target operator combination.

[0038] The embodiment of the present application also provides an operator combination determination device, including:

[0039] An acquisition module configured to acquire an operator to be replaced, wherein the operation of the operator to be replaced cannot be supported by the target operation module;

[0040] An input module configured to input first input data into the operator to be replaced to obtain first output data;

[0041] A determination module is configured to determine a target operator combination based on the first input data, the first output data and an operator set, wherein the operator set includes multiple candidate operators, the target operator combination includes at least one of the candidate operators, the operation of the candidate operator can be supported by the target operation module, and when the same data is input, the output of the operator to be replaced and the target operator combination satisfy a similarity relationship.

[0042] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores an executable program, and the processor executes the executable program to perform the steps of the method described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of a method for determining an operator combination according to an embodiment of the present application;

[0044] Figure 2 For the embodiments of this application Figure 1 Flow chart of a first embodiment of step S300;

[0045] Figure 3 For the embodiments of this application Figure 2 Flow chart of a first embodiment of step S320;

[0046] Figure 4 For the embodiments of this application Figure 2 Flow chart of the second embodiment of step S320;

[0047] Figure 5 For the embodiments of this application Figure 1 Flow chart of a second embodiment of step S300;

[0048] Figure 6 For the embodiments of this application Figure 1 Flow chart of a third embodiment of step S300;

[0049] Figure 7 For the embodiments of this application Figure 1 Flow chart of the fourth embodiment of step S300;

[0050] Figure 8 For the embodiments of this application Figure 1 Flow chart of the fifth embodiment of step S300;

[0051] Fig. 9 This is a structural block diagram of an operator combination determination device according to an embodiment of the present application. DETAILED DESCRIPTION

[0052] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0053] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0054] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0055] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0056] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.

[0057] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0058] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0059] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0060] In the related art, when some computing models are deployed to specific target computing modules, the following problems are often encountered: the target computing module does not support certain operators in the computing model; even if the computing module supports a certain operator, its performance is not satisfactory. In order to solve these problems, engineers usually use replacement operators with similar functions to replace unsupported operators. For example, if a certain activation function cannot be supported by the target computing module, an alternative function with similar performance can be selected to adapt to the characteristics of the target computing module. However, when encountering custom operators or there are no suitable alternative operators, the above method is no longer applicable. Custom operators are usually tailored for specific applications, and it is difficult to find alternative solutions with exactly the same functions.

[0061] It should be noted that the target computing module mentioned in this application refers to a specific functional unit implemented in software or hardware, which is responsible for performing a certain type of computing task. The target computing module can be a function or algorithm implementation in a software library, or a hardware accelerator such as an ASIC (Application-Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU, TPU or NPU.

[0062] Therefore, how to more efficiently solve the operator mismatch problem encountered in cross-platform model migration, especially for those custom operators that are difficult to replace simply, has become an important obstacle to the widespread application of deep learning and high-performance computing technologies. It is necessary to explore new methods to better cope with this challenge and promote the further development of technologies such as deep learning and high-performance computing.

[0063] An operator combination determination method of an embodiment of the present application can be used to find the optimal or approximately optimal operator combination on a given target computing module or hardware platform to complete a specific computing task. For example, by adopting the operator combination determination method provided by the present application, a suitable operator combination can be obtained, which can significantly improve computing performance, reduce energy consumption, improve resource utilization, and simplify the development process in deep learning; improve computing efficiency and performance in scientific computing and engineering simulation; and run complex algorithms on low-power, low-cost hardware in embedded systems.

[0064] The operator combination determination method of the present application is described in detail below with reference to the accompanying drawings. Figure 1 Flow chart of the operator combination determination method of the embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0065] S100: Obtain an operator to be replaced.

[0066] In the embodiments of the present application, operators can be understood as functional units that perform basic operations. Specifically, in deep learning, operators can represent a step or layer in a neural network, such as convolution, pooling, full connection, and activation functions. In high-performance computing, operators can be highly parallelized, computationally intensive tasks, such as linear algebra operations, Fourier transforms, and physical simulations. In embedded systems, common operators include signal processing, image processing, sensor data processing, control algorithms, and lightweight machine learning models, which are usually implemented using specialized libraries (such as CMSIS-DSP) and optimization techniques (such as quantization).

[0067] The operators to be replaced can be custom operators or operators without suitable alternatives. These operators cannot be supported by the target operation module, and engineers cannot replace them by using alternative operators with similar functions. Specifically, when deploying deep learning models in hardware platforms, it is often encountered that operators are not supported or poorly supported. This situation will cause the deep learning model to be unable to run directly or to run inefficiently. For example, when deploying a convolutional neural network on an embedded device, the embedded device does not support the activation function Swish. For another example, when a deep learning model containing some custom operators is deployed on a GPU, the GPU does not support the custom operators.

[0068] In addition, the operators to be replaced can also be operators with low execution efficiency on a specific hardware architecture. Replacing these operators with operators with higher execution efficiency can improve the overall running speed of the program. For example, in embedded systems or dedicated accelerators such as GPUs and TPUs, the operators to be replaced can be replaced with operators optimized for these hardware.

[0069] Furthermore, in some edge computing devices or low-power devices with limited processing power and memory, the operators to be replaced may also be operators with more resource requirements. Replacing these operators with operators with fewer resource requirements can save resources and improve computing efficiency.

[0070] It is understandable that the operator to be replaced may also be an operator to be replaced based on compatibility, security, or debugging and testing needs.

[0071] S200 , inputting first input data into the operator to be replaced to obtain first output data.

[0072] It can be understood that, since an operator is a functional unit that performs basic operations, first output data can be obtained by inputting first input data into the operator to be replaced.

[0073] For example, the operator to be replaced is a user-defined matrix multiplication function, and the first input data may be matrix A and matrix B. By inputting these two matrices into the user-defined matrix multiplication function, a first output data matrix C can be obtained.

[0074] S300. Determine a target operator combination according to the first input data, the first output data, and an operator set, wherein the operator set includes multiple candidate operators, and the target operator combination includes at least one of the candidate operators. When the same data is input, the output of the operator to be replaced and the target operator combination satisfy a similarity relationship.

[0075] It should be noted that in the embodiments of the present application, the candidate operator refers to other implementation methods that can replace the operator to be replaced. These candidate operators may have different performance characteristics, and produce output results similar to the operator to be replaced by directly replacing them alone or combining them together and using the combined operator to replace them. The target operator combination can be understood as the operator combination finally determined. The similarity relationship satisfied by the output of the operator to be replaced and the target operator combination can be characterized by similarity, and specifically, the similarity can be measured by MSE (Mean Squared Error).

[0076] The operator combination finally determined includes more than one of the candidate operators.

[0077] When the finally determined operator combination includes one candidate operator, and each candidate operator in the operator set can directly replace the operator to be replaced individually:

[0078] The first input data is input into the candidate operator of the operator combination finally determined to obtain the output data corresponding to the candidate operator. The output of the target operator combination is the output data corresponding to the candidate operator. The similarity between the output of the operator to be replaced, i.e., the first output data, and the output data corresponding to the candidate operator needs to meet a preset similarity threshold.

[0079] When the operator combination finally determined includes two or more candidate operators, and each candidate operator in the operator set can directly replace the operator to be replaced individually:

[0080] The first input data is input into each of the candidate operators to obtain output data corresponding to each of the candidate operators. The output of the target operator combination is the output data corresponding to each of the candidate operators. The similarity between the output of the operator to be replaced, i.e., the first output data, and the output data corresponding to each of the candidate operators needs to meet a preset similarity threshold.

[0081] Among them, when the finally determined operator combination includes two or more candidate operators, when each candidate operator in the operator set cannot be replaced individually and the operator to be replaced needs to be replaced by the combined operator:

[0082] The first input data is input into the operators after the combination of the candidate operators to obtain the output data corresponding to the operators after the combination of the candidate operators. The output of the target operator combination is the output data corresponding to the operators after the combination of the candidate operators. The similarity between the output of the operator to be replaced, i.e., the first output data, and the output data corresponding to the operators after the combination of the candidate operators needs to meet a preset similarity threshold.

[0083] Exemplarily, the operator to be replaced is a custom matrix multiplication function, and the operator set includes multiple candidate operators, namely, the matrix multiplication function numpy.dot() of NumPy (Numerical Python, an open source numerical computing extension of Python), the matrix multiplication function tf.matmul() of TensorFlow (an open source machine learning framework developed and released by Google), and the matrix multiplication function np.matmul() of NumPy. These three candidate operators can individually replace the operator to be replaced.

[0084] When the target operator combination includes one candidate operator numpy.dot() among the three candidate operators, the output of the target operator combination is the output data corresponding to the candidate operator numpy.dot(). The similarity between the output data corresponding to the candidate operator numpy.dot() and the first output data needs to meet the similarity threshold.

[0085] When the target operator combination includes two of the three candidate operators, numpy.dot() and tf.matmul(), the output of the target operator combination is the output data corresponding to the candidate operator numpy.dot() and the output data corresponding to the candidate operator tf.matmul(). The similarity between the output data corresponding to the candidate operator numpy.dot() and the first output data and the similarity between the output data corresponding to the candidate operator tf.matmul() and the first output data both need to meet the similarity threshold.

[0086] Exemplarily, the operator to be replaced is a user-defined matrix multiplication function, and the multiple candidate operators included in the operator set are a multiplication operator and an addition operator. These two candidate operators cannot replace the operator to be replaced alone, and the operator to be replaced needs to be replaced by the combined operator. In this case, the output of the target operator combination is the output data corresponding to the operator after the combination of the multiple candidate operators, and the similarity between the output data and the first output data needs to meet the similarity threshold.

[0087] In this embodiment, the operation of the operator to be replaced cannot be supported by the target operation module, and the operation of the candidate operator can be supported by the target operation module. When the operation of the operator to be replaced cannot be supported by the target operation module, the operator combination determination method provided in the embodiment of the present application is adopted to determine the target operator combination according to the operator set including multiple candidate operators, and the target operator combination including the candidate operator supported by the target operation module is used to replace the operator to be replaced, so as to solve the problem that the operator to be replaced is not supported.

[0088] The operator combination determination method provided in the embodiment of the present application obtains first output data by inputting first input data into the operator to be replaced, and then determines the target operator combination according to the first input data, the first output data and an operator set including multiple candidate operators. It can dynamically use a combination of one or more candidate operators to replace the operator to be replaced according to actual application requirements, which can not only improve the performance and flexibility of the target operation module, but also enhance the robustness and resource utilization of the target operation module.

[0089] In one embodiment, in the above step S300, a target operator combination is determined according to the first input data, the first output data and the operator set, such as Figure 2 As shown, specifically including:

[0090] S310: Input the first input data and the first output data into a feature extraction model to obtain first feature information.

[0091] Among them, the feature extraction model is used to extract the first feature information from the first input data and the first output data. The first feature information can be a vector or a matrix, which is used to represent the attribute information of the operator to be replaced. The specific content of the first feature information varies depending on the actual application requirements, depending on the specific processing task, data type and the operator used. For example, in an image processing task, according to the actual application requirements, the first feature information can be a convolutional neural network feature, a color feature, a texture feature, a shape feature, etc.

[0092] S320: Determine the target operator combination according to the first feature information and the operator set.

[0093] For example, in an image processing task, a convolution operation is performed on an image, and a pre-trained convolutional neural network VGG16 is used as a feature extraction model. The operator to be replaced is a custom convolution layer, and the first input data uses a first input image, and the first input image is input into the custom convolution layer to obtain a first output image.

[0094] The first input image and the first output image are input into the convolutional neural network VGG16 to obtain the first feature information including the attribute features of the custom convolutional layer. The operator set includes three alternative operators: standard convolutional layer Conv2D, depthwise separable convolutional layer DepthwiseConv2D, and separable convolutional layer SeparableConv2D.

[0095] The target operator combination is obtained according to the inclusion relationship between the first feature information and the operator set. Specifically, when the three candidate operators in the operator set all contain the first feature information, the three candidate operators can be used as the target operator combination.

[0096] By flexibly determining the best operator combination, consistent performance can be obtained on different target operation modules, thereby improving the computational efficiency of the target operation modules.

[0097] In this embodiment, in the above step S320, the target operator combination is determined according to the first feature information and the operator set, such as Figure 3 As shown, it may specifically include:

[0098] S3201. Input the first feature information into an operator combination calculation model to obtain target operator combination information output by the operator combination calculation model, wherein the operator combination calculation model is obtained by training an initial operator combination calculation model with target sample data, and the target sample data includes multiple sample operator combination information and sample feature information corresponding to each sample operator combination information.

[0099] It should be noted that the operator combination calculation model is a machine learning model. For example, a fully connected neural network can be used as the operator combination calculation model. The target sample data is a data set used to train the operator combination calculation model. Each sample operator combination information can represent the configuration information of a specific operator combination, for example, different convolutional layer types. The sample feature information represents the features of the sample operator combination corresponding to the sample operator combination information.

[0100] S3202. Determine the target operator combination according to the target operator combination information and the operator set.

[0101] Exemplarily, target sample data is obtained, and the initial operator combination calculation model is trained using the target sample data to obtain a trained operator combination calculation model. The first feature information is input into the trained operator combination calculation model to obtain target operator combination information. And the target operator combination is obtained according to the matching features between the target operator combination information and multiple candidate operators in the operator set. For example, MLP (Multilayer Perceptron) can be used as the operator combination calculation model.

[0102] The embodiment of the present application directly uses the first feature information, obtains the target operator combination information by inputting the first feature information into the pre-trained operator combination calculation model, and determines the optimal target operator combination based on the target operator combination information and the operator set. This can not only improve the performance and flexibility of the model where the target operator combination is located, but also enhance the robustness and resource utilization of the target operation module.

[0103] In this embodiment, in the above step S320, the target operator combination is determined according to the first feature information and the operator set, such as Figure 4 As shown, it may also specifically include:

[0104] S3203: Determine multiple candidate operator combinations from the operator set.

[0105] S3204: Input the first input data into a plurality of candidate operator combinations to obtain candidate output data corresponding to each candidate operator combination.

[0106] S3205. Input each candidate output data and the first input data into the feature extraction model respectively to obtain second feature information corresponding to each candidate operator combination output by the feature extraction model.

[0107] S3206: Determine the target operator combination according to a matching relationship between each of the second feature information and the first feature information and a plurality of the candidate operator combinations.

[0108] The embodiment of the present application indirectly uses the first feature information by inputting the first input data into a plurality of candidate operator combinations determined from the operator set to obtain the alternative output data corresponding to each of the candidate operator combinations; then the alternative output data and the first input data are input into the feature extraction model together to obtain the second feature information, wherein the second feature information includes the attribute characteristics of the candidate operator combination; and then the final target operator combination is obtained based on the matching relationship between the first feature information and the second feature information and combined with the plurality of candidate operator combinations.

[0109] The matching relationship here can be understood as the similarity between the first feature information and the second feature information, which can be specifically measured by the Jaccard similarity coefficient. The larger the Jaccard coefficient value, the higher the similarity between the first feature information and the second feature information. The final target operator combination can be obtained based on the second feature information with the highest similarity to the first feature information and multiple candidate operator combinations. By selecting and determining the optimal operator combination, hardware resources can be used more efficiently, reducing energy consumption and delay.

[0110] In another embodiment, in the above step S300, a target operator combination is determined according to the first input data, the first output data and the operator set, such as Figure 5 As shown, it may also include:

[0111] S330: Determine a plurality of candidate operator combinations from the operator set.

[0112] S340: Input the first input data into a plurality of candidate operator combinations to obtain candidate output data corresponding to each candidate operator combination.

[0113] S350: Determine the target operator combination according to a matching relationship between each candidate output data and the first output data and a plurality of candidate operator combinations.

[0114] It should be noted that the matching relationship here can be understood as the degree of proximity between the candidate output data and the first output data, which can be specifically measured by Euclidean distance. The smaller the Euclidean distance, the higher the degree of proximity between the candidate output data and the first output data.

[0115] The embodiment of the present application utilizes the degree of proximity between the first output data obtained by inputting the first input data into the operator to be replaced and the alternative output data obtained by inputting the first input data into the alternative operator combination, and can determine the alternative operator combination corresponding to the alternative output data closest to the first output data from multiple alternative operator combinations, and use the alternative operator combination as the final target operator combination.

[0116] In another embodiment, in the above step S300, a target operator combination is determined according to the first input data, the first output data and the operator set, such as Figure 6 As shown, it may also include:

[0117] S360: Determine a plurality of matching operator combinations according to the first input data, the first output data, and the operator set.

[0118] In an embodiment of the present application, the first input data is input into each alternative operator combination in the operator set to obtain alternative output data corresponding to each alternative operator combination. According to the degree of proximity between the alternative output data and the first output data, an enumeration method or a random search algorithm can be used to determine multiple matching operator combinations in the operator set.

[0119] S370: Obtain operation step information corresponding to each matching operator combination.

[0120] For each matching operator combination, its specific operation step information may be recorded, including the operator type used, parameter configuration, and execution order, etc.

[0121] S380, performing a screening operation on the plurality of matching operator combinations according to the operation step information. Specifically, the performance index of each matching operator combination can be obtained according to the operation step information. The performance index of the matching operator combination can be the accuracy of each matching operator combination. A threshold of accuracy, such as 0.85, is preset to screen out matching operator combinations with an accuracy greater than 0.85.

[0122] S390: Determine the target operator combination according to the screening result corresponding to the screening operation.

[0123] Based on the screening results corresponding to the screening operation, the final target operator combination can be determined in combination with factors such as computing time or consumption of hardware resources.

[0124] The embodiment of the present application uses a cross-validation algorithm to perform screening operations on each of the matching operator combinations, and finally screens out the best operator combination, so that the screened target operator combination can maintain good performance in a variety of scenarios, thereby enhancing the robustness and stability of the target operator combination.

[0125] In another embodiment, in the above step S300, a target operator combination is determined according to the first input data, the first output data and the operator set, such as Figure 7 As shown, it may also include:

[0126] S3100. Determine a preliminary operator combination according to the first input data, the first output data, and the operator set.

[0127] According to the first input data and the first output data, a preliminary operator combination having a high similarity to the operator to be replaced can be determined in the operator set.

[0128] S3110. Input the first input data into the preliminary selection operator combination to obtain preliminary selection output data.

[0129] S3120: Adjust the preliminary selected operator combination according to the difference information between the first output data and the preliminary selected output data.

[0130] The loss function may be used to calculate the MSE between the first output data and the preliminary output data, and then the operator type, parameter settings, etc. of the preliminary operator combination may be adjusted according to the MSE of the two.

[0131] S3130. Obtain the target operator combination according to the adjustment result corresponding to the adjustment operation.

[0132] Repeat steps S3110 and S3120 until a preset stop condition is met, for example, the maximum number of iterations is reached, the performance of the preliminary operator combination is no longer improved, etc. The determined operator combination is the target operator combination.

[0133] The embodiment of the present application is based on a feedback optimization strategy, which minimizes the difference between the preliminary output data and the first output data by continuously adjusting the preliminary operator combination, and ultimately obtains the optimal operator combination.

[0134] In another embodiment, in the above step S300, a target operator combination is determined according to the first input data, the first output data and the operator set, such as Figure 8 As shown, it may also include:

[0135] S3140. Determine an operator combination to be verified according to the first input data, the first output data, and an operator set.

[0136] According to the first input data and the first output data, a candidate operator or a candidate operator combination may be selected from an operator set as an operator combination to be verified.

[0137] S3150: Input the second input data into the operator to be replaced to obtain second output data.

[0138] S3160: Input the second input data into the operator combination to be verified to obtain output data to be verified.

[0139] S3170: When the second output data and the output data to be verified satisfy a verification condition, determine the operator combination to be verified as the target operator combination.

[0140] The verification condition may be implemented by comparing a threshold value. For example, the accuracy of the second output data is used as a threshold value. If the accuracy of the output data to be verified is greater than the threshold value, it means that the verification condition is satisfied.

[0141] Exemplarily, for a binary classification problem, the input data is a data point containing multiple features, and the output data is a binary classification label (0 or 1). The operator to be replaced is a combination of standardization and logistic regression. The operator set includes a standardization operation, a PCA (Principal Component Analysis) dimensionality reduction operation, a logistic regression operation, and a support vector machine. According to the first input data and the first output data, an operator combination to be verified is determined in the operator combination, and the operator combination to be verified is a combination of a standardization operation, a PCA dimensionality reduction operation, and a support vector machine.

[0142] The second input data is processed by the operator to be replaced to obtain the second output data. The second input data is processed by the operator combination to be verified to obtain the output data to be verified. The accuracy of the second output data is calculated according to the second output data and the real data, and the accuracy of the output data to be verified is calculated according to the output data to be verified and the real data. When the accuracy of the output data to be verified is greater than the accuracy of the second output data, the operator combination to be verified can be used as the final target operator combination.

[0143] The embodiment of the present application can systematically evaluate whether the operator combination to be verified is better than the operator combination to be replaced by selecting appropriate performance indicators, calculating these indicators, setting verification conditions, comparing performance indicators, and finally deciding whether to adopt the operator combination to be verified, so as to gradually optimize and obtain the final target operator combination. In practical applications, different performance indicators and verification conditions can be selected according to specific needs to adapt to different application scenarios.

[0144] The present application also provides an operator combination determination device, such as Fig. 9 As shown, including:

[0145] An acquisition module is configured to acquire the operator to be replaced.

[0146] In the embodiments of the present application, operators can be understood as functional units that perform basic operations. Specifically, in deep learning, operators represent a step or layer in a neural network, such as convolution, pooling, full connection, and activation functions. In high-performance computing, operators are usually highly parallelized, computationally intensive tasks, such as linear algebra operations, Fourier transforms, and physical simulations. In embedded systems, common operators include signal processing, image processing, sensor data processing, control algorithms, and lightweight machine learning models, which are usually implemented using specialized libraries (such as CMSIS-DSP) and optimization techniques (such as quantization).

[0147] Operators to be replaced can be custom operators or operators without suitable alternatives. These operators are not supported by the target operation module, and engineers cannot replace them by using alternative operators with similar functions. Specifically, when deploying deep learning models in hardware platforms, it is often encountered that operators are not supported or poorly supported. This situation will cause the deep learning model to be unable to run directly or to run inefficiently. For example, when deploying convolutional neural networks on embedded devices, the embedded devices do not support the activation function Swish. When a deep learning model containing some custom operators is deployed on a GPU, the GPU does not support the custom operators.

[0148] In addition, the operators to be replaced can also be operators with low execution efficiency on a specific hardware architecture. Replacing these operators with operators with higher execution efficiency can improve the overall running speed of the program. For example, in embedded systems or dedicated accelerators such as GPUs and TPUs, the operators to be replaced can be replaced with operators optimized for these hardware.

[0149] Furthermore, in some edge computing devices or low-power devices with limited processing power and memory, the operators to be replaced may also be operators with more resource requirements. Replacing these operators with operators with fewer resource requirements can save resources and improve computing efficiency.

[0150] It is understandable that the operator to be replaced may also be an operator to be replaced based on compatibility, security, or debugging and testing needs.

[0151] The input module is configured to input first input data into the operator to be replaced to obtain first output data.

[0152] It can be understood that, since an operator is a functional unit that performs basic operations, first output data can be obtained by inputting first input data into the operator to be replaced.

[0153] For example, the operator to be replaced is a user-defined matrix multiplication function, and the first input data may be matrix A and matrix B. By inputting these two matrices into the user-defined matrix multiplication function, a first output data matrix C can be obtained.

[0154] A determination module is configured to determine a target operator combination based on the first input data, the first output data and an operator set, wherein the operator set includes multiple candidate operators, and the target operator combination includes at least one of the candidate operators. When the same data is input, the output of the operator to be replaced and the target operator combination satisfy a similarity relationship.

[0155] It should be noted that in the embodiments of the present application, the candidate operator refers to other implementation methods that can replace the operator to be replaced. These candidate operators may have different performance characteristics, and produce output results similar to the operator to be replaced by directly replacing them alone or combining them together and using the combined operator to replace them. The target operator combination can be understood as the operator combination finally determined. The similarity relationship satisfied by the output of the operator to be replaced and the target operator combination can be characterized by similarity, and specifically, the similarity can be measured by MSE (Mean Squared Error).

[0156] The operator combination finally determined includes more than one of the candidate operators.

[0157] When the finally determined operator combination includes one candidate operator, and each candidate operator in the operator set can directly replace the operator to be replaced individually:

[0158] The first input data is input into the candidate operator of the operator combination finally determined to obtain the output data corresponding to the candidate operator. The output of the target operator combination is the output data corresponding to the candidate operator. The similarity between the output of the operator to be replaced, i.e., the first output data, and the output data corresponding to the candidate operator needs to meet a preset similarity threshold.

[0159] When the operator combination finally determined includes two or more candidate operators, and each candidate operator in the operator set can directly replace the operator to be replaced independently:

[0160] The first input data is input into each of the candidate operators to obtain output data corresponding to each of the candidate operators. The output of the target operator combination is the output data corresponding to each of the candidate operators. The similarity between the output of the operator to be replaced, i.e., the first output data, and the output data corresponding to each of the candidate operators needs to meet a preset similarity threshold.

[0161] When the operator combination finally determined includes two or more candidate operators, and each candidate operator in the operator set cannot be replaced individually and needs to be replaced by the combined operator:

[0162] The first input data is input into the operators after the combination of the candidate operators to obtain the output data corresponding to the operators after the combination of the candidate operators. The output of the target operator combination is the output data corresponding to the operators after the combination of the candidate operators. The similarity between the output of the operator to be replaced, i.e., the first output data, and the output data corresponding to the operators after the combination of the candidate operators needs to meet a preset similarity threshold.

[0163] Exemplarily, the operator to be replaced is a custom matrix multiplication function, and the operator set includes multiple candidate operators, namely, the matrix multiplication function numpy.dot() of NumPy (Numerical Python, an open source numerical computing extension of Python), the matrix multiplication function tf.matmul() of TensorFlow (an open source machine learning framework developed and released by Google), and the matrix multiplication function np.matmul() of NumPy. These three candidate operators can individually replace the operator to be replaced.

[0164] When the target operator combination includes one candidate operator numpy.dot() among the three candidate operators, the output of the target operator combination is the output data corresponding to the candidate operator numpy.dot(). The similarity between the output data corresponding to the candidate operator numpy.dot() and the first output data needs to meet the similarity threshold.

[0165] When the target operator combination includes two of the three candidate operators, numpy.dot() and tf.matmul(), the output of the target operator combination is the output data corresponding to the candidate operator numpy.dot() and the output data corresponding to the candidate operator tf.matmul(). The similarity between the output data corresponding to the candidate operator numpy.dot() and the first output data and the similarity between the output data corresponding to the candidate operator tf.matmul() and the first output data both need to meet the similarity threshold.

[0166] Exemplarily, the operator to be replaced is a user-defined matrix multiplication function, and the multiple candidate operators included in the operator set are a multiplication operator and an addition operator. These two candidate operators cannot replace the operator to be replaced alone, and the operator to be replaced needs to be replaced by the combined operator. In this case, the output of the target operator combination is the output data corresponding to the operator after the combination of the multiple candidate operators, and the similarity between the output data and the first output data needs to meet the similarity threshold.

[0167] The operator combination determination method provided in the embodiment of the present application obtains first output data by inputting first input data into the operator to be replaced, and then determines the target operator combination according to the first input data, the first output data and an operator set including multiple candidate operators. It can dynamically use a combination of one or more candidate operators to replace the operator to be replaced according to actual application requirements, which can not only improve the performance and flexibility of the target operation module, but also enhance the robustness and resource utilization of the target operation module.

[0168] In this embodiment, the operation of the operator to be replaced cannot be supported by the target operation module, and the operation of the candidate operator can be supported by the target operation module. When the operation of the operator to be replaced cannot be supported by the target operation module, the operator combination determination method provided in the embodiment of the present application is adopted to determine the target operator combination according to the operator set including multiple candidate operators, and the target operator combination including the candidate operator supported by the target operation module is used to replace the operator to be replaced, so as to solve the problem that the operator to be replaced is not supported.

[0169] Optionally, the determination module is further configured as:

[0170] The first input data and the first output data are input into a feature extraction model to obtain first feature information output by the feature extraction model.

[0171] The target operator combination is determined according to the first feature information and the operator set.

[0172] Optionally, the determination module is further configured as:

[0173] A plurality of candidate operator combinations are determined from the operator set.

[0174] The first input data is input into a plurality of the candidate operator combinations to obtain candidate output data corresponding to each of the candidate operator combinations.

[0175] The target operator combination is determined according to a matching relationship between each candidate output data and the first output data and a plurality of candidate operator combinations.

[0176] Optionally, the determination module is further configured as:

[0177] A plurality of matching operator combinations are determined according to the first input data, the first output data, and the operator set.

[0178] Obtain operation step information corresponding to each matching operator combination.

[0179] A screening operation is performed on the plurality of matching operator combinations according to the operation step information.

[0180] The target operator combination is determined according to the screening result corresponding to the screening operation.

[0181] Optionally, the determination module is further configured as:

[0182] A preliminary operator combination is determined according to the first input data, the first output data, and the operator set.

[0183] The first input data is input into the preliminary selection operator combination to obtain preliminary selection output data.

[0184] An adjustment operation is performed on the preliminary selection operator combination according to difference information between the first output data and the preliminary selection output data.

[0185] The target operator combination is obtained according to the adjustment result corresponding to the adjustment operation.

[0186] Optionally, the determination module is further configured as:

[0187] An operator combination to be verified is determined according to the first input data, the first output data, and an operator set.

[0188] The second input data is input into the operator to be replaced to obtain second output data.

[0189] The second input data is input into the operator combination to be verified to obtain output data to be verified.

[0190] When the second output data and the output data to be verified satisfy a verification condition, the operator combination to be verified is determined as the target operator combination.

[0191] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores an executable program, and the processor executes the executable program to perform the steps of the method described above.

[0192] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A method for determining an operator combination, comprising: Get the operator to be replaced; Inputting the first input data into the operator to be replaced to obtain first output data; A target operator combination is determined according to the first input data, the first output data and an operator set, the operator set includes a plurality of candidate operators, the target operator combination includes at least one of the candidate operators, and when the same data is input, the output of the operator to be replaced and the target operator combination satisfy a similarity relationship.

2. According to the method of claim 1, the operation of the operator to be replaced cannot be supported by the target operation module, and the operation of the alternative operator can be supported by the target operation module.

3. The method according to claim 1, wherein determining a target operator combination according to the first input data, the first output data and an operator set comprises: Inputting the first input data and the first output data into a feature extraction model to obtain first feature information output by the feature extraction model; The target operator combination is determined according to the first feature information and the operator set.

4. According to the method according to claim 3, determining the target operator combination according to the first feature information and the operator set comprises: Inputting the first feature information into an operator combination calculation model to obtain target operator combination information output by the operator combination calculation model, wherein the operator combination calculation model is obtained by training an initial model with target sample data, and the target sample data includes a plurality of sample operator combination information and sample feature information corresponding to each of the sample operator combination information; The target operator combination is determined according to the target operator combination information and the operator set.

5. The method according to claim 3, wherein determining the target operator combination according to the first feature information and the operator set comprises: Determining a plurality of candidate operator combinations from the operator set; Inputting the first input data into a plurality of the candidate operator combinations to obtain candidate output data corresponding to each of the candidate operator combinations; Inputting each of the candidate output data and the first input data into the feature extraction model respectively, to obtain second feature information corresponding to each of the candidate operator combinations output by the feature extraction model; The target operator combination is determined according to a matching relationship between each of the second feature information and the first feature information and a plurality of the candidate operator combinations.

6. The method according to claim 1, wherein determining a target operator combination according to the first input data, the first output data and an operator set comprises: Determining a plurality of candidate operator combinations from the operator set; Inputting the first input data into a plurality of the candidate operator combinations to obtain candidate output data corresponding to each of the candidate operator combinations; The target operator combination is determined according to a matching relationship between each candidate output data and the first output data and a plurality of candidate operator combinations.

7. The method according to claim 1, wherein determining a target operator combination according to the first input data, the first output data and an operator set comprises: Determine a plurality of matching operator combinations according to the first input data, the first output data, and the operator set; Obtaining operation step information corresponding to each matching operator combination; Performing a screening operation on a plurality of matching operator combinations according to the operation step information; The target operator combination is determined according to the screening result corresponding to the screening operation.

8. The method according to claim 1, wherein determining a target operator combination according to the first input data, the first output data and an operator set comprises: Determine a preliminary operator combination according to the first input data, the first output data and the operator set; Inputting the first input data into the primary selection operator combination to obtain primary selection output data; performing an adjustment operation on the preliminary selection operator combination according to difference information between the first output data and the preliminary selection output data; The target operator combination is obtained according to the adjustment result corresponding to the adjustment operation.

9. The method according to claim 1, wherein determining a target operator combination according to the first input data, the first output data and an operator set comprises: Determining an operator combination to be verified according to the first input data, the first output data and an operator set; Inputting the second input data into the operator to be replaced to obtain second output data; Inputting the second input data into the operator combination to be verified to obtain output data to be verified; When the second output data and the output data to be verified satisfy a verification condition, the operator combination to be verified is determined as the target operator combination.

10. An operator combination determination device, comprising: An acquisition module configured to acquire an operator to be replaced, wherein the operation of the operator to be replaced cannot be supported by the target operation module; An input module configured to input first input data into the operator to be replaced to obtain first output data; A determination module is configured to determine a target operator combination based on the first input data, the first output data and an operator set, wherein the operator set includes multiple candidate operators, the target operator combination includes at least one of the candidate operators, the operation of the candidate operator can be supported by the target operation module, and when the same data is input, the output of the operator to be replaced and the target operator combination satisfy a similarity relationship.

Images