Model deployment method and device, equipment, storage medium and program product

Abstract

The invention relates to a model deployment method and device, equipment, a storage medium and a program product, and relates to the technical field of artificial intelligence. The method comprises the following steps: converting a first model according to a model framework of the edge end equipment, and generating a second model capable of running in the model framework; according to the computing power of the edge end equipment, a first operator which is not supported by the computing power in the second model is replaced by a second operator, a third model is generated, and the second operator is an operator which has the same function as the first operator, is supported by the computing power and is supported by the second model; performing model training on the third model according to the training set to obtain a target model; and deploying the target model to the edge end device. According to the technical scheme, the effect of improving the model development efficiency can be achieved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a model deployment method, device, equipment, storage medium and program product. Background Art

[0002] With the improvement of edge device computing power, combined with the technical advantages of high real-time performance of offline models, the deployment of models on edge hardware devices has become an effective means to improve the real-time performance of hardware device services. Considering the limited computing power of edge hardware devices, existing technologies will perform lightweight processing on models during model development to minimize the model's demand for computing resources.

[0003] In the existing technology, model development mainly adopts a two-stage separation method of training and deployment, reducing the model's demand for computing power during the model training process, and then deploying the trained model in the corresponding hardware equipment.

[0004] However, when the model is deployed using existing technologies, the model development process is long and the development efficiency is low. Summary of the invention

[0005] One of the objects of the present invention is to provide a model deployment method, apparatus, equipment, storage medium and program product to solve the problem of long model development process and low development efficiency when the model is deployed through the existing technology; the second object is to provide a model deployment device; the third object is to provide an electronic device; the fourth object is to provide a computer-readable storage medium; the fifth object is to provide a computer program product.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A model deployment method, comprising:

[0008] According to the model framework of the edge device, the first model is converted to generate a second model that can be run in the model framework;

[0009] According to the computing capability of the edge device, the first operator in the second model that is not supported by the computing capability is replaced with the second operator to generate a third model, where the second operator is an operator that has the same function as the first operator and is supported by the computing capability and the second model;

[0010] Perform model training on the third model according to the training set to obtain a target model;

[0011] Deploy the target model to the edge device.

[0012] According to the above technical means, the model cannot be deployed because the hardware of the edge device does not support some operators in the model. Therefore, during the development of the model, the first model that needs to be developed and deployed can be converted to generate a second model that meets the operating requirements of the edge device deployment model framework; at the same time, the first operator that is not supported by the computing power of the edge device is obtained, and the first operator in the second model is replaced with the second operator supported by the computing power of the edge device. The situation of unsuccessful deployment caused by operator unsupport can be avoided when the model is finally deployed. Compared with the prior art, the present invention replaces the operators of the model and simplifies the corresponding model in the model development stage. While reducing the amount of model calculation, it takes into account the hardware support capabilities of the edge device, thereby improving the success rate of model deployment and achieving the technical effect of improving the efficiency of model development.

[0013] Further, according to the model framework of the edge device, the first model is converted to generate a second model that can be run in the model framework, including:

[0014] According to the model framework, the first model is transformed to generate a fourth model that can be run in the model framework;

[0015] According to the data flow of the fourth model, redundant operators in the fourth model are deleted to obtain the second model.

[0016] According to the above technical means, the calculation amount of the model is reduced because the redundant operators generated in the model conversion process are deleted.

[0017] Further, according to the computing capability of the edge device, the first operator that is not supported by the computing capability in the second model is replaced with the second operator to generate a third model, including:

[0018] Determine the computing power of the edge device based on the chip information of the edge device;

[0019] Determine the first operator in the second model according to the computing capability of the edge device;

[0020] Determine a third operator that has the same function as the first operator and is supported by the computing capability and the second model;

[0021] The operator with the smallest computing resource requirement among the third operators is determined as the second operator;

[0022] The first operator in the second model is replaced by the second operator to generate a third model.

[0023] According to the above technical means, the first operator that is not supported by the edge device is replaced with a second operator with lower computational complexity, thereby reducing the computational complexity and improving the success rate of model deployment.

[0024] Furthermore, the target model is deployed to the edge device, including:

[0025] Quantize the target model to obtain the quantized target model;

[0026] Deploy the quantized target model to the edge device.

[0027] According to the above technical means, the trained target model is quantized to reduce the computational complexity of the target model during runtime and improve computational efficiency.

[0028] Furthermore, the quantized target model is deployed to the edge device, including:

[0029] According to the chip information of the edge device, the operators in the quantized target model are compiled to generate a compiled target model;

[0030] Allocate computing units to operators in the compiled target model based on the chip information of the edge device;

[0031] For each computing unit, determine the scheduling strategy of the computing unit based on the pre-configured tasks in the computing unit, the priority of each task, and the priority of each operator assigned to the computing unit;

[0032] The operators in the compiled target model are loaded into the chip where the corresponding computing unit is located.

[0033] According to the above technical means, the chip information of the edge device is used to allocate computing units to the operators in the compiled target model, and resources are scheduled considering the computing resources of the edge device and the resource requirements of the operators, thereby improving the operating efficiency of the target model in the edge device.

[0034] Furthermore, before converting the first model according to the model framework of the edge device to generate a second model that can be run in the model framework, the method also includes:

[0035] Simplifying the structure of the initial model to obtain a first model;

[0036] The simplification process includes at least one of the following: reducing the number of convolution layers, reducing the number of convolution channels, and replacing two layers of convolution in the initial model with channel convolution and point convolution respectively.

[0037] According to the above technical means, the initial model that needs to be developed is simplified, thereby reducing the calculation amount of the model.

[0038] Further, the third model is trained according to the training set to obtain a target model, including:

[0039] Merging at least one first network layer in the third model into the second network layer to obtain a fifth model, wherein an output end of at least one first network layer is connected to an input end of the second network layer;

[0040] The fifth model is trained according to the training set to obtain a target model.

[0041] According to the above technical means, the network layers in the model are integrated during model training to further optimize the model.

[0042] Furthermore, the edge device is a vehicle, the model framework is a TFLite framework, and the target model is a sound zone separation model.

[0043] According to the above technical means, the model can be optimized during the development process of the sound zone separation model based on the model framework to obtain a sound zone separation model that can be deployed in a vehicle.

[0044] A model deployment device, comprising:

[0045] A first processing module is used to convert the first model according to the model framework of the edge device to generate a second model that can be run in the model framework;

[0046] A second processing module is used to replace the first operator in the second model that is not supported by the computing capability with the second operator according to the computing capability of the edge device, and generate a third model, where the second operator is an operator that has the same function as the first operator and is supported by the computing capability and the second model;

[0047] A third processing module is used to perform model training on the third model according to the training set to obtain a target model;

[0048] The fourth processing module is used to deploy the target model to the edge device.

[0049] According to the above technical means, the model cannot be deployed because the hardware of the edge device does not support some operators in the model. Therefore, the model deployment device can convert the first model that needs to be developed and deployed during the development of the model to generate a second model that meets the operating requirements of the edge device deployment model framework; at the same time, obtain the first operator that is not supported by the computing power of the edge device, and replace the first operator in the second model with the second operator supported by the computing power of the edge device. The situation of unsuccessful deployment caused by operator unsupport can be avoided when the model is finally deployed. Compared with the prior art, the present invention replaces the operators of the model and simplifies the corresponding model in the model development stage. While reducing the amount of model calculation, it takes into account the hardware support capabilities of the edge device, thereby improving the success rate of model deployment and achieving the technical effect of improving the efficiency of model development.

[0050] Furthermore, the first processing module is also used for:

[0051] According to the model framework, the first model is transformed to generate a fourth model that can be run in the model framework;

[0052] According to the data flow of the fourth model, redundant operators in the fourth model are deleted to obtain the second model.

[0053] According to the above technical means, since the first processing module deletes the redundant operators generated in the model conversion process, the calculation amount of the model is reduced.

[0054] Furthermore, the second processing module is also used for:

[0055] Determine the computing power of the edge device based on the chip information of the edge device;

[0056] Determine the first operator in the second model according to the computing capability of the edge device;

[0057] Determine a third operator that has the same function as the first operator and is supported by the computing capability and the second model;

[0058] The operator with the smallest computing resource requirement among the third operators is determined as the second operator;

[0059] The first operator in the second model is replaced by the second operator to generate a third model.

[0060] According to the above technical means, the first operator not supported by the edge device is replaced with a second operator with lower computational complexity based on the second processing module, thereby reducing the computational complexity and improving the success rate of model deployment.

[0061] Further, the fourth processing module is also used for:

[0062] Quantize the target model to obtain the quantized target model;

[0063] Deploy the quantized target model to the edge device.

[0064] According to the above technical means, the trained target model is quantized based on the fourth processing module to reduce the calculation complexity of the target model during operation and improve the calculation efficiency.

[0065] Furthermore, the fourth processing module is also used for:

[0066] According to the chip information of the edge device, the operators in the quantized target model are compiled to generate a compiled target model;

[0067] Allocate computing units to operators in the compiled target model based on the chip information of the edge device;

[0068] For each computing unit, determine the scheduling strategy of the computing unit based on the pre-configured tasks in the computing unit, the priority of each task, and the priority of each operator assigned to the computing unit;

[0069] The operators in the compiled target model are loaded into the chip where the corresponding computing unit is located.

[0070] According to the above-mentioned technical means, the fourth processing module uses the chip information of the edge device to allocate computing units to the operators in the compiled target model, and schedules resources considering the computing resources of the edge device and the resource requirements of the operators, thereby improving the operating efficiency of the target model in the edge device.

[0071] Furthermore, the first processing module is also used for:

[0072] Simplifying the structure of the initial model to obtain a first model;

[0073] The simplification process includes at least one of the following: reducing the number of convolution layers, reducing the number of convolution channels, and replacing two layers of convolution in the initial model with channel convolution and point convolution respectively.

[0074] According to the above technical means, the first processing module simplifies the initial model that needs to be developed, thereby reducing the calculation amount of the model.

[0075] Furthermore, the third processing module is also used for:

[0076] Merging at least one first network layer in the third model into the second network layer to obtain a fifth model, wherein an output end of at least one first network layer is connected to an input end of the second network layer;

[0077] The fifth model is trained according to the training set to obtain a target model.

[0078] According to the above technical means, the third processing module fuses the network layers in the model during model training, thereby further optimizing the model.

[0079] Furthermore, the edge device is a vehicle, the model framework is a TFLite framework, and the target model is a sound zone separation model.

[0080] According to the above technical means, the model deployment device can optimize the model based on the model framework during the development process of the sound zone separation model to obtain the sound zone separation model that can be deployed in the vehicle.

[0081] An electronic device, comprising: a memory, a processor;

[0082] Memory stores computer-executable instructions;

[0083] The processor executes the computer execution instructions stored in the memory, so that the processor executes the above model deployment method and any method further in the model deployment method.

[0084] A computer-readable storage medium stores computer-executable instructions, which are used to implement the above model deployment method and any method further described in the model deployment method when executed by a processor.

[0085] A computer program product includes a computer program, which, when executed by a processor, implements the above model deployment method and any method further described in the model deployment method.

[0086] Beneficial effects of the present invention:

[0087] (1) In the process of model development, the present invention replaces the first operator that is not supported by the computing power in the second model after the first model is converted based on the computing power of the edge device where the model is deployed, thereby avoiding the problem of model deployment failure caused by operator unsupport during model deployment, and achieving the technical effect of improving model development efficiency;

[0088] (2) During the model development process, the present invention deletes redundant operators generated when the first model is converted into the second model, thereby reducing the amount of calculation of the model and achieving the effect of optimizing the model performance.

[0089] (3) After the model training is completed, the present invention performs model quantization processing on the acquired target model to reduce the computational complexity during model operation. At the same time, the chip information of the edge device is combined with the operators in the target model to reasonably allocate computing units, thereby achieving the effect of improving the model operation efficiency.

[0090] (4) When developing the original model, the present invention simplifies the original model to reduce the amount of calculation of the model, thereby obtaining a first model with reduced structural complexity and reduced calculation complexity, thereby achieving the effect of optimizing model performance.

[0091] (5) During model training, the present invention integrates the network layers in the third model to reduce unnecessary calculation steps, reduce the complexity of the calculation process, and optimize the computing performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 Schematic diagram of the process of the model deployment method provided by the present invention Figure 1 ;

[0093] Figure 2A schematic diagram of the structure of the model block provided by the present invention before and after simplification;

[0094] Figure 3 A schematic diagram of the structure of the model block before and after the operator is deleted provided by the present invention;

[0095] Figure 4 A schematic diagram of the structure of the model network before and after operator replacement provided by the present invention;

[0096] Figure 5 A schematic diagram of the flow of the operator fusion method provided by the present invention;

[0097] Figure 6 Schematic diagram of the process of the model deployment method provided by the present invention Figure 2 ;

[0098] Figure 7 A schematic diagram of a flow chart of a method for deploying a sound zone separation model provided by the present invention;

[0099] Figure 8 A schematic diagram of the structure of the model deployment device provided by the present invention;

[0100] Fig. 9 This is a schematic structural diagram of an electronic device provided by the present invention.

[0101] The above drawings have shown clear embodiments of the present invention, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present invention in any way, but to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0102] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0103] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0104] First, the terms involved in the present invention are explained:

[0105] 2D Convolution (conv2d) Convolution: is an operation widely used in image processing and computer vision. Conv2d usually refers to the convolution operation performed on a two-dimensional input (such as an image), which is one of the core building blocks in convolutional neural networks.

[0106] TFLite (full name: TensorFlow Lite): is a lightweight solution for TensorFlow, designed specifically for mobile and edge devices, enabling machine learning models to run efficiently in resource-limited environments.

[0107] Batch Normalization (batchnorm): is a technique used to improve the efficiency and stability of deep learning model training. It normalizes the input of each layer so that it remains within a reasonable distribution range during training, thereby reducing internal covariate shift, accelerating the convergence process, and helping to improve model performance.

[0108] Rectified Linear Unit (Relu): is a widely used activation function, especially in neural networks in deep learning.

[0109] In the prior art, when developing an offline model deployed on an edge device, the two stages of model training and model deployment are separated; the model is optimized during the model training stage to ultimately obtain an optimized model; during the model deployment stage, the trained model is converted and the converted model is deployed on the edge device.

[0110] However, in actual model development, since model training is completed in the cloud or the center, only the optimization of the model itself is considered during the training process; when the model is deployed, the converted model still has inconsistencies between the model operator and the operator supported by the edge device. When the operator is inconsistent, the edge device cannot correctly or successfully deploy the model, resulting in the need to re-optimize and retrain the model in the end. Therefore, the overall model development process is extended and the workload of model development is increased, resulting in the technical problem of low model development efficiency in the existing technology.

[0111] In response to the above technical problems, the present invention proposes the following technical concept: the model cannot be deployed because the hardware of the edge device does not support some operators in the model. Therefore, during the development of the model, the first model that needs to be developed and deployed can be converted to generate a second model that meets the operating requirements of the edge device deployment model framework; at the same time, the first operator that is not supported by the computing power of the edge device is obtained, and the first operator in the second model is replaced with the second operator supported by the computing power of the edge device. The situation of unsuccessful deployment caused by operator unsupport can be avoided when the model is finally deployed. Compared with the prior art, the present invention replaces the operators of the model and simplifies the corresponding model in the model development stage. While reducing the amount of model calculation, it takes into account the hardware support capabilities of the edge device, thereby improving the success rate of model deployment and achieving the technical effect of improving the efficiency of model development.

[0112] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0113] Figure 1 Schematic diagram of the process of the model deployment method provided by the present invention Figure 1 ,like Figure 1 As shown, the method includes:

[0114] S101. According to the model framework of the edge device, convert the first model to generate a second model that can be run in the model framework.

[0115] In this step, edge devices refer to devices that run machine learning models or deep learning models in an edge computing environment. These devices can perform model reasoning tasks without a continuous network connection. Edge computing refers to moving data processing and computing power to the source of data production, which means that data does not have to be transmitted to a centralized cloud server for processing, but is processed on the device closest to the data, thereby reducing latency and improving data security.

[0116] A model framework refers to a model framework that is suitable for specific scenarios or specific hardware devices. In edge computing, different edge devices or hardware have different model frameworks that are suitable for their characteristics and application scenarios.

[0117] Optionally, before generating the second model, it is necessary to obtain the first model based on the initial model. A possible implementation method for determining the first model based on the initial model is: simplifying the structure of the initial model to obtain the first model; wherein the simplification includes at least one of the following: reducing the number of convolution layers, reducing the number of convolution channels, and replacing two layers of convolution in the initial model with channel convolution and point convolution, respectively.

[0118] Based on the above method of simplifying the initial model and determining the first model, a possible implementation of the method is:

[0119] a1. Determine at least one model block in the initial model based on the initial model.

[0120] In this step, the model block refers to the basic component unit of the initial model, which is used to construct the initial model.

[0121] a2. Simplify each model block and reduce the number of convolution layers when the model block contains multiple layers of convolution.

[0122] a3. For the multi-channel convolution in each model block, reduce the number of convolution channels.

[0123] For example, if there are 4 layers of convolution in a model block and the number of channels of the convolution network is 256, the simplified processing for the model block can be: reducing the 4 layers of convolution to 1 layer of convolution, reducing the number of channels of the convolution network to 128, reducing the computational cost of convolution without changing the convolution kernel, thereby optimizing the computational efficiency of the model.

[0124] a4. Replace the convolution in each model block and replace the two layers of continuous convolution with channel convolution and point convolution respectively.

[0125] In this step, when there is a Relu activation function after two layers of continuous convolution, the activation function can also be replaced by a common linear activation function.

[0126] For example, Figure 2 The schematic diagram of the structure of the model block before and after simplification provided by the present invention is as follows: Figure 2 As shown in the figure, there are two consecutive conv2d convolutional layers, and there is a Relu activation layer after the convolutional layer. The two conv2d convolutional layers are replaced with channel convolution and point convolution respectively, and the activation function of the Relu activation layer is replaced with a linear activation function with lower computational complexity.

[0127] Optionally, a possible implementation of generating the second model is:

[0128] b1. According to the model framework, the first model is converted to generate a fourth model that can be run in the model framework.

[0129] In this step, a possible implementation method of converting the first model into the fourth model is:

[0130] b11. Determine the difference between the first model and the model framework.

[0131] The differences between the first model and the model framework may include: differences in parameter initialization methods, differences in supported operation types, differences in data input and data output formats, and differences in implementation methods of different computing layers.

[0132] b12. Determine the tools used to implement model transformation based on the model framework.

[0133] b13. The model conversion tool imports the first model and performs model conversion.

[0134] b2. According to the data flow of the fourth model, the redundant operators in the fourth model are deleted to obtain the second model.

[0135] In this step, redundant operators generated during the model conversion process are deleted. The determination of redundant operators depends on the change in data flow in the fourth model. It is compared with the original first model to obtain the change in data flow and determine whether the change has actual computational significance. If not, redundant operators are determined based on the change and deleted.

[0136] For example, Figure 3 This is a schematic diagram of the structure of the model block before and after the operator is deleted provided by the present invention. Figure 3 As shown in the figure, the input parameters of the model block are: 16, 3, 64, 64, there is a transposition operator in the middle, and there is a convolution layer after the transposition operator. Through data flow analysis, the transposition operator is determined to be a redundant operator, and the calculation of the input parameters does not form the calculation of the convolution. Therefore, the transposition operator is deleted, and the resulting model block only contains the input layer and the convolution layer.

[0137] It should be understood that other existing means or tools may also be used to determine which operators in the fourth model are redundant so as to delete them. The embodiment of the present invention does not specifically limit the manner and means of determining redundant operators.

[0138] Among them, the types of operators generated in the actual model conversion process may include: transpose operator and reshape operator. The transpose operator is used to convert the dimensions of a matrix or tensor, and the reshape operator is used to change the tensor or shape of an array.

[0139] S102. According to the computing capability of the edge device, replace the first operator in the second model that is not supported by the computing capability with the second operator to generate a third model.

[0140] In this step, the second operator is an operator that has the same function as the first operator and is supported by the computing capability and the second model.

[0141] Optionally, a possible implementation of generating the third model is:

[0142] c1. Determine the computing power of the edge device based on the chip information of the edge device.

[0143] In this step, a possible implementation method of determining the computing capacity based on the chip information is: based on the chip version, chip name and chip specifications contained in the chip information, query the description document corresponding to the chip, and obtain the corresponding computing capacity based on the description document.

[0144] Among them, computing power can be: the core architecture of the chip, the pipeline depth of the chip, the computing power of each core in the chip, the expansion capability of the chip, and the storage capacity of the chip.

[0145] c2. Determine the first operator in the second model according to the computing capability of the edge device.

[0146] In this step, the first operator in the second model may be determined by:

[0147] c21. Evaluate the operator types supported by edge devices based on their computing capabilities;

[0148] c22. Obtain the operator type in the second model;

[0149] c23. Determine whether there is an operator type that is not supported by the edge device in the second model;

[0150] c24. Determine a first operator in the second model based on the unsupported operator type.

[0151] c3. Determine a third operator that has the same function as the first operator and is supported by the computing capability and the second model.

[0152] In this step, the determination of the third operator can be specifically as follows: for the first operator that needs to be replaced, obtain other operators that have functional consistency with the operator; search for operators supported by the computing power of the current edge device among other operators with consistent functions, and determine them as the operators to be selected; determine whether the operator to be selected can run in the second model, that is, whether the input and output of the operator to be selected are consistent with the input and output of the first operator at the same position in the second model, and if they are consistent, determine it as the third operator.

[0153] c4. Determine the operator with the smallest computing resource requirement among the third operators as the second operator.

[0154] In this step, in order to avoid increasing the computational complexity or complexity of the model operation after operator replacement, it is necessary to ensure that the computing resource requirement of the second operator used for replacement is low when replacing the operator. Therefore, it is necessary to select the third operator with the lowest computing resource requirement as the second operator to be replaced based on the computing resource requirement.

[0155] In this step, before determining the second operator, if the number of third operators is relatively large, further restrictions can be made to ensure that the calculation amount of the second operator is lower than that of the first operator. Specifically, the theoretical calculation amount of the first operator and the second operator is evaluated, and the number of floating-point operations in each operator is calculated; the third operator whose calculation amount is lower than that of the first operator is determined as the second operator that can be replaced.

[0156] c5. Replace the first operator in the second model with the second operator to generate a third model.

[0157] Based on the above steps, the present invention illustrates the replacement of operators. Figure 4 This is a schematic diagram of the structure of the model network before and after the operator replacement provided by the present invention. For example, Figure 4 As shown, there is a deconvolution operator with an input of [1, 8, 1, 32] and an output of [1, 32, 1, 161]. It is detected that the deconvolution operator is the first operator that is not supported by the edge device. The operator is replaced with a combination of a convolution operator and a linear operator. The input of the convolution operator is [1, 8, 1, 32], and the output is [1, 32, 1, 32]; the input of the linear operator is [1, 32, 1, 32], and the output is [1, 32, 1, 161].

[0158] S103: Perform model training on the third model according to the training set to obtain a target model.

[0159] Optionally, a possible implementation method of performing model training on the third model according to the training set to obtain the target model is: fusing at least one first network layer in the third model into the second network layer to obtain a fifth model. Thereafter, performing model training on the fifth model according to the training set to obtain the target model.

[0160] Therein, at least one output end of the first network layer is connected to an input end of the second network layer.

[0161] In practical applications, if the output of at least one of the first network layers is only connected to the input of the second network layer, it means that at least one of the first network layers is only used to calculate the parameters required by the second network layer. However, when the output of at least one of the first network layers is also connected to other network layers, at least one of the first network layers needs to provide parameters for both the second network layer and the other network layers. Therefore, it is also possible to further determine whether the output of at least one of the first network layers is only connected to the input of the second network layer. When it is determined that the output of at least one of the first network layers is only connected to the input of the second network layer, the at least one first network layer is then merged into the second network layer.

[0162] Exemplarily, the present invention provides an operator fusion method, Figure 5 A flow chart of the operator fusion method provided by the present invention is shown in FIG. Figure 5 As shown, it includes two operators: conv2d convolution layer and batchnrom normalization layer. The formula corresponding to the conv2d convolution layer is:

[0163] xi=w*xi-1+b

[0164] Among them, w is the weight, b is the bias, X i is the convolution parameter.

[0165] The formula corresponding to batchnrom normalization layer is:

[0166]

[0167] Where: u is the mean of a normalized layer, σ is the standard deviation, ε, γ, β are learnable parameters, and y is the result value of the normalized layer.

[0168] Fusion of two layers: Substitute the formula of the conv2d convolutional layer into the formula of the batchnrom normalization layer. The formula after substitution is:

[0169]

[0170] The fused formula is:

[0171]

[0172] It should be understood that the principle and process of model training the third model (or fifth model) according to the training set can be determined based on the training purpose and the input and output format of the third model (or fifth model), and the embodiments of the present invention do not impose specific restrictions on this.

[0173] S104: deploy the target model to the edge device.

[0174] In a specific scenario, the edge device can be implemented as a vehicle, the model framework can be implemented as a TFLite framework, and the target model can be implemented as a sound zone separation model.

[0175] It should be noted that the edge device, model framework and target model mentioned in the present invention are only used as an example and do not limit the specific edge device, model framework and target model.

[0176] In this step, the implementation method of deploying the target model to the edge device includes: quantizing the target model to obtain the quantized target model; and deploying the quantized target model to the edge device. The deployment of the target model is described in detail in the following embodiments, and no unnecessary details are given in this step.

[0177] This embodiment proposes a model deployment method, which replaces the first operator that is not supported by the computing power in the second model after the conversion of the first model based on the computing power of the edge device where the model is deployed during the model development process to obtain a third model. Model training and model deployment are performed on the third model, thereby avoiding the problem of model deployment failure due to operator unsupport during model deployment. At the same time, when the original model is developed, the original model is simplified to reduce the amount of calculation of the model, thereby obtaining a first model with reduced structural complexity and reduced calculation complexity; when the model is trained, the network layer in the third model is integrated to reduce unnecessary calculation steps and reduce the complexity of the calculation process. Compared with the prior art, the present invention replaces the compatible operator for the device to be deployed on the basis of the optimized model, thereby ensuring the model performance while avoiding repetitive model development caused by compatibility, thereby achieving the technical effect of improving the efficiency of model development.

[0178] Based on the above embodiments, Figure 1 The deployment of the model in step S104 in the illustrated embodiment is further elaborated in detail. Figure 6 Schematic diagram of the process of the model deployment method provided by the present invention Figure 2 ,like Figure 2 As shown, the method includes:

[0179] S601: quantize the target model to obtain the quantized target model.

[0180] In this step, the target model is quantized by static quantization. A possible implementation method of model quantization is:

[0181] d1. Data flow: Quantize the data flow in the target model calculation to int8, reduce the data bit width, and improve the memory access speed. Int8 refers to low-precision integers.

[0182] d2. Weight quantization: After training, the dynamic range of each layer’s weights must be determined, which refers to the maximum and minimum values.

[0183] d3, mapping: Map floating point weights to int8 range.

[0184] The mapping range can be [-128,127].

[0185] d4. Activation quantization:

[0186] d41. First, feed the model batches of data through the network and calculate the resulting distribution of different activations.

[0187] The specific implementation can be: inserting an observer into the model, and the observer records the maximum value, minimum value and distribution of the activation value of each layer.

[0188] d42. Based on the activation distribution information collected previously, the appropriate scaling factor and zero offset can be calculated to convert the floating point activation value to int8.

[0189] d43. Quantize the floating point activation values ​​to integers using the calculated scaling factor and zero offset.

[0190] S602: Compile the operators in the quantized target model according to the chip information of the edge device to generate a compiled target model.

[0191] In this step, the purpose of model compilation is to convert high-level program code into low-level executable code or format optimized for specific hardware so that the target model can run efficiently on edge devices.

[0192] Optionally, the specific implementation method of model compilation may be: selecting a suitable compilation tool, using the compilation tool to compile each operator in the target model, performing specific optimization compilation for the edge device, and saving the compiled target model.

[0193] S603: Allocate computing units to operators in the compiled target model according to the chip information of the edge device.

[0194] In this step, the operator information in the target model can be extracted based on the compiled target model, the characteristics of the operator can be determined according to the operator information, and then the computing unit of the operator can be allocated according to the chip information of the edge device.

[0195] For example, operators with high computing requirements can be allocated to use graphics processing units, and operators with low computing requirements can use central processing units. If the edge device supports multi-core processing, different operators can be allocated to different cores, and the computing resource requirements of the operators are matched with the computing resources of the cores to obtain the optimal resource allocation.

[0196] S604: For each computing unit, determine a scheduling strategy for the computing unit according to the pre-configured tasks in the computing unit, the priority of each task, and the priority of each operator allocated in the computing unit.

[0197] In this step, the scheduling of computing units means that when one computing unit is responsible for processing multiple operators and multiple tasks, the computing unit can be scheduled according to the priority of each task or the priority of the operator to ensure that the computing unit gives priority to processing high-priority tasks and the operators corresponding to the tasks, thereby ensuring the reasoning efficiency of the model.

[0198] S605: Load the operators in the compiled target model to the chip where the corresponding computing unit is located.

[0199] In this step, the deployment of operators is related to the computing units to which the operators are assigned. After determining the computing units corresponding to the operators and the scheduling strategies of the operators in each computing unit, the operators need to be assigned to the corresponding computing units for deployment. The chips to which the computing units belong are different, and the chips on which the operators are deployed are different.

[0200] In this embodiment, after the model training is completed, when the model is deployed, it is necessary to quantize the target model to reduce the computational complexity of the target model and further optimize the operating efficiency of the model. At the same time, the model deployment stage needs to perform operator scheduling based on the dependence of operators in the model on different computing units to optimize model deployment and improve the reasoning efficiency of the model.

[0201] Based on the above embodiments, the present invention also provides a method for deploying a sound zone separation model implemented in a vehicle based on a TFLite framework. Figure 7 A flow chart of a method for deploying a sound zone separation model provided by the present invention, the method comprising:

[0202] e1. Model simplification: Simplify the model blocks in the tone zone separation model, reduce the number of convolutional layers in the model blocks, and reduce the number of convolutional channels in the convolutional layers; replace the complex convolutional layers; determine the main structure of the tone zone separation model through model simplification.

[0203] e2. Optimize the calculation graph layout: After determining the main structure of the model, adjust the operators of the model.

[0204] The framework to which the model will be converted and deployed is the TFLite framework. This step adjusts the model operators based on the target conversion framework. Use the model conversion methods or functions provided by TFLite to convert the original model into a model that adapts to the deployment of the target framework; perform operator optimization during the conversion process.

[0205] Specifically:

[0206] Data flow adjustment: data format adjustment, ensure that the input data format meets the requirements of TFLite, usually NHWC (batch size, height, width, channel) format; data dimension adjustment, reduce data dimension.

[0207] Operator removal: redundant operators are generated during the model conversion process, and these operators are removed, such as the transpose operator and the reshape operator.

[0208] e3. Model fusion: Fusion of the convolutional layer and batch normalization layer in the sound zone separation model.

[0209] The fusion method is to use the convolution layer as a parameter and the batch normalization layer as a calculation formula, and then bring in the parameters to achieve operator fusion.

[0210] e4. Model conversion: obtain operators in the sound zone separation model that are incompatible with the target hardware; replace these incompatible operators.

[0211] This step means that when the sound zone separation model is converted for the tifilte framework, it is also necessary to consider the compatibility between the operators in the sound zone separation model and the target hardware on which the model needs to be deployed, that is, to further optimize the operators for the target deployment hardware.

[0212] For example, if there is a long short-term memory network operator in the sound zone separation model, which is not compatible with the target hardware, it will be replaced with convolution and linear layers. Because the long short-term memory network operator generally involves a large amount of calculation, general edge devices or ordinary vehicle equipment do not support such calculations, so it needs to be replaced. However, in actual applications, it is necessary to determine which operators need to be replaced in combination with specific deployment devices.

[0213] e5. Model training: For models that have completed hardware device adaptation, model training is performed in combination with model training optimization strategies.

[0214] Model training optimization strategies include: tuning network weights, testing model accuracy, cleaning data sets, increasing the number of bad cases, and tuning learning rates.

[0215] e6. Determine whether the target sound zone separation model meets the standards based on the test data.

[0216] If the standard is met, execute e7; if the standard is not met, repeat step e1.

[0217] e7. Model quantization: When the target sound range separation model meets the standard, static quantization is used to reduce the size of the target sound range separation model and improve the inference speed of the target sound range separation model.

[0218] e8. Model deployment: compile the target sound zone separation model, optimize the model operator distribution, generate a scheduling strategy, and load the operator to the chip end of the vehicle.

[0219] Specifically: Use a specific compilation tool to convert the operators in the sound zone separation model into a format suitable for the target platform. Analyze the computing requirements and resource consumption of each operator. According to the characteristics of the operator and the hardware architecture, determine how to distribute each operator among different computing units to achieve load balancing and maximize resource utilization. Based on the distribution tuning results of the operator, generate a suitable scheduling strategy, which may include: determining the execution order of each operator; determining when to load data, execute operators, and write results back to memory; optimizing data flow paths to reduce latency and resource contention. After the scheduling strategy is generated, load the optimized operators and their related parameters onto the vehicle's chip. This may involve: placing model weights and parameters in appropriate storage areas; ensuring that the hardware can correctly parse and execute these operators, including all necessary initialization steps.

[0220] In this embodiment, a detailed explanation is given for deploying a sound zone separation model in a vehicle. The model is simplified during the model development process, and redundant operators are removed from the simplified model; the amount of calculation is reduced by using model fusion; the problem of incompatible operators on the vehicle side is solved by using operator replacement; the model reasoning efficiency is improved by using model quantization, and the model operation efficiency is optimized by using model deployment, so as to achieve the technical effect of improving the development efficiency of the sound zone separation model.

[0221] Figure 8 A schematic diagram of the structure of the model deployment device provided by the present invention, such as Figure 8 As shown, the model deployment device 80 provided in this embodiment includes:

[0222] The first processing module 801 is used to convert the first model according to the model framework of the edge device to generate a second model that can be run in the model framework;

[0223] The second processing module 802 is used to replace the first operator that is not supported by the computing capability in the second model with the second operator according to the computing capability of the edge device, and generate a third model, where the second operator is an operator that has the same function as the first operator and is supported by the computing capability and the second model;

[0224] The third processing module 803 is used to perform model training on the third model according to the training set to obtain a target model;

[0225] The fourth processing module 804 is used to deploy the target model to the edge device.

[0226] Furthermore, the first processing module 801 is further used for:

[0227] According to the model framework, the first model is transformed to generate a fourth model that can be run in the model framework;

[0228] According to the data flow of the fourth model, redundant operators in the fourth model are deleted to obtain the second model.

[0229] Further, the second processing module 802 is also used for:

[0230] Determine the computing power of the edge device based on the chip information of the edge device;

[0231] Determine the first operator in the second model according to the computing capability of the edge device;

[0232] Determine a third operator that has the same function as the first operator and is supported by the computing capability and the second model;

[0233] The operator with the smallest computing resource requirement among the third operators is determined as the second operator;

[0234] The first operator in the second model is replaced by the second operator to generate a third model.

[0235] Further, the fourth processing module 804 is further used for:

[0236] Quantize the target model to obtain the quantized target model;

[0237] Deploy the quantized target model to the edge device.

[0238] Furthermore, the fourth processing module 804 is also used for:

[0239] According to the chip information of the edge device, the operators in the quantized target model are compiled to generate a compiled target model;

[0240] Allocate computing units to operators in the compiled target model based on the chip information of the edge device;

[0241] For each computing unit, determine the scheduling strategy of the computing unit based on the pre-configured tasks in the computing unit, the priority of each task, and the priority of each operator assigned to the computing unit;

[0242] The operators in the compiled target model are loaded into the chip where the corresponding computing unit is located.

[0243] Furthermore, the first processing module 801 is also used for:

[0244] Simplifying the structure of the initial model to obtain a first model;

[0245] The simplification process includes at least one of the following: reducing the number of convolution layers, reducing the number of convolution channels, and replacing two layers of convolution in the initial model with channel convolution and point convolution respectively.

[0246] Furthermore, the third processing module 803 is also used for:

[0247] Merging at least one first network layer in the third model into the second network layer to obtain a fifth model, wherein an output end of at least one first network layer is connected to an input end of the second network layer;

[0248] The fifth model is trained according to the training set to obtain a target model.

[0249] Furthermore, the edge device is a vehicle, the model framework is a TFLite framework, and the target model is a sound zone separation model.

[0250] The model deployment device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effects are similar, and will not be described in detail in this embodiment.

[0251] Fig. 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Fig. 9 As shown, the electronic device 90 provided in this embodiment includes: at least one processor 901 and a memory 902. Optionally, the device 90 further includes a communication component 903. The processor 901, the memory 902 and the communication component 903 are connected via a bus 904.

[0252] In a specific implementation process, at least one processor 901 executes the computer execution instructions stored in the memory 902, so that at least one processor 901 executes the above method.

[0253] The specific implementation process of the processor 901 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0254] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0255] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0256] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present invention is not limited to only one bus or one type of bus.

[0257] The present invention also provides a computer program product, comprising a computer program, and the computer program implements the above method when executed by a processor.

[0258] The present invention also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0259] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0260] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0261] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0262] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0263] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0264] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0265] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0266] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

[0267] The above embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Any equivalent substitution or change made by a person skilled in the art based on the present invention is within the protection scope of the present invention.

Claims

1. A model deployment method, characterized in that: include: According to the model framework of the edge device, convert the first model to generate a second model that can be run in the model framework; According to the computing capability of the edge device, replacing the first operator in the second model that is not supported by the computing capability with a second operator to generate a third model, wherein the second operator is an operator that has the same function as the first operator and is supported by the computing capability and the second model; Performing model training on the third model according to the training set to obtain a target model; Deploy the target model to the edge device.

2. The method according to claim 1, characterized in that The converting the first model according to the model framework of the edge device to generate a second model that can be run in the model framework includes: According to the model framework, converting the first model to generate a fourth model that can be run in the model framework; According to the data flow of the fourth model, redundant operators in the fourth model are deleted to obtain the second model.

3. The method according to claim 1 or 2, characterized in that: The step of replacing the first operator that is not supported by the computing capability in the second model with the second operator according to the computing capability of the edge device to generate the third model includes: Determining the computing capability of the edge device according to the chip information of the edge device; Determining the first operator in the second model according to the computing capability of the edge device; Determine a third operator that has the same function as the first operator, is supported by the computing capability, and is supported by the second model; Determine the operator with the smallest computing resource requirement among the third operators as the second operator; The first operator in the second model is replaced by the second operator to generate the third model.

4. The method according to claim 1 or 2, characterized in that: The deploying the target model to the edge device includes: Performing model quantization on the target model to obtain a quantized target model; Deploy the quantized target model to the edge device.

5. The method according to claim 4, characterized in that The step of deploying the quantized target model to the edge device includes: Compiling the operators in the quantized target model according to the chip information of the edge device to generate a compiled target model; Allocating computing units to operators in the compiled target model according to chip information of the edge device; For each computing unit, determining a scheduling strategy for the computing unit according to pre-configured tasks in the computing unit, the priority of each task, and the priority of each operator assigned to the computing unit; The operators in the compiled target model are loaded into the chip where the corresponding computing unit is located.

6. The method according to claim 1 or 2, characterized in that: Before converting the first model according to the model framework of the edge device to generate a second model that can be run in the model framework, the method further includes: Simplifying the structure of the initial model to obtain the first model; The simplification process includes at least one of the following: reducing the number of convolution layers, reducing the number of convolution channels, and replacing two layers of convolution in the initial model with channel convolution and point convolution respectively.

7. The method according to claim 1 or 2, characterized in that: The performing model training on the third model according to the training set to obtain a target model includes: Merging at least one first network layer in the third model into the second network layer to obtain a fifth model, wherein the output end of the at least one first network layer is connected to the input end of the second network layer; The fifth model is trained according to the training set to obtain the target model.

8. The method according to claim 1 or 2, characterized in that: The edge device is a vehicle, the model framework is a TFLite framework, and the target model is a sound zone separation model.

9. A model deployment device, characterized in that: include: A first processing module, configured to convert the first model according to a model framework of the edge device to generate a second model that can be run in the model framework; A second processing module is used to replace the first operator that is not supported by the computing capability in the second model with a second operator according to the computing capability of the edge device, to generate a third model, wherein the second operator is an operator that has the same function as the first operator and is supported by the computing capability and the second model; A third processing module is used to perform model training on the third model according to the training set to obtain a target model; The fourth processing module is used to deploy the target model to the edge device.

10. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed by a processor.

12. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 8 when being executed by a processor.

Images