A method and apparatus for model deployment

Abstract

The specification discloses a model deployment method and device, the model deployment method comprising: determining each network branch contained in a preset model and needing to be replaced as each target network branch; determining, according to initial network parameters corresponding to the each target network branch, network parameters corresponding to a specified network branch after the each target network branch is replaced with the specified network branch; updating the initial network parameters according to the network parameters corresponding to the specified network branch, and performing pseudo-quantization processing on the updated initial network parameters; replacing the each target network branch contained in the model after the pseudo-quantization processing with the specified network branch, and performing model deployment according to the model after the replacement.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence, and in particular to a method and apparatus for deploying a model. Background Technology

[0002] With the development of deep learning technology, the application of neural networks has become increasingly mature. Currently, neural network models are widely used in various business scenarios in industry and daily life. Some of these fields (such as industry) typically have very strict requirements for the accuracy and latency of neural network models, requiring models deployed on terminals to maintain high accuracy while having low latency to fully meet business needs. To achieve this goal, many neural network compression and optimization techniques have been studied in depth, including two classic model optimization methods: reparameterization and model quantization.

[0003] Because quantization of network parameters (i.e., compressing high-precision network parameters into low-precision network parameters, such as compressing floating-point precision network parameters into integer precision network parameters) introduces certain errors, the current common practice is to first train the model, then fuse multiple network branches of the trained model, and then train the fused single-branch model. During this training process, pseudo-quantization is performed on the network parameters of the single-branch model (i.e., first compressing the high-precision network parameters into low-precision network parameters with some quantization error, and then restoring these low-precision network parameters with quantization error to high-precision network parameters, but these high-precision network parameters now contain quantization error). Finally, the corresponding network parameters in the model after this training are quantized. However, this optimization process reduces the model's accuracy, which may not meet the needs of some applications with strict accuracy requirements.

[0004] Therefore, how to optimize the model and improve its running efficiency while ensuring its accuracy is an urgent problem to be solved. Summary of the Invention

[0005] This specification provides a model deployment method and apparatus to partially solve the aforementioned problems existing in the prior art.

[0006] The following technical solution is adopted in this specification:

[0007] This specification provides a method for model deployment, including:

[0008] Identify the network branches that need to be replaced in the preset model and use them as target network branches.

[0009] Based on the initial network parameters corresponding to each target network branch, determine the network parameters corresponding to the specified network branch after assuming that each target network branch is equivalently replaced with the specified network branch.

[0010] The initial network parameters are updated according to the network parameters corresponding to the specified network branch, and pseudo-quantization is performed on the updated initial network parameters.

[0011] The target network branches in the model after pseudo-quantization are equivalently replaced with the specified network branches, and the model is deployed according to the replaced model.

[0012] Optionally, before replacing each target network branch in the pseudo-quantized model with a specified network branch, the method further includes:

[0013] The model after pseudo-quantization is trained to obtain the optimized model.

[0014] Deploy the model based on the replaced model, specifically including:

[0015] The optimized model is then deployed.

[0016] Optionally, before deploying the model based on the replaced model, the method further includes:

[0017] The network parameters corresponding to the specified network branch are quantized.

[0018] Deploy the model based on the replaced model, specifically including:

[0019] The quantized model is then deployed.

[0020] Optionally, the model after pseudo-quantization is trained to obtain an optimized model, specifically including:

[0021] Get the input data;

[0022] The input data is input into the model after pseudo-quantization processing, and the output result corresponding to the input data is determined;

[0023] The optimized model is obtained by training the model after pseudo-quantization with the optimization objective of minimizing the deviation between the output result and the actual label corresponding to the input data.

[0024] Optionally, input data is obtained, specifically including:

[0025] Obtain initial input data;

[0026] The initial input data is subjected to pseudo-quantization to obtain the input data.

[0027] Optionally, the input data is input into the model after pseudo-quantization processing to determine the output result corresponding to the input data, specifically including:

[0028] Based on the input data, determine the data distribution corresponding to the input data.

[0029] Based on the data distribution, the input data is regularized, and based on the processed input data, the corresponding output result is determined.

[0030] Optionally, the method further includes:

[0031] Determine the data distribution corresponding to the historical input data, and use it as the historical data distribution;

[0032] The historical data distribution is updated based on the input data to obtain the updated data distribution;

[0033] Deploying the optimized model specifically includes:

[0034] Based on the updated data distribution, the optimized model is deployed.

[0035] This specification provides an apparatus for model deployment, comprising:

[0036] The first determining module identifies the network branches that need to be replaced in the preset model, which are then used as the target network branches.

[0037] The second determining module determines, based on the initial network parameters corresponding to each target network branch, the network parameters corresponding to the specified network branch after assuming that each target network branch is equivalently replaced with the specified network branch;

[0038] The processing module updates the initial network parameters according to the network parameters corresponding to the specified network branch, and performs pseudo-quantization processing on the updated initial network parameters.

[0039] The deployment module replaces each target network branch in the model after pseudo-quantization with a specified network branch, and deploys the model based on the replaced model.

[0040] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for deploying the above-described model.

[0041] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a method for deploying the above-described model.

[0042] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0043] In the model deployment method provided in this specification, the network parameters corresponding to the specified network branch after assuming that each target network branch is equivalently replaced with the specified network branch are first determined based on the initial network parameters corresponding to each target network branch. The initial network parameters are then updated based on the network parameters corresponding to the specified network branch, and pseudo-quantization is performed on the updated initial network parameters. Finally, each target network branch in the pseudo-quantized model is equivalently replaced with the specified network branch, and the final model is then deployed.

[0044] Existing techniques quantize the network parameters corresponding to the fewer target network branches only after effectively replacing a larger number of target network branches with a smaller number of specified network branches. However, this quantization process introduces significant errors. As the aforementioned method demonstrates, this specification performs pseudo-quantization on the network parameters during the model training phase and maintains a model structure with a larger number of target network branches during training. This eliminates some of the errors from pseudo-quantized network parameters during training. Therefore, compared to existing methods that train models with pseudo-quantized parameters using a model structure with fewer specified network branches, this approach further improves model accuracy. Attached Figure Description

[0045] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0046] Figure 1 This document provides a schematic diagram of an existing model optimization method.

[0047] Figure 2 This is a flowchart illustrating a method for deploying a model provided in this specification;

[0048] Figure 3 This document provides a schematic diagram of a model deployment method.

[0049] Figure 4 A schematic diagram of a model deployment device provided in this specification;

[0050] Figure 5 This specification provides a corresponding Figure 1 A schematic diagram of an electronic device. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0052] Since each network branch of the model corresponds to different network parameters, if these network parameters are pseudo-quantized first, the differences between these network parameters will be further amplified after training. This makes it impossible to perform equivalent replacements on the various network branches of the model during subsequent optimization. Therefore, existing techniques typically employ methods such as... Figure 1 The method shown is used to deploy the model.

[0053] Figure 1 This is a schematic diagram of an existing model optimization method provided in this specification.

[0054] In this model, network branch 1, network branch 2, and network branch 3 are the target network branches that need to be replaced in the subsequent optimization stage. The existing method first trains the model with many target network branches to obtain a model with multiple target network branches. Then, each target network branch in the model after the first training is equivalently replaced with a specified network branch to obtain a model with the specified network branch. Then, the model parameters of the model with the specified network branch are pseudo-quantized. After processing, the model is trained to obtain a pseudo-quantized model with the specified branch network structure. Finally, the model is quantized and then deployed.

[0055] In this process, the model with pseudo-quantized network parameters can only be trained using a network structure with fewer specified network branches. Compared to training a model with more target network branches, the model accuracy is very limited.

[0056] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0057] Figure 2 This document provides a flowchart illustrating a method for deploying a model, which includes the following steps:

[0058] S201: Determine the network branches that need to be replaced in the preset model, and use them as the target network branches.

[0059] Training neural network models typically takes place in cloud data centers with high processing power and memory resources, such as servers. This ensures that even complex neural network models can be trained successfully while maintaining accuracy. After training, the server usually needs to install the model on client devices such as mobile phones, laptops, tablets, and desktop computers to complete the model deployment.

[0060] However, in practical applications, due to limitations in the configuration of some devices (such as processors and memory), the processing power and memory resources of terminal devices differ significantly from those of servers. If a trained model is directly deployed on the terminal device, the lower processing power of the terminal device often causes the model to run slowly or even fail to start normally, and it also consumes a large amount of memory on the terminal device.

[0061] For example, mobile phones typically have far lower processing power and memory resources than servers. If a model trained on a server is directly deployed to a client installed on a mobile phone, the phone's own configuration simply cannot make the model run smoothly on the client, and may even cause the client to crash. Moreover, models trained on servers usually consume a lot of memory resources, and if they are deployed directly on mobile phones, it will inevitably affect the user experience.

[0062] Therefore, it is necessary to optimize the trained model first to simplify its logical structure and reduce the memory resources it occupies, and then deploy the optimized model.

[0063] Based on this, this specification provides a method for model deployment, wherein, during the training process of the preset model on the server, it is necessary to determine the network branches that need to be replaced in the preset model as the target network branches.

[0064] Specifically, after model training is complete, the server typically optimizes the trained model. This involves using equivalent formulas or algorithms to replace equivalent network branches with specified network branches, thereby merging and simplifying multiple network branches in the trained model and optimizing its running speed. Therefore, the server can identify the network branches in the preset model that require equivalent replacement, as well as those used during training but no longer relevant after deployment, and designate these branches as target network branches.

[0065] It should be noted that the specified network branch can be one or more, but the number of specified network branches must be less than the number of target network branches in order to optimize the model.

[0066] In this specification, the execution subject of the method used to implement model deployment can refer to a specified device such as a server set up on the business platform. For ease of description, this specification will only use the server as the execution subject as an example to illustrate one model deployment method provided in this specification.

[0067] S202: Based on the initial network parameters corresponding to each target network branch, determine the network parameters corresponding to the specified network branch after assuming that each target network branch is equivalently replaced with the specified network branch.

[0068] During model training, the network parameters (such as convolution weights) corresponding to each network branch have certain differences. If pseudo-quantization is performed on the initial network parameters corresponding to each target network branch in the model before training, these differences between the different initial network parameters will often be further amplified after the model is trained. This makes it impossible to equivalently replace these target network branches by specifying a network branch during model optimization. Forcing a replacement will cause significant errors. The pseudo-quantization process will be described in detail below and will not be elaborated on here.

[0069] Therefore, the server can first determine the network parameters corresponding to the specified network branch after assuming that each target network branch is equivalently replaced by the specified network branch, and then use these network parameters to update the initial network parameters of each target network branch. In this way, after the model training is completed, there will be small differences or even no differences between each target network branch. Thus, by specifying the network branch, the target network branches whose network parameters are updated by the specified branch can be equivalently replaced.

[0070] Specifically, after determining the target network branches, the server can perform equivalent replacement simulations on these target network branches. Through simulation, it can determine the network parameters corresponding to the specified network branches after these target network branches are equivalently replaced with the specified network branches.

[0071] It should be noted that the simulation here is only to determine the network parameters corresponding to the specified network branch. In the actual training process, the model with the initial network structure (i.e., the network structure containing the target network branch) is still trained.

[0072] S203: Update the initial network parameters according to the network parameters corresponding to the specified network branch, and perform pseudo-quantization processing on the updated initial network parameters.

[0073] After the server determines that each target network branch is equivalently replaced with the network parameters corresponding to the specified network branch, the server can update the target network parameters based on the network parameters corresponding to the specified network branch. For example, the server can replace or adjust at least some of the target network parameters based on the network parameters corresponding to the specified network branch to obtain the updated network parameters corresponding to each target network branch.

[0074] Since the configuration of the terminal device (such as a mobile phone or tablet) running the model is usually lower than that of the server after the model is deployed on the client, it is necessary to quantize the corresponding network parameters in the replaced model to compress them to a range that matches the configuration of the terminal device.

[0075] However, quantizing the network parameters in the trained model will introduce certain errors, which will affect the accuracy of the deployed model. Therefore, in order to reduce these errors, the server can set corresponding pseudo-quantization nodes in the model. When the server trains the model, it can use these pseudo-quantization nodes to perform pseudo-quantization on the updated initial network parameters, thereby reducing or eliminating the errors generated in the subsequent model quantization process.

[0076] Specifically, the server can use pseudo-quantization nodes to first quantize the network parameters, obtaining quantized network parameters with quantization errors. Then, the quantized network parameters are restored to their original state, yielding restored network parameters. The server can compare these restored parameters with the original network parameters to determine the quantization errors generated during the quantization process. These quantization errors are then combined with the original network parameters to obtain pseudo-quantized network parameters carrying further quantization errors. In this way, these quantization errors can at least partially offset the errors generated during the subsequent optimization process of quantizing the network parameters, thereby improving the model's recognition accuracy.

[0077] For example, for a 32-bit floating-point network parameter in a target network branch, the server can quantize it using certain quantization parameters (such as quantization ratio, quantization difference, etc.), thereby representing it as an 8-bit integer network parameter. This quantized integer network parameter will carry a certain quantization error. Then, this integer network parameter with quantization error is restored using the aforementioned quantization parameters. The restored floating-point network parameter is compared with the original floating-point network parameter to determine the quantization error generated during the quantization process. Finally, this quantization error is combined with the original floating-point network parameter to obtain a floating-point network parameter carrying the quantization error.

[0078] Of course, in practical applications, the server can also first perform pseudo-quantization on the network parameters corresponding to the specified network branch, and then update the initial network parameters corresponding to each target network branch using these pseudo-quantized network parameters.

[0079] S204: Replace each target network branch in the model after pseudo-quantization with a specified network branch, and deploy the model according to the replaced model.

[0080] After the server performs pseudo-quantization on the updated initial network parameters, it can obtain the pseudo-quantized model. In this model, the network parameters corresponding to each target network branch are obtained by updating the network parameters of the specified network branch and then performing pseudo-quantization on the updated network parameters.

[0081] The server can then train the pseudo-quantized model. During the training process, the server can first acquire input data and input it into the pseudo-quantized model. The input data is then regularized through the Batch Normalization (BN) layer in the model to obtain processed input data, thereby limiting the distribution range of the data during training and preventing overfitting of the trained model.

[0082] Specifically, the server can determine the data distribution of these input data through the Batch Normalization (BN) layer in the model. This data distribution can include the mean and variance of the input data input into the pseudo-quantized model, and then perform regularization processing on these input data based on this data distribution.

[0083] Since the model is deployed on a client, the input data is usually quantized during its actual operation. For example, when performing image recognition through a specific client installed on a mobile phone, the model deployed on the client will typically compress images with a higher initial resolution into images with a lower resolution to improve the model's recognition efficiency. Moreover, since pseudo-quantization has already been performed on the initial network parameters during training, pseudo-quantization of the input data can ensure the model's training effect.

[0084] Therefore, to simulate the input effect during actual operation, to better match the trained model with the deployment environment configuration, and to ensure the model's training effect, the server can first obtain the initial input data, and then perform pseudo-quantization on this initial data to obtain pseudo-quantized input data. The method for pseudo-quantization of the initial input data is the same as the method for quantizing the updated initial network parameters described above, and will not be elaborated further here.

[0085] Furthermore, the server can determine the historical input data distribution (such as the mean of the input data distribution for each training round) during the training of the pseudo-quantized model, using this distribution as the historical data distribution. This historical data distribution is then updated based on the input data for each subsequent round, resulting in the updated data distribution. It should be noted that during each training round of the pseudo-quantized model, only the data distribution corresponding to the input data from that round is used to perform regularization on the input data.

[0086] The pseudo-quantized model can then determine the recognition result corresponding to the input data using the processed input data. The optimization objective is to minimize the deviation between the recognition result and the actual information corresponding to the input data. The pseudo-quantized model is then trained until the training objective is met. This training objective can be that the model converges to a preset threshold range, or that a preset number of training iterations are reached, to ensure the model's recognition accuracy. The preset threshold range and the preset number of training iterations can be set according to actual circumstances; this specification does not provide specific limitations.

[0087] In practical applications, the initial input data can be image data, voiceprint data, text data, etc. Correspondingly, the actual information corresponding to the initial input data can be image information contained in image data, voiceprint information contained in voiceprint data, and text information contained in text data, etc. This specification does not make specific limitations on this.

[0088] After training the model, the server can optimize the trained model to obtain an optimized model.

[0089] Specifically, the server can equivalently replace each network branch in the trained model with a specified network branch to obtain a replaced model, thereby simplifying the model's network structure and improving its running efficiency. During this process, each Batch Normalization (BN) network in the model will also be equivalently replaced with the specified network in the specified branch.

[0090] In addition, the server can quantize the network parameters corresponding to a specified branch in the model after the replacement is completed, so as to reduce its memory usage and thus obtain an optimized model.

[0091] The server can then deploy the optimized model on the client.

[0092] It is important to emphasize that, since pseudo-quantization has already been performed on the updated network parameters in the model during training, the network parameters contained in each target network branch in the trained model already have a certain quantization error. When the target network branch is equivalently replaced with the specified network branch during optimization, the network parameters corresponding to the specified network branch also have the aforementioned quantization error. These quantization errors can at least partially offset the errors generated when quantizing the network parameters corresponding to the replaced model, thereby improving the model accuracy.

[0093] It should be noted that all actions involving the acquisition of signals, information, or data in this manual are performed in accordance with the relevant data protection laws and regulations of the country where the device is located, and with the authorization granted by the owner of the relevant device.

[0094] As can be seen from the above method, the model deployment method provided in this specification can update the initial network parameters by specifying the network parameters corresponding to the network branches. This allows the model to replace the target network branch with the specified network branch in subsequent optimization processes, even if these updated network parameters are pseudo-quantized during training. This enables the model to be trained with pseudo-quantized parameters through a multi-network branch model structure. Compared with existing methods that train models with pseudo-quantized parameters using fewer specified network branches, this method further improves the recognition accuracy of the model.

[0095] To facilitate understanding, this specification also provides a schematic diagram of a model deployment method, which clearly distinguishes it from existing methods.

[0096] Figure 3 This is a schematic diagram illustrating a method for deploying a model as provided in this specification.

[0097] In this scheme, network branch 1, network branch 2, and network branch 3 are target network branches included in the preset model that can be replaced in the subsequent optimization stage. The server first determines the network parameters corresponding to the specified network branch after assuming that the target network branch is equivalently replaced with the specified network branch. Then, the initial network parameters of each target branch are updated using the network parameters to obtain the updated parameters. The updated parameters are then pseudo-quantized to obtain a pseudo-quantized model containing the target network branch and the pseudo-quantized parameters corresponding to each target network branch. The model is then trained.

[0098] After training, the server can replace each target network branch in the model with a specified network branch, quantize the network parameters corresponding to the specified network branch, and then deploy the quantized model.

[0099] Compared to existing methods, this approach can train models with pseudo-quantization parameters using a network structure with more target network branches during model training. Compared to the current method of training models with extrinsic quantization parameters using a network structure with fewer specified network branches, the accuracy of the deployed model is significantly improved.

[0100] The above describes one or more methods for deploying implementation models in this specification. Based on the same approach, this specification also provides corresponding devices for model deployment, such as... Figure 4 As shown.

[0101] Figure 4 A schematic diagram of a model deployment apparatus provided in this specification includes:

[0102] The first determining module 401 determines each network branch that needs to be replaced in the preset model, as each target network branch;

[0103] The second determining module 402 determines, based on the initial network parameters corresponding to each target network branch, the network parameters corresponding to the specified network branch after assuming that each target network branch is equivalently replaced with the specified network branch;

[0104] The processing module 403 updates the initial network parameters according to the network parameters corresponding to the specified network branch, and performs pseudo-quantization processing on the updated initial network parameters.

[0105] The deployment module 404 replaces each target network branch in the model after pseudo-quantization with a specified network branch, and deploys the model according to the replaced model.

[0106] Optionally, the device further includes:

[0107] Training module 405 is used to train the model after pseudo-quantization to obtain the optimized model;

[0108] The deployment module 404 is specifically used to deploy the optimized model.

[0109] Optionally, the deployment module 404 is further configured to quantize the network parameters corresponding to the specified network branch;

[0110] The deployment module 404 is specifically used to deploy the quantized model.

[0111] Optionally, the training module 405 is specifically used to: acquire input data; input the input data into the model after pseudo-quantization processing; determine the output result corresponding to the input data; and train the model after pseudo-quantization processing with the optimization objective of minimizing the deviation between the output result and the actual label corresponding to the input data to obtain an optimized model.

[0112] Optionally, the training module 405 is specifically used to: acquire initial input data; and perform pseudo-quantization processing on the initial input data to obtain the input data.

[0113] Optionally, the training module 405 is specifically used to: determine the data distribution corresponding to the input data based on the input data; perform regularization processing on the input data based on the data distribution; and determine the output result corresponding to the input data based on the processed input data.

[0114] Optionally, the training module 405 is further configured to: determine the data distribution corresponding to the historical input data as the historical data distribution; and update the historical data distribution according to the input data to obtain the updated data distribution;

[0115] The deployment module 404 is specifically used to deploy the optimized model according to the updated data distribution.

[0116] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 This provides a method for deploying a model.

[0117] This instruction manual also provides Figure 5 One of the corresponding Figure 1 A schematic diagram of the structure of an electronic device. (e.g.) Figure 5At the hardware level, the electronic device includes a processor, an internal bus, a network interface, memory, and non-volatile storage, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile storage into memory and then runs it to achieve the above-mentioned functions. Figure 1 The method for deploying the model described herein. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0118] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0119] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0120] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0121] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0122] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0126] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0127] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0128] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0129] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0130] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0132] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0133] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for deploying a model, characterized in that, The method is applied to the deployment of neural network models for processing image data, including: Identify the network branches that need to be replaced in the preset neural network model as target network branches, wherein the network branches include convolutional layers and batch regularization layers; Based on the initial network parameters corresponding to each target network branch, determine the network parameters corresponding to the specified network branch after assuming that each target network branch is equivalently replaced with the specified network branch. The initial network parameters are updated according to the network parameters corresponding to the specified network branch, and pseudo-quantization is performed on the updated initial network parameters. The method further includes replacing each target network branch in the pseudo-quantized model with a specified network branch, and deploying the model based on the replaced model. Before replacing each target network branch in the pseudo-quantized model with the specified network branch, the method also includes: The model after pseudo-quantization is trained to obtain the optimized model. Deploy the model based on the replaced model, specifically including: The optimized model is then deployed.

2. The method as described in claim 1, characterized in that, Before deploying the model based on the replaced model, the method further includes: The network parameters corresponding to the specified network branch are quantized. Deploy the model based on the replaced model, specifically including: The quantized model is then deployed.

3. The method as described in claim 1, characterized in that, The model after pseudo-quantization is trained to obtain the optimized model, which includes: Get the input data; The input data is input into the model after pseudo-quantization processing, and the output result corresponding to the input data is determined; The optimized model is obtained by training the model after pseudo-quantization with the optimization objective of minimizing the deviation between the output result and the actual label corresponding to the input data.

4. The method as described in claim 3, characterized in that, Obtaining input data specifically includes: Obtain initial input data; The initial input data is subjected to pseudo-quantization to obtain the input data.

5. The method as described in claim 3, characterized in that, The input data is input into the model after pseudo-quantization processing, and the output result corresponding to the input data is determined, specifically including: Based on the input data, determine the data distribution corresponding to the input data; Based on the data distribution, the input data is regularized, and based on the processed input data, the corresponding output result is determined.

6. The method as described in claim 3, characterized in that, The method further includes: Determine the data distribution corresponding to the historical input data, and use it as the historical data distribution; The historical data distribution is updated based on the input data to obtain the updated data distribution; Deploying the optimized model specifically includes: Based on the updated data distribution, the optimized model is deployed.

7. An apparatus for deploying a model, the apparatus being used for deploying a neural network model that processes image data, characterized in that, include: The first determining module determines each network branch that needs to be replaced in the preset neural network model as each target network branch. The network branch includes a convolutional layer and a batch regularization layer. The second determining module determines, based on the initial network parameters corresponding to each target network branch, the network parameters corresponding to the specified network branch after assuming that each target network branch is equivalently replaced with the specified network branch; The processing module updates the initial network parameters according to the network parameters corresponding to the specified network branch, and performs pseudo-quantization processing on the updated initial network parameters. The deployment module replaces each target network branch in the model after pseudo-quantization with a specified network branch, and deploys the model according to the replaced model. Before replacing each target network branch in the pseudo-quantized model with a specified network branch, the process also includes: The model after pseudo-quantization is trained to obtain the optimized model. Deploy the model based on the replaced model, specifically including: The optimized model is then deployed.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 6.

Images