Processing system and method of a neural network model

Abstract

The application relates to the technical field of neural networks, and discloses a processing system and method for a neural network model, which can reduce the consumption of the computing resources of a server device or a terminal device and can meet the requirements of different terminal devices for a model. The system comprises a server device, a first terminal device and a second terminal device, wherein: the server device adopts a first bit operator to train a to-be-trained model, and obtains a first target model; the server device adopts a second bit operator to train the first target model based on the first target model, and obtains a second target model, the second bit operator being higher than the first bit operator; the server device deploys the second target model on the first terminal device and the second terminal device respectively; the first terminal device runs the second target model; the second terminal device performs bit operator quantization on the second target model, and converts to obtain the first target model; and the second terminal device runs the first target model.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of neural network, in particular to a processing system and method of neural network model. BACKGROUND

[0002] With the development of semiconductor technology and software technology, various forms of terminal devices appear; for example, mobile phones, tablet computers, televisions, vehicle-mounted devices, and various household appliances. And based on the ecological concept of Internet of Everything, at present, by deploying neural network models in different terminal devices, more and more interactive functions can be realized between terminal devices, and thus intelligent functions can be realized to build an intelligent life.

[0003] Due to the performance difference of different terminal devices, different terminal devices can support inconsistent neural network models. At present, it is usually necessary to train multiple different target models, and then deploy multiple different target models on different terminal devices. Taking a speech recognition model as an example, due to the performance difference of smart screens, mobile phones and smart speakers, the currently commonly used implementation manner is: based on the computing power of smart screens, mobile phones and smart speakers respectively, three different versions of speech recognition models are trained respectively, and then are deployed on smart screens, mobile phones and smart speakers. Moreover, different neural network models also need to be deployed in different resource occupation scenarios of the same terminal device to adapt to different scenarios. In this way, in order to realize the deployment of appropriate neural network models on terminal devices, multiple models usually need to be trained and maintained, which will cause the consumption of computing resources and high cost. SUMMARY

[0004] Embodiments of the present application provide a processing system and method of neural network model, which can reduce the consumption of computing resources of a server device or a terminal device, and can meet the requirements of different terminal devices for models, thereby improving user experience.

[0005] In a first aspect, embodiments of the present application provide a processing system of neural network model. Optionally, the system includes a server device, a first terminal device and a second terminal device, the server device trains a to-be-trained model by using a first bit operator to obtain a first target model; the server device trains the first target model by using a second bit operator based on the first target model to obtain a second target model, the second bit operator is higher than the first bit operator; the server device deploys the second target model on the first terminal device and the second terminal device respectively; the first terminal device runs the second target model; the second terminal device performs bit operator quantization on the second target model to convert to obtain the first target model; and the second terminal device runs the first target model.

[0006] Another optional, the system includes a server device, a first terminal device and a second terminal device, the server device trains a to-be-trained model using a first bit operator to obtain a first target model; the server device trains the first target model based on the first target model using a second bit operator to obtain a second target model, the second bit operator is higher than the first bit operator; the server device deploys the second target model on the first terminal device; and the second target model is quantized by the bit operator, and the first target model is converted and deployed on the second terminal device; the first terminal device runs the second target model; and the second terminal device runs the first target model.

[0007] Through the method, by designing a model and training a plurality of different bit operators, a model with precision on a plurality of bit operators can be obtained, thereby meeting the needs of terminal devices with different performance. Compared with the implementation manner in the related art that different models need to be trained for terminal devices with different performance, the method can reduce the consumption of computing resources in the training process and save the cost in the training process. Moreover, for a certain terminal device, a low-bit model with precision can be obtained through quantization processing of a high-bit model, thereby meeting various scenarios with different terminal device resource occupancy rates. Compared with the implementation manner in the related art that a plurality of models need to be deployed, the method can reduce the occupation of storage resources.

[0008] In a possible design, the server device trains the first target model using a second bit operator based on the first target model, including: preloading a first model weight corresponding to the first target model; when training the first target model using the second bit operator, if a second model weight at the time of training does not satisfy a first preset mapping relationship with the first model weight, assigning the first model weight as the second model weight as an updated second model weight, and training based on the updated second model weight. The first preset mapping relationship can be understood as that the first model weight and the second model weight having a mapping relationship with the first model weight have consistent signs or corresponding intervals.

[0009] Through this design, the server device preloads the model weight of the target model corresponding to the lower bit operator which has been trained, and adjusts the model weight of the target model corresponding to the higher bit operator based on the model weight of the target model corresponding to the lower bit operator, which can improve the convergence speed of the model being trained.

[0010] In a possible design, the system further includes a third terminal device, and the server device trains the second target model by using a third bit operator to obtain a third target model based on the second target model, where the third bit operator is higher than the second bit operator; the server device deploys the third target model on the third terminal device; and the third terminal device runs the third target model.

[0011] Through this design, the number of target models of bit operators that need to be trained can be determined according to specific scenarios in actual implementation, so that the target models corresponding to the required bit operators can all have precision, thereby reducing the consumption of computing resources in the training process and saving costs in the training process.

[0012] In a possible design, the server device trains the second target model by using a third bit operator based on the second target model, including: preloading a first model weight corresponding to the first target model and a second model weight corresponding to the second target model; when the second target model is trained by using the third bit operator, if a third model weight during training does not satisfy a second preset mapping relationship with the first model weight, the second model weight is assigned to the third model weight as an updated third model weight; if the third model weight during training satisfies the second preset mapping relationship with the first model weight and the third model weight during training does not satisfy a third preset mapping relationship with the second model weight, the second model weight is assigned to the third model weight as the updated third model weight; and training is performed based on the updated third model weight. The second preset mapping relationship can be understood as that the first model weight and the third model weight having a mapping relationship with the first model weight have the same sign or correspond to the same interval, and the third preset mapping relationship can be understood as that the second model weight and the third model weight having a mapping relationship with the second model weight have the same sign or correspond to the same interval. In this possible design, it can be understood that the first model weight, the second model weight, and the third model weight are model weight values having a mapping relationship.

[0013] Through this design, when the target model corresponding to a higher bit operator is trained, if there are multiple target models corresponding to lower bit operators, the model weight of the target model corresponding to the lowest bit operator can be roughly judged first, and if a preset mapping condition is satisfied, the model weight of the target model with higher precision is judged one by one. In this way, the model weight is adjusted in a hierarchical manner, which can improve the efficiency of model training.

[0014] In a possible design, the first terminal device is further configured to quantize the second target model by using a bit operator based on resource occupation information to obtain the first target model.

[0015] In a possible design, if there are multiple target models with lower bits that can be converted for the terminal device, a suitable target model can be selected for conversion according to resource occupation information; for example, the third terminal device can perform bit operator quantization on the third target model based on the resource occupation information, to obtain the second target model or the first target model. It can be understood that the higher the resource occupation rate of the terminal device, the lower the bit operator of the target model selected by the terminal device for conversion.

[0016] With this design, a suitable target model can be converted in a timely manner based on the resource occupation information of the terminal device, so that the problem of freezing caused by a high resource occupation rate on the terminal device can be avoided, and user experience can be improved.

[0017] In a second aspect, an embodiment of the present application provides a processing method of a neural network model. Optionally, a service end device deploys a second target model on a second terminal device. The method includes: performing bit operator quantization on the second target model by the second terminal device to obtain a first target model; wherein the second target model is obtained by training the first target model by the service end device using a second bit operator; and running the first target model by the second terminal device.

[0018] Optionally, a service end device deploys a first target model on a second terminal device. The method includes: running the first target model by the second terminal device; wherein the first target model is obtained by performing bit operator quantization on a second target model; and the second target model is obtained by training the first target model by the service end device using a second bit operator.

[0019] In a third aspect, an embodiment of the present application provides a processing method of a neural network model. Optionally, the method includes: training a to-be-trained model by a service end device using a first bit operator to obtain a first target model; training the first target model by the service end device using a second bit operator to obtain a second target model, the second bit operator being higher than the first bit operator; and deploying the second target model on a first terminal device and a second terminal device by the service end device.

[0020] Optionally, the method further comprises: the server device training the to-be-trained model by using a first bit operator to obtain a first target model; the server device training the first target model by using a second bit operator based on the first target model to obtain a second target model, the second bit operator being higher than the first bit operator; the server device deploying the second target model on the first terminal device; and performing bit operator quantization on the second target model to convert the first target model, and deploying the first target model on the second terminal device.

[0021] In a possible design, the server device training the first target model by using a second bit operator based on the first target model comprises: preloading first model weights corresponding to the first target model; when training the first target model by using the second bit operator, if second model weights at a training time do not satisfy a first preset mapping relationship with the first model weights, assigning the first model weights to the second model weights as updated second model weights, and training based on the updated second model weights.

[0022] In a possible design, the method further comprises: the server device training the second target model by using a third bit operator based on the second target model to obtain a third target model, the third bit operator being higher than the second bit operator; and the server device deploying the third target model on the third terminal device.

[0023] In a possible design, the server device training the second target model by using a third bit operator based on the second target model comprises: preloading first model weights corresponding to the first target model and second model weights corresponding to the second target model; when training the second target model by using the third bit operator, if third model weights at a training time do not satisfy a second preset mapping relationship with the first model weights, assigning the second model weights to the third model weights as updated third model weights; if the third model weights at the training time satisfy the second preset mapping relationship with the first model weights and the third model weights at the training time do not satisfy a third preset mapping relationship with the second model weights, assigning the second model weights to the third model weights as the updated third model weights; and training based on the updated third model weights.

[0024] In a fourth aspect, the present application provides a terminal device, which comprises a plurality of functional modules; the plurality of functional modules interact to implement the method performed by the first terminal device or the second terminal device or the third terminal device in any of the above aspects and embodiments. The plurality of functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the plurality of functional modules can be combined or divided based on specific implementation.

[0025] In a fifth aspect, the present application provides a terminal device, which comprises at least one processor and at least one memory, and the at least one memory stores computer program instructions; when the terminal device is running, the at least one processor executes the method performed by the first terminal device or the second terminal device or the third terminal device in any of the above aspects and embodiments.

[0026] In a sixth aspect, the present application provides a server device, which comprises a plurality of functional modules; the plurality of functional modules interact to implement the method performed by the server device in any of the above aspects and embodiments. The plurality of functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the plurality of functional modules can be combined or divided based on specific implementation.

[0027] In a seventh aspect, the present application provides a server device, which comprises at least one processor and at least one memory, and the at least one memory stores computer program instructions; when the server device is running, the at least one processor executes the method performed by the server device in any of the above aspects and embodiments.

[0028] In an eighth aspect, the present application further provides a neural network model processing system, which comprises: at least two terminal devices (for example, the first terminal device and the second terminal device in any of the above aspects and embodiments) according to the fifth aspect, and a server device according to the seventh aspect.

[0029] In a ninth aspect, the present application provides a computer program product, which comprises: a computer program (also referred to as code or instructions), when the computer program is running, causing a computer to execute the method of the first terminal device or the second terminal device or the third terminal device or the server device in any of the above aspects and embodiments.

[0030] In a tenth aspect, the present application further provides a computer readable storage medium, which stores a computer program; when the computer program is executed by a computer, causing the computer to execute the method performed by the first terminal device or the second terminal device or the third terminal device or the server device in any of the above aspects and embodiments.

[0031] In a eleventh aspect, the present application also provides a chip for reading a computer program stored in a memory, and executing the method performed by the first terminal device or the second terminal device or the third terminal device or the server device according to any one of the first aspect and the implementation forms thereof.

[0032] In a twelfth aspect, the present application also provides a chip system comprising a processor for supporting a computer device to implement the method performed by the first terminal device or the second terminal device or the third terminal device or the server device according to any one of the first aspect and the implementation forms thereof. In a possible design, the chip system further comprises a memory for storing programs and data necessary for the computer device. The chip system can be composed of a chip, or can include a chip and other discrete devices.

[0033] The advantages of any one of the second aspect to the twelfth aspect above can refer to the advantages of the various possible designs of the first aspect above, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1a A schematic diagram of a scenario of deploying a neural network model;

[0035] Figure 1b A schematic diagram of another scenario of deploying a neural network model;

[0036] Figure 2 A schematic diagram of a possible hardware structure of a terminal device provided by an embodiment of the present application;

[0037] Figure 3 A software structure block diagram of a terminal device provided by an embodiment of the present application;

[0038] Figure 4 A schematic diagram of a scenario of a processing method of a neural network model provided by an embodiment of the present application;

[0039] Figure 5 A schematic diagram of an interaction flow of a processing method of a neural network model provided by an embodiment of the present application;

[0040] Figure 6 A schematic diagram of a flow of a processing method of a neural network model provided by an embodiment of the present application;

[0041] Figure 7 Another schematic diagram of a flow of a processing method of a neural network model provided by an embodiment of the present application;

[0042] Figure 8 Another schematic diagram of a scenario of a processing method of a neural network model provided by an embodiment of the present application;

[0043] Figure 9 FIG. 2 is a schematic diagram of an interaction flow of another embodiment of a method for processing a neural network model according to the present application. DETAILED DESCRIPTION

[0044] The technical solutions provided in the embodiments of the present application can be applied to the field of neural network technology. For example, the method provided in the embodiments of the present application can be applied to, but is not limited to, the fields of automatic speech recognition (ASR), computer vision (CV), natural language processing (NLP), optical character recognition (OCR), image processing, etc. In these fields, the artificial intelligence (AI) applications using neural network models are increasing year by year, and in actual applications, these AI applications need to be deployed on various terminal devices. Taking an ASR model as an example, the ASR model can be deployed on a smart screen, a mobile phone and a smart speaker, respectively, so that each terminal device can realize the function of voice interaction.

[0045] However, in combination with the content introduced in the background, due to the different performances of different terminal devices, different terminal devices can support different neural network models. At present, it is usually necessary to train multiple different target models, and then deploy the multiple different target models on different terminal devices, respectively.

[0046] One implementation in the related art is a multi-model training and multi-model deployment scheme, which mainly designs and trains different model structures for different terminal devices, and separately trains and deploys them. Figure 1aA schematic diagram of a scenario of deploying a neural network model is shown, taking a mobile phone, a smart screen and a smart speaker as an example. Since the mobile phone, the smart screen and the smart speaker have different performance, in order to simultaneously meet the more suitable model structure deployed on each terminal device, a separate training and separate deployment method is usually adopted, that is, training model 1 for the mobile phone and deploying model 1 on the mobile phone, training model 2 for the smart screen and deploying model 2 on the smart screen, and training model 3 for the smart speaker and deploying model 3 on the smart speaker. Among them, compared with the smart screen and the smart speaker, the mobile phone usually has better computing power, so the model 1 can have higher bit operators than the model 2 or the model 3, thereby achieving higher processing accuracy. The computing performance of the smart screen and the smart speaker also differs, so the bit operators of the model 2 and the model 3 are also different. However, through this implementation method, in the scenario of realizing the same or similar function on multiple terminal devices (such as voice recognition function), multiple models (such as multiple versions of ASR model) need to be designed, trained and maintained, so a large amount of time cost and labor cost is required, and a large amount of computing resources is wasted. Moreover, for a terminal device, due to the different usage or remaining conditions of the computing resources at different times, in order to adapt to different resource occupation conditions, there may be a need to deploy multiple models to select different models in different conditions to balance the processing effect and the optimal bit operator that can be supported, but in this scenario, the consumption of the storage resources of the terminal device is additionally increased.

[0047] Another implementation method existing in the related art is a single model training and multiple model deployment scheme, which mainly designs and trains a super network (that is, a large model). The large model can include multiple sub-models, each of which has accuracy in the training process. Moreover, each sub-model has different bit operators and can be suitable for different terminal devices, so that a suitable sub-model can be selected and deployed to the corresponding terminal device. Figure 1bAs another scenario of deploying a neural network model, still taking a mobile phone, a smart screen and a smart speaker as an example. First, an super network is trained and designed, which at least includes sub-model 1, sub-model 2 and sub-model 3. Based on the performance of the mobile phone, the smart screen and the smart speaker, sub-model 1 with higher bit operators is deployed on the mobile phone, sub-model 2 with general bit operators is deployed on the smart screen, and sub-model 3 with lower bit operators is deployed on the smart speaker. However, through this implementation manner, in order to ensure the accuracy of each sub-model, the obtained super network is generally very large, thereby causing difficulty in training the super network, and high demand for computing resources and long time consumption in the training process. In addition, from the perspective of the terminal device, similar to the multi-model training and multi-model deployment scheme, if a terminal device needs to deploy multiple different models under different resource occupancy conditions, multiple different sub-models also need to be deployed, thereby additionally increasing the consumption of storage resources of the terminal device and the like.

[0048] Therefore, embodiments of the present application provide a neural network model processing method, give a model weight of a low-bit model obtained by training, retrain a model weight of multiple high-bit models, and through low-bit quantization of the model weight of the high-bit model, a low-bit model can be obtained. In this way, on the one hand, the target model has accuracy under different model weights through training, thereby realizing deployment of a model corresponding to a model weight suitable for different terminal devices with different performance. On the other hand, after the model is deployed on the terminal device, the terminal device can switch the model running different model weights according to the resource usage of the device itself. Therefore, through the method provided by the embodiments of the present application, the consumption of computing resources of the server device or the terminal device can be reduced, and the requirements of different terminal devices for the model can be met, and in some scenarios, the occupation of storage resources can also be reduced, thereby improving the user experience.

[0049] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0050] It can be understood that the terminal device (for example, the first terminal device, the second terminal device, or the third terminal device) of the embodiments of the present application can be a device such as a smart home device (for example, a smart television, a smart screen, a smart speaker, and the like), a mobile phone, a tablet computer, a wearable device (for example, a watch, a helmet, a headset, and the like), an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and the like. It can be understood that the embodiments of the present application do not make any limitation on the specific type of the terminal device.

[0051] The terminal device to which the embodiments of the present application can be applied, exemplary embodiments include, but are not limited to, a portable terminal device equipped with The portable terminal device can also be another portable terminal device, such as a laptop computer (Laptop) having a touch-sensitive surface (for example, a touch panel).

[0052] Figure 2 A possible hardware structure schematic diagram of a terminal device is shown. The terminal device 200 includes radio frequency (RF) circuit 210, power supply 220, processor 230, memory 240, input unit 250, display unit 260, audio circuit 270, communication interface 280, and wireless-fidelity (Wi-Fi) module 290, and the like. Those skilled in the art can understand that the terminal device 200 shown in the above embodiment is only an example, and the terminal device 200 provided by the embodiments of the present application can include more or less components than the diagram, can combine two or more components, or can have a different component configuration. Figure 2 The hardware structure of the terminal device 200 shown in the above embodiment does not constitute a limitation on the terminal device 200, and the terminal device 200 provided by the embodiments of the present application can include more or less components than the diagram, can combine two or more components, or can have a different component configuration. Figure 2 The various components shown in the above embodiment can be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application specific integrated circuits.

[0053] The following will be described in combination with Figure 2 The specific introduction of each component of the terminal device 200 is as follows:

[0054] The RF circuit 210 can be used for receiving and sending data in the process of communication or conversation. In particular, the RF circuit 210 sends the downlink data received from the base station to the processor 230 for processing, and sends the uplink data to be sent to the base station. Generally, the RF circuit 210 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc.

[0055] In addition, the RF circuit 210 can also communicate with other devices through a wireless communication network. The wireless communication can use any communication standard or protocol, including but not limited to global system of mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc.

[0056] The Wi-Fi technology belongs to a short-range wireless transmission technology. The terminal device 200 can connect to an access point (AP) through the Wi-Fi module 290 to access a data network. The Wi-Fi module 290 can be used for receiving and sending data in the process of communication.

[0057] The terminal device 200 can be physically connected to other devices through the communication interface 280. Optionally, the communication interface 280 of the terminal device 200 and the communication interface of the other device are connected through a cable to realize data transmission between the terminal device 200 and the other device.

[0058] The terminal device 200 can also realize communication services and interact with service-side devices or other terminal devices, so the terminal device 200 needs to have a data transmission function, that is, the terminal device 200 needs to include a communication module. Although Figure 2 The RF circuit 210, the Wi-Fi module 290, and the communication interface 280 are shown as communication modules, but it can be understood that at least one of the above components or other communication modules (such as a Bluetooth module) for realizing communication exists in the terminal device 200 to realize data transmission.

[0059] For example, when the terminal device 200 is a mobile phone, the terminal device 200 can include the RF circuit 210, and can further include the Wi-Fi module 290, or can include a Bluetooth module (not shown in the figure) ; when the terminal device 200 is a computer, the terminal device 200 can include the communication interface 280, and can further include the Wi-Fi module 290, or can include a Bluetooth module (not shown in the figure) ; when the terminal device 200 is a tablet computer, the terminal device 200 can include the Wi-Fi module, or can include a Bluetooth module (not shown in the figure). Figure 2 Figure 2 Figure 2

[0060] The memory 240 can be used to store software programs and modules. The processor 230 can execute various function applications and data processing of the terminal device 200 by running the software programs and modules stored in the memory 240. Optionally, the memory 240 can mainly include a program storage area and a data storage area. The program storage area can store an operating system (mainly including respective software programs or modules corresponding to a kernel layer, a system layer, an application program framework layer and an application program layer, etc.).

[0061] In addition, the memory 240 can include a high-speed random access memory, and can further include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. In the embodiment of the present application, the related data information of the neural network model deployed on the terminal device 200 can be stored in the memory 240.

[0062] The input unit 250 can be used to receive the editing operation of various types of data objects such as digital or character information input by the user, and to generate key signal input related to the user settings and function control of the terminal device 200. Optionally, the input unit 250 can include a touch panel 251 and other input devices 252.

[0063] The touch panel 251, also known as a touch screen, can collect the touch operation of the user on or near it (such as the operation of the user using a finger, a stylus or any suitable object or accessory on or near the touch panel 251), and drive the corresponding connection device according to the pre-set program.

[0064] Optionally, the other input devices 252 can include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control button, a switch button, etc.), a trackball, a mouse, a joystick, etc.

[0065] ​​​The display unit 260 can be used to display information input by the user or information provided to the user, as well as various menus of the terminal device 200. The display unit 260 is the display system of the terminal device 200, used to present the interface and realize human-computer interaction. The display unit 260 may include a display panel 261. Optionally, the display panel 261 may be configured as a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0066] The processor 230 is the control center of the terminal device 200. It connects various components via various interfaces and lines, and executes software programs and / or modules stored in the memory 240, as well as calling data stored in the memory 240, to perform various functions and process data of the terminal device 200, thereby realizing multiple services based on the terminal device 200. In this embodiment, the processor 230 is used to implement the methods provided in this embodiment and to run a neural network model stored in the memory 240 to perform corresponding processing based on the neural network model. For example, if the neural network model is an ASR model, the terminal device 200 can perform speech recognition processing.

[0067] The terminal device 200 also includes a power supply 220 (such as a battery) for supplying power to various components. Optionally, the power supply 220 can be logically connected to the processor 230 through a power management system, thereby enabling the power management system to manage functions such as charging, discharging, and power consumption.

[0068] like Figure 2 As shown, the terminal device 200 also includes an audio circuit 270, a microphone 271, and a speaker 272, providing an audio interface between the user and the terminal device 200. The audio circuit 270 converts audio data into a signal recognizable by the speaker 272 and transmits the signal to the speaker 272, where it is converted into a sound signal for output. The microphone 271 collects external sound signals (such as human speech or other sounds) and converts the collected external sound signals into a signal recognizable by the audio circuit 270, sending it to the audio circuit 270. The audio circuit 270 can also convert the signal sent by the microphone 271 into audio data, outputting the audio data to the RF circuit 220 for transmission to, for example, another terminal device, or outputting the audio data to the memory 240 for further processing. In this embodiment, for example, audio data received through the microphone 271 can be used as input to an ASR model, thereby enabling subsequent processing based on the ASR model.

[0069] Although not shown, the terminal device 200 can also include at least one sensor, a camera, and the like, which will not be described here. The at least one sensor can include, but is not limited to, a pressure sensor, a barometric pressure sensor, an acceleration sensor, a distance sensor, a fingerprint sensor, a touch sensor, a temperature sensor, and the like.

[0070] The operating system (OS) involved in the embodiments of the present application is the most basic system software running on the terminal device 200. Taking a mobile phone as an example, the operating system can be a HarmonyOS or an Android system or an IOS system. The software system of the terminal device 200 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application take the operating system adopting a layered architecture as an example to exemplarily illustrate the software structure of the terminal device 200.

[0071] Figure 3 A software structure block diagram of a terminal device is provided in the embodiments of the present application. As shown in Figure 3 , the software structure of the terminal device can be a layered architecture, for example, the software can be divided into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the operating system is divided into five layers, from top to bottom, the application program layer, the application program framework layer (FWK), the runtime and system library, the kernel layer, and the hardware layer.

[0072] The application program layer can include a series of application program packages. As shown in Figure 3 , the application program layer can include a camera, settings, a skin module, a user interface (UI), a third-party application, and the like. Among them, the third-party application can include WLAN, music, calls, Bluetooth, video, and the like.

[0073] In a possible implementation manner, the application program can be developed using the java language, and is completed by calling the application programming interface (API) provided by the application program framework layer. The developer can interact with the bottom layer (such as the hardware layer, the kernel layer, and the like) of the operating system through the application program framework layer to develop his own application program. The application program framework layer is mainly a series of services and management systems of the operating system.

[0074] The application program framework layer provides the application programming interface and the programming framework for the application program of the application program layer. The application program framework layer includes some predefined functions. As shown in Figure 3As shown, the application framework layer can include a shortcut icon management module, a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.

[0075] The shortcut icon management module is used to manage the shortcut icons displayed on the terminal device, such as creating shortcut icons, removing shortcut icons, monitoring whether the shortcut icons meet the display conditions, etc.

[0076] The window manager is used to manage window programs. The window manager can acquire the size of the display screen, determine whether there is a status bar, lock the screen, intercept the screen, etc. The content provider is used to store and acquire data, and make the data accessible to the application programs. The data can include videos, images, audios, dialed and received calls, browsing history and bookmarks, phone books, etc.

[0077] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build application programs. A display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying pictures.

[0078] The phone manager is used to provide the communication functions of the terminal device. For example, the management of call states (including call connection, call hang-up, etc.).

[0079] The resource manager provides various resources for the application programs, such as localized strings, icons, pictures, layout files, video files, etc.

[0080] The notification manager enables the application programs to display notification information in the status bar, which can be used to convey messages of the notification type, which can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify the completion of downloading, message reminders, etc. The notification manager can also be a notification in the form of a chart or a scroll bar text appearing in the top status bar of the system, such as a notification of an application program running in the background, and can also be a notification in the form of a dialogue window appearing on the screen. For example, prompting text information in the status bar, issuing a prompt sound, terminal device vibration, indicator light blinking, etc.

[0081] The runtime includes a core library and a virtual machine. The runtime is responsible for the scheduling and management of the operating system.

[0082] The core library includes two parts: one part is the function functions required by the java language to call, and the other part is the core library of the operating system. The application program layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application program layer and the application framework layer into binary files. The virtual machine is used to perform the management of object life cycle, stack management, thread management, security and exception management, and garbage collection, etc.

[0083] The system library can include a plurality of functional modules. For example, a surface manager, media libraries, a three-dimensional graphics processing library (e.g., OpenGL ES), a 2D graphics engine (e.g., SGL), etc.

[0084] The surface manager is used to manage the display subsystem and provides fusion of 2D and 3D layers for a plurality of applications.

[0085] The media libraries support a plurality of commonly used audio, video format playback and recording, and static image files, etc. The media libraries can support a plurality of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0086] The three-dimensional graphics processing library is used to implement three-dimensional graphics drawing, image rendering, synthesis, and layer processing, etc.

[0087] The 2D graphics engine is a drawing engine for 2D drawing.

[0088] In some embodiments, the three-dimensional graphics processing library can be used to draw a three-dimensional motion trajectory image, and the 2D graphics engine can be used to draw a two-dimensional motion trajectory image.

[0089] The kernel layer is a layer between hardware and software. The kernel layer at least includes a display driver, a camera driver, an audio driver, and a sensor driver.

[0090] The hardware layer can include various sensors, such as an acceleration sensor, a gyroscope sensor, and a touch sensor, etc.

[0091] Generally, the terminal device 200 can simultaneously run a plurality of applications. More simply, one application can correspond to one process, and more complexly, one application can correspond to a plurality of processes. Each process has a process number (process ID).

[0092] In combination with the above Figure 2 introduction to the hardware structure of the terminal device, and Figure 3 the introduction to the software framework of the terminal device, the working principles of the software and hardware of the terminal device performing the processing method of the neural network model proposed in the embodiments of the present application are exemplarily described below in combination with a plurality of embodiments and the accompanying drawings.

[0093] It should be understood that "at least one" in the embodiments of the present application means one or more, and "multiple" means two or more. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or the like means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b or c can represent a, b, c, a and b, a and c, b and c, or a, b and c, wherein a, b and c can be single or multiple.

[0094] The multiple referred to in the embodiments of the present application means greater than or equal to two.

[0095] In addition, it should be understood that in the description of the present application, the words "first", "second", etc. are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor indicating or implying order.

[0096] In addition, "terminal device", "device" and the like can be used interchangeably in the embodiments of the present application, that is, various devices that can be used to implement the embodiments of the present application.

[0097] It should be understood that the hardware structure of the terminal device can be as shown in Figure 2 The software architecture can be as shown in Figure 3 The software program and / or module corresponding to the software architecture in the terminal device can be stored in the memory 240, and the processor 230 can run the software program and application stored in the memory 240 to execute the flow of the processing method of the neural network model provided in the embodiments of the present application.

[0098] In order to facilitate understanding of the processing method of the neural network model provided in the present application, the following will be combined with Figures 4 to 8The content shown describes the implementation process of the method provided in this application. For example, in the embodiments of this application, the training process of the neural network model can be performed on a server device or a terminal device with strong computing power. After the trained neural network model is deployed to the target terminal device, the target terminal device can perform further processing based on the neural network model, as well as switching between low-bit and high-bit models. In the embodiments of this application, "low-bit model" refers to the target model trained under the model weights of low-bit operators, and "high-bit model" refers to the target model trained under the model weights of high-bit model operators. Furthermore, "high-bit" and "low-bit" in the embodiments of this application are relative and are not intended to limit specific bit operators.

[0099] Figure 4 This is a schematic diagram illustrating a scenario for a neural network model processing method provided in an embodiment of this application. In the following embodiments, a server-side device is used as the execution entity for training the neural network model to illustrate the method provided in this application. First, the server-side device can train a first-bit model (which can also be understood as a low-bit model) according to the target business requirements and achieve the target accuracy. For example, the low-bit model can be... Figure 4 The diagram shows a 1-bit model. The server device can then increase the number of operator bits in the low-bit model to continue training the second-bit model and achieve the target accuracy. For example, the second-bit model could be... Figure 4 The diagram shows an 8-bit model (or a 4-bit model). Furthermore, the server device can further increase the number of operator bits in the second-bit model to continue training the third-bit model and achieve the target accuracy. For example, the third-bit model could be... Figure 4 The diagram shows a 32-bit model (or a 16-bit model). Similarly, if there is a need for other bit operator models, refer to the above training process for processing, which will not be repeated here. Thus, through the above training process, the trained target neural network model has accuracy under multiple bit operators, including 1-bit, 8-bit, and 32-bit models, which can meet the deployment requirements of neural network models on multiple terminal devices with different performance levels. For example, the target neural network model can be deployed on smart speakers, smart screens, and mobile phones to achieve the following: on a smart speaker, the target neural network model can be quantized and run as a 1-bit model; on a smart screen, the target neural network model can be quantized and run as an 8-bit model; and on a mobile phone, a 32-bit model can be run.

[0100] It should be noted that the service end device can determine the actual bits of each bit model to be trained and determine how many bit models to train according to actual target business requirements. For example, if the target business requirement needs to be deployed on a mobile phone and a smart screen, the mobile phone can support running a 32-bit model, and the smart screen can support running a 4-bit model. The service end device can start training from a 4-bit operator model and train a 32-bit operator to obtain a target model that has precision under both 4-bit and 32-bit operators.

[0101] Figure 5 An interactive flowchart of a processing method of a neural network model provided by an embodiment of the present application is shown. The specific process of the method can include:

[0102] Step 501: The service end device trains a to-be-trained (neural network) model using a first bit operator to obtain a first target (neural network) model.

[0103] For example, the service end device can determine which bit operator training needs to be performed on the to-be-trained model according to the target business requirement. For example, the service end device can perform 1-bit operator, 8-bit operator and 32-bit operator training, or can perform 4-bit operator and 16-bit operator training. The number and specific bit operators can be determined according to actual application scenarios in specific implementation, which is not limited in the present application. The first bit operator is the lowest bit operator, for example, the 1-bit operator in the foregoing example, or the 4-bit operator in the foregoing another example. In this way, by performing the training of the first bit operator, the to-be-trained model converges, and the model can have precision under the bit operator of the first bit operator to obtain the first target model, for example, the first target model can meet the requirement of running on some terminal devices with low performance. Taking the to-be-trained model as an ASR model for example, the first target model is a first target ASR model, which can be run on a smart speaker to obtain a first target ASR model to realize the functions of voice recognition and voice interaction of the smart speaker.

[0104] Step 502: The service end device trains the first target model using a second bit operator based on the first target model to obtain a second target model, the second bit operator being higher than the first bit operator.

[0105] Exemplarily, the first model weight corresponding to the first target model can be obtained when the first bit operator is used for training in step 501. Since the target models corresponding to different bit operators have different accuracies, but the model weights have a certain preset mapping relationship (such as consistent signs or interval correspondence), the server device can train the second target model by referring to the first model weight, thereby improving the convergence efficiency of the second target model. The higher the bit operator, the higher the accuracy of the target model corresponding to the training, and the higher the performance requirement corresponding to the running of the corresponding target model.

[0106] The model weight represents a model parameter applied in model training, and the model weight is continuously adjusted to train a better target neural network model. Each model training process can include one or more model weights, for example, each input value can correspond to different model weights. In the embodiment of the application, the "first model weight" can represent any model weight value used in the training process of the first target model, and the "second model weight" can represent the model weight value having a preset mapping relationship with the first model weight in the training process of the second target model. In the embodiment of the application, "first" and "second" are only used for distinguishing description; wherein the model weight values corresponding to the same input value under different target models have a certain preset mapping relationship. It should be noted that in the following embodiments, the model weight takes any model weight value involved in the training process of the target neural network model as an example, and does not limit the specific model weight corresponding to which input value.

[0107] In an optional embodiment, Figure 6 A flowchart of a processing method of a neural network model is provided in the embodiment of the application. The flowchart is used to illustrate the implementation process of training the first target model using the second bit operator based on the first target model, which includes:

[0108] In step 601, the server device preloads the first model weight corresponding to the 1-bit model (the first target model). It can be understood that by preloading the model weight information of the 1-bit model, the training efficiency of the 8-bit model can be improved.

[0109] In step 602, the server device trains the 8-bit model (the second target model).

[0110] In step 603, the server device judges whether the sign of the second model weight corresponding to the 8-bit model is consistent with the sign of the first model weight. In the implementation of the application, if it is judged that the signs are consistent, the step 602 can be returned to continue to be executed; if it is judged that the signs are inconsistent, the step 604 can be executed.

[0111] Step 604: The server device assigns the first model weight to the 8-bit model as the updated second model weight. For example, assuming the 1-bit model has a model weight of 1, during the training of the 8-bit model, if the corresponding model weight is -4.52, since the sign of 1 is inconsistent with the sign of -4.52, and considering that the 1-bit model has already been trained, the model weight 1 of the 1-bit model can be assigned to the 8-bit model, that is, -4.52 is updated to 1, to accelerate the training convergence of the 8-bit model.

[0112] In this application, after step 604, the process returns to step 602, that is, to continue training the 8-bit model until the 8-bit model converges and an 8-bit model with accuracy is obtained.

[0113] In another alternative embodiment, Figure 7 This is another flowchart illustrating a method for processing a neural network model provided in an embodiment of this application. The flowchart describes the implementation process where, based on the first target model and the second target model, a third bit operator can be used to further train the second target model, including:

[0114] Step 701: The server device preloads the second model weights corresponding to the 8-bit model (second target model). It can be understood that preloading the model weights of the 8-bit model can improve the training efficiency of the 32-bit model. Alternatively, the server device can also preload the model weights of the 1-bit model. However, the accuracy difference between the 32-bit and 8-bit models is smaller than the accuracy difference between the 32-bit and 1-bit models. Therefore, based on the model weights corresponding to the adjacent lower-bit operator models, the training efficiency and accuracy of the target model corresponding to the current bit operator can be better guaranteed.

[0115] Step 702a: The server-side device trains a 32-bit model (third target model).

[0116] Step 702b: The server device obtains the first model weight corresponding to the 1-bit model.

[0117] Step 703: The server device determines whether the sign of the third model weight corresponding to the 32-bit model is consistent with the sign of the first model weight. In this application, if the signs are consistent, step 704a can be continued. If the signs are inconsistent, the first model weight corresponding to the 1-bit model can be assigned to the 32-bit model. Figure 7 (not shown in the image) to accelerate the convergence of training the 32-bit model; or, to further improve the training efficiency of the 32-bit model, based on the smaller accuracy difference between the 32-bit model and the 8-bit model, step 704b can be continued.

[0118] Step 704a, the server device determines whether the third model weight corresponding to the 32-bit model is in the second model weight interval. In the implementation of the present application, if it is determined that it is, step 702a can be returned for execution; if it is determined that it is not, step 704b can be continued to execute.

[0119] In the implementation of the present application, the server device can directly determine through step 704a without first determining the sign through steps 702b and 703 (not shown in the figure). It can be understood that through the determination of steps 702b and 703, the training efficiency and speed of the 32-bit model training process can be improved. First, through steps 702b and 703, it is roughly determined whether there is a deviation in the training of the 32-bit model. If there is no deviation, it is further determined through step 704a whether there is a deviation in the training of the 32-bit model. The speed of the rough determination process is greater than the speed of the accurate determination process.

[0120] For example, assuming that the model weight of the 1-bit model is 1, and during the training process of the 32-bit model, the model weight corresponding to the 32-bit model is -5.02. Since the signs of 1 and -5.02 are inconsistent, based on the fact that the 1-bit model has been trained, it can be determined that there is a deviation in the training of the 32-bit model. Therefore, the model weight of the 1-bit model that has been trained can be assigned to the 32-bit model, or the model weight of the 8-bit model that has been trained and has higher accuracy can also be assigned to the 32-bit model to speed up the convergence of the training of the 32-bit model. Further, if the model weight corresponding to the 32-bit model is 5.02, it can be determined that the signs of 1 and 5.02 are consistent, and the model weight of the 32-bit model can be further accurately determined through the model weight of the 8-bit model. The model weight of the 8-bit model can usually be distributed in multiple intervals, and the model weight of the 32-bit model has a certain preset mapping relationship with the model weight of the 8-bit model. During normal training, the model weight of the 32-bit model can correspond to the preset interval of the 8-bit model based on the preset mapping relationship. In this way, if it is determined that the interval of the 32-bit model mapped to the 8-bit model is inconsistent with the preset interval, it can be determined that there is a deviation in the training of the 32-bit model. Therefore, the model weight of the 8-bit model that has been trained can be assigned to the 32-bit model. Assuming that the model weight of the 8-bit model is 4.52, the model weight of the 32-bit model is updated from 5.02 to 4.52.

[0121] Step 704b, the server device assigns the second model weight (the model weight corresponding to the 8-bit model) to the 32-bit model as the updated third model weight.

[0122] It should be noted that the embodiments shown in Figure 6 and Figure 7 are only possible examples provided by the embodiments of the present application. It can be understood that in actual scenarios, if a 1-bit model is taken as the lowest bit model, when training a higher bit model, the symbol consistency can be preliminarily judged based on the 1-bit model that has been trained, and the model weight in the training process is adjusted based on the judgment result; if a 4-bit model or an 8-bit model or the like is taken as the lowest bit model, when training a higher bit model, whether the intervals match can be judged based on the lower bit model that has been trained, and the model weight in the training process is adjusted based on the matching result. Moreover, in the process of training a higher bit model, the judgment in the model weight can be directly made according to the adjacent lower bit model, so as to ensure the accuracy of the training. For example, assuming that the target training model includes a 1-bit model, a 4-bit model, an 8-bit model and a 16-bit model, when training a 16-bit model, the model weight of the 16-bit model in the training process can be directly adjusted according to the model weight interval of the 8-bit model; or the model weight of the 16-bit model in the training process can be preliminarily judged according to the model weight of the 1-bit model or the 4-bit model, and if the judgment result is that the symbols are consistent or the intervals are consistent, a more accurate judgment is further made according to the model weight interval of the adjacent 8-bit model. In specific implementation, the implementation manner can be selected according to the performance of the server device and the like, which is not limited in the present application.

[0123] After the server device trains the target model, the target model corresponding to the highest bit obtained by the training can be deployed on different terminal devices, as shown in steps 503a to 505 in Figure 5 . In this way, the server device can deploy the target model corresponding to the highest bit obtained by the training on different terminal devices, and then each terminal device can perform quantization processing of the target level according to its own business requirements to obtain a target model suitable for each terminal device. Alternatively, the target model corresponding to the highest bit obtained by the training can be quantized to different levels according to the business requirements of different terminal devices, and then the target models corresponding to different bits obtained by the quantization can be deployed on different terminal devices respectively. In this way, each terminal device can directly perform subsequent processing of the target business according to the deployed target model. It can be understood that in the method provided by the present application, the quantization processing of the target model corresponding to the highest bit in the target business requirement scenario can be implemented on the server device or each terminal device, which is not limited in the present application. In the following embodiments, the quantization processing is implemented on each terminal device as an example.

[0124] Step 503a, the server device deploys the second target model on the first terminal device.

[0125] Step 503b, the server device deploys the second target model on the second terminal device. Wherein, the second target model is, for example, the 32-bit model shown in the following table 1, that is, the highest bit model in the plurality of different bit models trained by the server device for the target service requirement scenario. Figure 4

[0126] It should be noted that the execution order between step 503a and step 504b is not limited during implementation of the present application, and can be determined according to actual business requirements.

[0127] Step 504a, the first terminal device runs the second target model. For example, taking a mobile phone in the target service requirement scenario as an example, the first terminal device has better performance, so it can support running the second target model, and thus can meet higher precision requirements, thereby meeting more business functions. Figure 4

[0128] Step 504b, the second terminal device quantizes the bit operators of the second target model to obtain the first target model. Wherein, quantizing the bit operators of the second target model can be implemented by mapping M intervals corresponding to high bits into N intervals corresponding to low bits, where M is greater than N, and M and N are positive integers. It can be understood that the mapping method from a high-bit operator model to a low-bit operator model can be applied to the method provided by the present application, such as linear mapping, custom mapping, etc. The specific mapping method is not limited by the present application. For ease of understanding, the following two examples are introduced to introduce the quantization processing method of the second target model.

[0129] In one possible example, taking the second target model as an 8-bit model and the first target model as a 1-bit model as an example, the quantization processing of the 8-bit model can refer to the following formula 1:

[0130]

[0131] Wherein, x represents the model weight or activation value corresponding to the 8-bit model, and y represents the model weight or activation value corresponding to the 1-bit model. For example, the model weight corresponding to the 1-bit model includes +1 and -1, and the model weight corresponding to the 8-bit model can be mapped to +1 or -1 according to the sign through formula 1.

[0132] In another possible example, taking the second target model as a 32-bit model and the first target model as an 8-bit model as an example, the quantization processing of the 32-bit model can refer to the following formulas 2-1 to 2-3:

[0133] ​​

[0134]

[0135] Z = Q max -R max ÷S Formula 2-3

[0136] In the formula 2-1, Q can represent the model weight or activation value of the 8-bit model, R can represent the model weight or activation value of the 32-bit model, S can represent the minimum interval value corresponding to the 8-bit model, and Z can represent the value corresponding to 0 in the 8-bit model in the 32-bit model. In the formula 2-2, R max may represent the maximum value in the 32-bit model, R min may represent the minimum value in the 32-bit model; Q max may represent the maximum value in the 8-bit model, Q min may represent the minimum value in the 32-bit model.

[0137] It can be understood that if the second target model and the first target model are conversions between other bit operators, the conversion process can also refer to the implementation mode shown in the above formulas 2-1 to 2-3, and the present application will not be repeated.

[0138] In addition, if the model weight or activation value of the second target model and the first target model exceeds the maximum range that can be represented respectively, the present application can be truncated according to the following formula 3 during implementation:

[0139] clamp(r; a, b) = min(max(x, a), b) Formula 3

[0140] In the formula 3, a represents the minimum value of the model weight or activation value of the bit model, and b represents the maximum value of the model weight or activation value of the bit model; r can represent the corresponding bit operator; x represents the model weight or activation value involved in the model training process. clamp represents the value after the input value x is truncated; for example, assuming that the model weight of the 8-bit model corresponds to b is 4.25, and a is -3.23, if there is x1 = 5 in the training process, x1 is updated to 4.25 through the truncation processing in the formula 3; or, if there is x2 = -4 in the training process, x2 is updated to -3.23 through the truncation processing in the formula 3. Wherein, x1, x2 are different input values of x.

[0141] Step 505, the second terminal device runs the first target model. For example, the second terminal device is a terminal device with poor performance and price in the target business demand scene (for example Figure 4As an example, the smart screen or smart speaker in the smart home can be taken as an example. Based on the fact that the second terminal device has low performance, even if the second terminal device can deploy the second target model, the system of the second terminal device cannot well support the running of the second target model, and there can be problems such as lag, thereby causing poor user experience. Therefore, the second terminal device can quantize the second target model according to the device performance, thereby obtaining a model with a lower bit operator, and further avoiding problems such as lag.

[0142] In another optional embodiment, each terminal device can also perform bit operator conversion on the running target model according to the resource occupancy rate and the like of the terminal device. Figure 8 Another scenario of the processing method of the neural network model provided in the embodiments of the present application is shown. Taking a terminal device as a mobile phone and a target neural network model as a 32-bit model as an example, if the resource occupancy rate of the mobile phone is low, the 32-bit model deployed on the mobile phone can be run, and further, the model with a higher bit operator can be run under the condition that the performance of the mobile phone can meet the requirement, thereby achieving a better accuracy requirement. If the resource occupancy rate of the mobile phone is medium (greater than the resource occupancy rate corresponding to the running of the 32-bit model), the 32-bit model deployed can be quantized and converted into an 8-bit model, and further, the 8-bit model can be run on the mobile phone. Further, if the resource occupancy rate of the mobile phone is high (greater than the resource occupancy rate corresponding to the running of the 8-bit model), at this time, the remaining available resources of the mobile phone are less, and if the model with a higher bit operator is run, problems such as lag can occur, therefore, the 32-bit model deployed can be quantized (or the quantized 8-bit model is quantized again), and converted into a 1-bit model, and further, the 1-bit model can be run on the mobile phone, thereby avoiding the problem of lag and improving user experience.

[0143] In another optional embodiment, in the case that the computing capability of the terminal device is poor and cannot support the processing of switching the target model with different bit operators, or the target model with different bit operators does not need to be switched, the bit quantum quantization of the target model can also be performed by the server device. Figure 9 Another interaction flow diagram of the processing method of the neural network model provided in the embodiments of the present application is shown. In the interaction flow diagram, the difference from the interaction flow diagram shown in FIG. 5b is that: Figure 5 The difference between the interaction flow diagram shown in FIG. 5b and the interaction flow diagram shown in FIG. 5b' is that:

[0144] Step 503b': quantizing the second target model by bit operators to obtain the first target model.

[0145] Step 504b': the server device deploys the first target model on the second terminal device. In this way, the bit operator quantization of the target model by the server device can reduce the requirement for the computing capability of the terminal device and save computing resources.

[0146] The method provided in the present application can obtain a model with precision on various bit operators by designing a model and training various different bit operators, so as to meet the needs of terminal devices with different performances. Compared with the implementation manner in the related art that different models need to be trained for terminal devices with different performances, the method can reduce the consumption of computing resources in the training process and save the cost in the training process. Moreover, for a certain terminal device, a low-bit model with precision can be obtained through quantization processing of a high-bit model, so as to meet various scenes with different resource occupancy rates of terminal devices. Compared with the implementation manner in the related art that multiple models need to be deployed, the method can reduce the occupation of storage resources.

[0147] Based on the above embodiments, the present application further provides a terminal device, which comprises a plurality of functional modules; the plurality of functional modules interact to realize the functions performed by the first terminal device or the second terminal device in the methods described in the embodiments of the present application. For example, the terminal device performs the step 504a in the embodiment shown in Figure 5 , or performs the step 504b and 505 in the embodiment shown in Figure 5 , or performs the step 505 in the embodiment shown in Figure 9 . The plurality of functional modules can be realized based on software, hardware or a combination of software and hardware, and the plurality of functional modules can be arbitrarily combined or divided based on specific implementation.

[0148] Based on the above embodiments, the present application further provides a terminal device, which comprises at least one processor and at least one memory, and the at least one memory stores computer program instructions; when the terminal device is running, the at least one processor performs the functions performed by the terminal device in the methods described in the embodiments of the present application. For example, the terminal device performs the step 504a in the embodiment shown in Figure 5 , or performs the step 504b and 505 in the embodiment shown in Figure 5 , or performs the step 505 in the embodiment shown in Figure 9 .

[0149] Based on the above embodiments, the present application further provides a server device, which comprises a plurality of functional modules; the plurality of functional modules interact to realize the functions performed by the server device in the methods described in the embodiments of the present application. For example, the server device performs the steps 501 to 503b in the embodiment shown in Figure 5 , or performs the steps 501 to 503b in the embodiment shown in Figure 9The steps 501 to 504b' are executed by the server device in the illustrated embodiment. The plurality of functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the plurality of functional modules can be combined or divided based on specific implementation.

[0150] Based on the above embodiments, the present application further provides a server device, which includes at least one processor and at least one memory. The at least one memory stores computer program instructions. When the server device is running, the at least one processor executes the functions of the server device in the methods described in the embodiments of the present application. For example, the at least one processor executes the functions of the server device in the steps 501 to 504b' in the illustrated embodiment. Figure 5 The steps 501 to 503b are executed by the server device in the illustrated embodiment, or the steps 501 to 504b' are executed by the server device in the illustrated embodiment. Figure 9 The steps 501 to 504b' are executed by the server device in the illustrated embodiment.

[0151] Based on the above embodiments, the present application further provides a processing system of a neural network model, which includes at least two terminal devices and a server device. For example, the at least two terminal devices can be the first terminal device and the second terminal device in the above embodiments, and the server device can be the server device in the above embodiments.

[0152] Based on the above embodiments, the present application further provides a computer program product, which includes a computer program (also referred to as code or instructions). When the computer program is executed, the computer program causes a computer to execute the methods described in the embodiments of the present application.

[0153] Based on the above embodiments, the present application further provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a computer, the computer program causes the computer to execute the methods described in the embodiments of the present application.

[0154] Based on the above embodiments, the present application further provides a chip, which is used to read a computer program stored in a memory and implement the methods described in the embodiments of the present application.

[0155] Based on the above embodiments, the present application provides a chip system, which comprises a processor for supporting a computer device to implement the methods described in the embodiments of the present application. In a possible design, the chip system further comprises a memory for storing the necessary programs and data of the computer device. The chip system can be composed of a chip, or can include a chip and other discrete devices. Those skilled in the art understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0156] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0157] These computer program instructions can also be stored in a computer readable memory that can direct the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a product including instruction means, which implements the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0158] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce a computer implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0159] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A processing system of a neural network model, characterized by, The system comprises a server device, a first terminal device and a second terminal device, wherein: The server device trains a to-be-trained model by using a first bit operator to obtain a first target model; The server device trains the first target model by using a second bit operator based on the first target model, to obtain a second target model, the second bit operator being higher than the first bit operator; The server device deploys the second target model on the first terminal device and the second terminal device respectively; The first terminal device runs the second target model; The second terminal device performs bit operator quantization on the second target model to convert the first target model; and the second terminal device runs the first target model; The server device trains the first target model by using a second bit operator based on the first target model, comprising: Preloading a first model weight corresponding to the first target model; When training the first target model by using the second bit operator, if a second model weight at the time of training and the first model weight do not satisfy a first preset mapping relationship, assigning the first model weight to the second model weight as an updated second model weight, and training based on the updated second model weight.

2. The system of claim 1, wherein, The system further comprises a third terminal device, wherein: The server device trains the second target model by using a third bit operator based on the second target model to obtain a third target model, the third bit operator being higher than the second bit operator; The server device deploys the third target model on the third terminal device; The third terminal device runs the third target model.

3. The system of claim 2, wherein, The server device trains the second target model by using a third bit operator based on the second target model, comprising: Preloading a first model weight corresponding to the first target model and a second model weight corresponding to the second target model; When training the second target model by using the third bit operator, if a third model weight at the time of training and the first model weight do not satisfy a second preset mapping relationship, assigning the second model weight to the third model weight as an updated third model weight; if the third model weight at the time of training and the first model weight satisfy the second preset mapping relationship, and the third model weight at the time of training and the second model weight do not satisfy a third preset mapping relationship, assigning the second model weight to the third model weight as an updated third model weight; Training based on the updated third model weight.

4. The system of claim 1, wherein, The first terminal device is further configured to perform bit operator quantization on the second target model based on resource occupation information to convert the first target model.

5. A processing system of a neural network model, characterized by, The system comprises a server device, a first terminal device and a second terminal device, wherein: The server device trains a to-be-trained model by using a first bit operator to obtain a first target model; The server-side device trains the first target model by using a second bit operator based on the first target model, to obtain a second target model, the second bit operator being higher than the first bit operator; The server-side device deploys the second target model on the first terminal device, and quantizes the second target model by bit operators to convert the first target model, and deploys the first target model on the second terminal device; The first terminal device runs the second target model; The second terminal device runs the first target model; The server-side device trains the first target model by using a second bit operator based on the first target model, including: Preloading a first model weight corresponding to the first target model; When training the first target model by using the second bit operator, if a second model weight at a training time does not satisfy a first preset mapping relationship with the first model weight, the first model weight is assigned as the second model weight as an updated second model weight, and training is performed based on the updated second model weight.

6. The system of claim 5, wherein, The system further includes a third terminal device, and wherein: The server-side device trains the second target model by using a third bit operator based on the second target model, to obtain a third target model, the third bit operator being higher than the second bit operator; The server-side device deploys the third target model on the third terminal device; The third terminal device runs the third target model.

7. The system of claim 6, wherein, The server-side device trains the second target model by using a third bit operator based on the second target model, including: Preloading a first model weight corresponding to the first target model and a second model weight corresponding to the second target model; When training the second target model by using the third bit operator, if a third model weight at a training time does not satisfy a second preset mapping relationship with the first model weight, the second model weight is assigned to the third model weight as an updated third model weight; if the third model weight at the training time satisfies the second preset mapping relationship with the first model weight and the third model weight at the training time does not satisfy a third preset mapping relationship with the second model weight, the second model weight is assigned to the third model weight as the updated third model weight; Training is performed based on the updated third model weight.

8. The system of claim 5, wherein, The first terminal device is further configured to quantize the second target model by bit operators based on resource occupation information, to convert the first target model.

9. A processing method of a neural network model, comprising: The server-side device deploys the second target model on the second terminal device, including: The second terminal device performs bit operator quantization on the second target model to obtain a first target model, where the bit operator quantization is used to indicate a mapping from a bit interval of the second target model to a bit interval of the first target model; the second target model is obtained by training the first target model based on the first target model by using a second bit operator; and the training of the first target model based on the first target model by using the second bit operator includes: preloading a first model weight corresponding to the first target model; and if a second model weight during the training does not satisfy a first preset mapping relationship with the first model weight, assigning the first model weight to the second model weight as an updated second model weight, and training based on the updated second model weight. The second terminal device runs the first target model.

10. A processing method of a neural network model, comprising: The service end device deploys the first target model on the second terminal device, including: The second terminal device runs the first target model. The first target model is obtained by performing bit operator quantization on a second target model and then converting the second target model; the second target model is obtained by training the first target model based on the first target model by using a second bit operator; and the training of the first target model based on the first target model by using the second bit operator includes: preloading a first model weight corresponding to the first target model; and if a second model weight during the training does not satisfy a first preset mapping relationship with the first model weight, assigning the first model weight to the second model weight as an updated second model weight, and training based on the updated second model weight.

11. A processing method of a neural network model, comprising: including: The service end device trains a to-be-trained model by using a first bit operator to obtain a first target model; The service end device trains the first target model based on the first target model by using a second bit operator to obtain a second target model, where the second bit operator is higher than the first bit operator; The service end device deploys the second target model on a first terminal device and a second terminal device respectively; The training of the first target model based on the first target model by using the second bit operator includes: Preloading a first model weight corresponding to the first target model; If a second model weight during the training does not satisfy a first preset mapping relationship with the first model weight, assigning the first model weight to the second model weight as an updated second model weight, and training based on the updated second model weight.

12. The method of claim 11, wherein, The method further includes: The server-side device trains the second target model by using a third bit operator based on the second target model, to obtain a third target model, the third bit operator being higher than the second bit operator; The server-side device deploys the third target model on a third terminal device.

13. The method of claim 12, wherein, The server-side device trains the second target model by using a third bit operator based on the second target model, including: Preloading a first model weight corresponding to the first target model and a second model weight corresponding to the second target model; When training the second target model by using the third bit operator, if a third model weight at a training time does not satisfy a second preset mapping relationship with the first model weight, assigning the second model weight to the third model weight as an updated third model weight; if the third model weight at the training time satisfies the second preset mapping relationship with the first model weight and the third model weight at the training time does not satisfy a third preset mapping relationship with the second model weight, assigning the second model weight to the third model weight as the updated third model weight; Training based on the updated third model weight.

14. A processing method of a neural network model, comprising: Including: The server-side device trains a to-be-trained model by using a first bit operator, to obtain a first target model; The server-side device trains the first target model by using a second bit operator based on the first target model, to obtain a second target model, the second bit operator being higher than the first bit operator; The server-side device deploys the second target model on a first terminal device, and performs bit operator quantization on the second target model, to convert the first target model, and deploys the first target model on a second terminal device; The server-side device trains the first target model by using a second bit operator based on the first target model, including: Preloading a first model weight corresponding to the first target model; When training the first target model by using the second bit operator, if a second model weight at a training time does not satisfy a first preset mapping relationship with the first model weight, assigning the first model weight to the second model weight as an updated second model weight, and training based on the updated second model weight.

15. The method of claim 14, wherein, The method further includes: The server-side device trains the second target model by using a third bit operator based on the second target model, to obtain a third target model, the third bit operator being higher than the second bit operator; The server-side device deploys the third target model on a third terminal device.

16. The method of claim 15, wherein, The server-side device trains the second target model by using a third bit operator based on the second target model, including: Preloading a first model weight corresponding to the first target model and a second model weight corresponding to the second target model; In the training of the second target model by the third bit operator, if the third model weight in the training does not satisfy the second preset mapping relationship with the first model weight, the second model weight is assigned to the third model weight as an updated third model weight; if the third model weight in the training satisfies the second preset mapping relationship with the first model weight and the third model weight in the training does not satisfy the third preset mapping relationship with the second model weight, the second model weight is assigned to the third model weight as an updated third model weight. Training is performed based on the updated third model weight.

17. A terminal device, comprising: The computer program product comprises a computer readable storage medium storing program code instructions executable by at least one processor coupled to at least one memory to perform the method of any one of claims 9 to 16.

18. A server device, comprising: The computer program product comprises a computer readable storage medium storing program code instructions executable by at least one processor coupled to at least one memory to perform the method of any one of claims 9 to 16.

19. A processing system of a neural network model, characterized by, The computer program product comprises a computer readable storage medium storing program code instructions executable by at least one processor coupled to at least one memory to perform the method of any one of claims 9 to 16. The computer program product comprises a computer readable storage medium storing program code instructions executable by at least one processor coupled to at least one memory to perform the method of any one of claims 9 to 16.

20. A computer-readable storage medium, characterized in that, The computer program product comprises a computer readable storage medium storing program code instructions executable by at least one processor coupled to at least one memory to perform the method of any one of claims 9 to 16.

21. A computer program product comprising instructions, wherein: ​

Images