Embedded system model optimization method and device and computer equipment

Abstract

The invention discloses an optimization method and device of an embedded system model and computer equipment. The method comprises the steps that a state function of equipment used for operating a neural network model is acquired, and the state function is used for representing the current load state of the equipment; determining a pruning proportion and a quantization bit required by the neural network model according to the state function; and according to a preset optimization target and the load condition of the equipment, the pruning proportion and the quantization digit are optimized, optimized neural network model configuration is obtained, and the neural network model configuration at least comprises the pruning proportion and the quantization digit. The technical problem of poor model stability caused by single model pruning and quantification strategy of the embedded system model in related technologies is at least solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to an optimization method, device and computer equipment for an embedded system model. Background Art

[0002] Running deep learning models on embedded systems, especially in resource-constrained environments such as the Internet of Things (IoT), drones, smart homes, and edge computing, faces huge challenges. Devices in these environments are usually characterized by low power consumption, limited memory, and computing power, while deep learning models often have a large number of parameters and complex operation structures, which directly makes the deployment and operation of models on embedded devices extremely difficult, inefficient, and energy-intensive. To solve this problem, traditional optimization methods mainly rely on model pruning and quantization, that is, reducing the computational complexity and memory usage of the model by reducing the number of model parameters and reducing the precision of the parameters. Model pruning reduces redundancy by removing connections or weights with smaller contributions in the model, while quantization converts floating-point weights into low-precision integers to reduce storage requirements and improve computational efficiency. However, most traditional pruning and quantization methods adopt a static optimization strategy, that is, a one-time optimization is performed before the model is deployed, and a fixed pruning ratio and quantization accuracy are set. Although this method can reduce the resource requirements of the model to a certain extent, it cannot be flexibly adjusted according to the real-time operating status of the device. In practical applications, the load conditions, power consumption limits, and memory usage of embedded devices are often in dynamic change. Static optimization strategies are difficult to adapt to such changes, resulting in over-optimization (such as insufficient accuracy at high loads) or under-optimization (such as waste of resources at low loads) in some scenarios, affecting the real-time performance of the model and the energy efficiency of the device. Summary of the invention

[0003] The embodiments of the present application provide an optimization method, apparatus and computer device for an embedded system model, so as to at least solve the technical problem in the related art that the model stability is poor due to the single model pruning and quantization strategy of the embedded system model.

[0004] According to one aspect of an embodiment of the present application, there is provided an optimization method for an embedded system model, comprising: obtaining a state function of a device for running a neural network model, the state function being used to represent a current load state of the device; determining a pruning ratio and a number of quantization bits required for the neural network model according to the state function; optimizing the pruning ratio and the number of quantization bits according to a preset optimization target and the load condition of the device to obtain an optimized neural network model configuration, the neural network model configuration comprising at least: the pruning ratio and the number of quantization bits.

[0005] Optionally, determining the pruning ratio and the number of quantization bits required for the neural network model according to the state function includes: obtaining the maximum value of the pruning ratio and the minimum value of the pruning ratio; determining the pruning ratio required for the neural network model based on the maximum value of the pruning ratio, the minimum value of the pruning ratio and the state function, wherein the state function is the ratio of the current load of the device to the maximum load of the device; determining the number of quantization bits required for the neural network model according to the state function.

[0006] Optionally, determining the number of quantization bits required for the neural network model according to the state function includes: obtaining a device state value corresponding to the state function, the device state value including at least one of the following: memory occupancy, task complexity, and power consumption; when the device state value is in a first interval, determining the number of quantization bits to be a first preset value; when the device state value is in a second interval, determining the number of quantization bits to be a second preset value; when the device state value is in a third interval, determining the number of quantization bits to be a third preset value, wherein the first preset value, the second preset value, and the third preset value increase successively, the maximum value of the first interval is less than the minimum value of the second interval, and the maximum value of the second interval is less than the minimum value of the third interval.

[0007] Optionally, the method also includes: obtaining multiple structural weights in the neural network model and the loss function of the neural network model; determining the importance of each structural weight based on each structural weight in the neural network model and the loss function of the neural network model; obtaining the total number of the multiple structural weights; determining the number of structural weights to be eliminated based on the total number and the pruning ratio; eliminating a target number of structural weights from the multiple structural weights, the importance of the target number of structural weights being less than any one of the multiple structural weights except the target number of structural weights.

[0008] Optionally, the method also includes: obtaining multiple network layers in the neural network model; when the network layer belongs to a preset key layer, quantizing the structural weights in the network layer according to a preset number of quantization bits; when the network layer does not belong to a preset key layer, quantizing the structural weights in the network layer according to the number of quantization bits required by the neural network model.

[0009] Optionally, the pruning ratio and the number of quantization bits are optimized according to a preset optimization target and the load status of the device to obtain an optimized neural network model configuration, including: obtaining an objective function corresponding to the preset optimization target, the objective function being used to optimize the neural network model configuration according to the goal of minimizing the error after pruning and quantization operations on the neural network model and minimizing the inference time of the neural network model to obtain the optimized neural network model configuration; obtaining the current load condition of the device, and updating the optimized neural network model configuration according to the current load condition of the device.

[0010] Optionally, obtaining an objective function corresponding to the preset optimization target includes: obtaining a pruning error after a pruning operation is performed on the neural network model and a quantization error after a quantization operation is performed on the neural network model; obtaining the inference time and a balance parameter, the balance parameter being used to balance the weight between the pruning error and the quantization error and the inference time; determining the objective function based on the pruning error, the quantization error, the inference time and the balance parameter.

[0011] According to another aspect of an embodiment of the present application, there is also provided an optimization device for an embedded system model, including: an acquisition module, used to acquire a state function of a device used to run a neural network model, wherein the state function is used to represent a current load state of the device; a determination module, used to determine a pruning ratio and a number of quantization bits required for the neural network model according to the state function; and an optimization module, used to optimize the pruning ratio and the number of quantization bits according to a preset optimization target and the load condition of the device to obtain an optimized neural network model configuration, wherein the neural network model configuration includes at least: the pruning ratio and the number of quantization bits.

[0012] According to another aspect of the embodiment of the present application, a computer device is provided, including: a memory and a processor, wherein the memory is used to store program instructions; and the processor is connected to the memory and is used to execute the above-mentioned embedded system model optimization method.

[0013] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned embedded system model optimization method by running the computer program.

[0014] According to another aspect of the embodiments of the present application, a computer program product is provided, including computer instructions, which implement the above-mentioned embedded system model optimization method when executed by a processor.

[0015] In an embodiment of the present application, a state function of a device for running a neural network model is obtained, the state function being used to represent the current load state of the device; the pruning ratio and the number of quantization bits required for the neural network model are determined according to the state function; the pruning ratio and the number of quantization bits are optimized according to a preset optimization goal and the load condition of the device to obtain an optimized neural network model configuration, the neural network model configuration comprising at least: a pruning ratio and the number of quantization bits, thereby achieving the purpose of adjusting the pruning ratio and the number of quantization bits of the neural network model according to the operating state of the device, thereby achieving the technical effect of flexibly determining the pruning and quantization strategies of the model, thereby solving the technical problem in the related art that the model stability is poor due to the single model pruning and quantization strategy of the embedded system model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 It is a hardware structure block diagram of a computer terminal for implementing an optimization method for an embedded system model according to an embodiment of the present application;

[0018] Figure 2 is a flow chart of an optimization method for an embedded system model according to an embodiment of the present application;

[0019] Figure 3 is a flow chart of a method for optimizing a visual model according to an embodiment of the present application;

[0020] Figure 4 It is a structural diagram of an optimization device for an embedded system model according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] The information collected in the embodiments of the present application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or reject automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered.

[0024] In order to facilitate those skilled in the art to better understand the embodiments of the present application, some technical terms or nouns involved in the embodiments of the present application are explained as follows:

[0025] 1. Embedded System: A special-purpose computer system embedded in a larger device with specific functions and limited resources, such as low power consumption, limited storage and computing power. It is widely used in the Internet of Things, drones, smart homes and other fields.

[0026] 2. Model pruning: By reducing redundant parameters or neuron connections in deep learning models, the model structure is optimized, computational complexity and storage usage are reduced, while keeping the model performance as stable as possible.

[0027] 3. Quantization, converting model parameters and calculations from high-precision floating-point formats (such as FP32) to low-precision integer formats (such as INT8 or INT4) to reduce memory requirements and improve computing efficiency.

[0028] 4. Dynamic Optimization: A real-time optimization strategy that dynamically adjusts model pruning and quantization strategies based on the operating status of the embedded device (such as load, power consumption, and memory usage) to achieve a balance between performance and resource utilization.

[0029] 5. Sparsity refers to the small proportion of non-zero parameters in the neural network. By improving the sparsity of the model, the amount of computation and storage requirements can be further reduced.

[0030] 6. Pruning Threshold: sets the lower limit of the parameter value. Model weights below this value will be pruned to control the complexity of the model.

[0031] 7. Dynamic Quantization Ratio: adjust the quantization accuracy (such as the ratio of FP16 to INT8) according to the real-time scenario requirements to achieve the best performance and accuracy balance under different load conditions.

[0032] 8. Memory Reuse: By optimizing memory allocation and recycling strategies, memory resources can be reused during operation to adapt to the limited storage resources of embedded devices.

[0033] 9. Adaptive Pruning: A pruning method that combines real-time feedback to dynamically adjust pruning depth and strategy based on device operating status and task requirements, thereby balancing performance and computing efficiency.

[0034] 10. Energy Optimization: Reduce the computational complexity of the inference process through pruning and quantization, reduce the energy consumption of embedded devices, and extend the battery life of devices.

[0035] In order to solve the problems existing in the related art, the embodiment of the present application provides an optimization method for an embedded system model, which can be run on Figure 1 In the computer terminal shown, the computer terminal is explained below.

[0036] The embedded system model optimization method embodiment provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing an optimization method for an embedded system model. Figure 1As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0037] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0038] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the optimization method of the embedded system model in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the optimization method of the embedded system model described above is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0039] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet wirelessly.

[0040] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0041] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0042] In the above-mentioned operating environment, an embodiment of the present application provides an embodiment of an optimization method for an embedded system model. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0043] Figure 2 is a flow chart of an optimization method for an embedded system model according to an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0044] Step S202, obtaining a state function of a device for running a neural network model, where the state function is used to represent a current load state of the device;

[0045] In step S202, the device running the neural network model is an embedded device. Due to the limited resources, power consumption, limited storage and computing power of embedded devices, they are widely used in the fields of Internet of Things, drones, smart homes, etc.

[0046] Step S204, determining the pruning ratio and the number of quantization bits required for the neural network model according to the state function;

[0047] In step S204, the number of quantization bits refers to the number of binary digits used to represent a value, which is a direct determinant of quantization accuracy. Quantization accuracy describes the degree of difference between the quantized value and the original value. The fewer the number of bits, the lower the quantization accuracy is usually, that is, the larger the quantization error may be. For example, the number of quantization bits refers to the number of binary digits to which the parameters of the model are quantized, such as 8 bits (INT8), 4 bits (INT4) or 16 bits (FP16).

[0048] Step S206, optimizing the pruning ratio and the number of quantization bits according to the preset optimization target and the load of the device to obtain an optimized neural network model configuration, wherein the neural network model configuration at least includes: a pruning ratio and a number of quantization bits.

[0049] Through the above steps S202 to S206, by obtaining the state function of the device used to run the neural network model, the state function is used to represent the current load state of the device; the pruning ratio and the number of quantization bits required for the neural network model are determined according to the state function; the pruning ratio and the number of quantization bits are optimized according to the preset optimization target and the load of the device, and the optimized neural network model configuration is obtained, and the neural network model configuration at least includes: pruning ratio and quantization bit number, thereby achieving the purpose of adjusting the pruning ratio and the number of quantization bits of the neural network model according to the running state of the device, thereby achieving the technical effect of flexibly determining the pruning and quantization strategy of the model, and then solving the technical problem of poor model stability due to the single model pruning and quantization strategy of the embedded system model in the related technology. The following is a detailed description.

[0050] The optimization method of the embedded system model provided in the embodiment of the present application mainly achieves the best balance between performance and resource utilization efficiency by dynamically adjusting the pruning ratio and quantization accuracy.

[0051] In some embodiments of the present application, the specific steps for determining the pruning ratio and the number of quantization bits required for the neural network model according to the state function are as follows: obtaining the maximum value of the pruning ratio and the minimum value of the pruning ratio; determining the pruning ratio required for the neural network model based on the maximum value of the pruning ratio, the minimum value of the pruning ratio and the state function, wherein the state function is the ratio of the current load of the device to the maximum load of the device; determining the number of quantization bits required for the neural network model according to the state function.

[0052] Specifically, the state function is as follows:

[0053]

[0054] In the formula, f(S(t)) represents the state function, S(t) represents the device state value, Load currentIndicates the current device load, Load max represents the maximum load of the equipment, and t represents the time.

[0055] At the same time, the pruning strategy of the neural network model can be determined by the following formula:

[0056] P(t)=P min +(P max -P min )×f(S(t))

[0057] Where P(t) represents the pruning ratio, P min Indicates the minimum value of the pruning ratio, P max Indicates the maximum pruning ratio.

[0058] It can be understood that the state function can be determined according to the device state value.

[0059] In some embodiments of the present application, determining the number of quantization bits required for the neural network model according to the state function includes: obtaining a device state value corresponding to the state function, the device state value including at least one of the following: memory occupancy, task complexity, and power consumption; when the device state value is in a first interval, determining the number of quantization bits to be a first preset value; when the device state value is in a second interval, determining the number of quantization bits to be a second preset value; when the device state value is in a third interval, determining the number of quantization bits to be a third preset value, wherein the first preset value, the second preset value, and the third preset value increase successively, the maximum value of the first interval is less than the minimum value of the second interval, and the maximum value of the second interval is less than the minimum value of the third interval.

[0060] Specifically, as shown in the following formula:

[0061]

[0062] Wherein, n(t) represents the number of quantization bits at time t, (S(t)<0.5) represents the first interval, (0.5≤S(t)≤0.8) represents the second interval, and (S(t)>0.8) represents the third interval.

[0063] Before performing pruning operations on the neural network model, it is also necessary to determine the importance of each structural weight in the neural network model, and perform pruning operations on the neural network model according to the importance of each structural weight. Specifically, multiple structural weights in the neural network model and the loss function of the neural network model are obtained; the importance of each structural weight is determined according to each structural weight in the neural network model and the loss function of the neural network model; the total number of the multiple structural weights is obtained; the number of structural weights to be eliminated is determined according to the total number and the pruning ratio; and a target number of structural weights are eliminated from the multiple structural weights, wherein the importance of the target number of structural weights is less than any structural weight among the multiple structural weights except the target number of structural weights.

[0064] It should be noted that structural weights refer to the weight parameters connecting each neuron in the neural network model.

[0065] Each structural weight I(w ij ) can be determined by the following formula:

[0066]

[0067] In the formula, w ij represents the j-th structural weight of the i-th network layer in the neural network model, and L represents the loss function of the neural network model.

[0068] It can be understood that the structural weight I(w ij ) is used to indicate the influence of structural weights on the loss function.

[0069] After determining the importance of each structural weight, the P(t)×N weights with the least importance are removed according to the pruning ratio P(t), where N is the total number of weights.

[0070] The calculation formula of the pruned network layer Y is:

[0071] Y=X×W pruned

[0072] Among them, W pruned represents the remaining structural weight matrix after pruning, and X is the input of the network layer.

[0073] In the neural network model, the importance of different network layers is also different. In order to better save the consumption of equipment resources, the different network layers in the neural network model are divided into key layers and non-key layers. The network layers involving complex feature extraction, such as the convolution layer, are key layers (belonging to the preset key layers), and other network layers can be set as non-key layers.

[0074] The specific quantization method is as follows: obtain multiple network layers in the neural network model; when the network layer belongs to a preset key layer, quantize the structural weights in the network layer according to a preset number of quantization bits; when the network layer does not belong to a preset key layer, quantize the structural weights in the network layer according to the number of quantization bits required by the neural network model.

[0075] As shown below:

[0076]

[0077] In the formula, Q l Represents the lth network layer, FP16 represents half-precision floating point, which is a floating point format stored in 16 bits (binary bits). The number of quantization bits of FP16 is 16. l∈critical indicates that the lth network layer belongs to the preset critical layer. Indicates that the lth network layer does not belong to the preset key layer.

[0078] In some embodiments of the present application, the pruning ratio and the number of quantization bits are optimized according to a preset optimization target and the load status of the device, and the specific steps for obtaining the optimized neural network model configuration are as follows: obtaining an objective function corresponding to the preset optimization target, and the objective function is used to optimize the neural network model configuration according to the goal of minimizing the error after pruning and quantization operations on the neural network model and shortening the inference time of the neural network model, to obtain the optimized neural network model configuration; obtaining the current load condition of the device, and updating the optimized neural network model configuration according to the current load condition of the device.

[0079] Among them, obtaining the objective function corresponding to the preset optimization target includes: obtaining the pruning error after the neural network model is pruned and the quantization error after the neural network model is quantized; obtaining the inference time and a balance parameter, the balance parameter is used to balance the weight between the pruning error and the quantization error and the inference time; determining the objective function according to the pruning error, the quantization error, the inference time and the balance parameter.

[0080] Specifically, the following formula represents the structural weight W of the lth network layer: l Mapped to low-precision quantization operation through quantization function Q:

[0081]

[0082] Where W l q Represents the structural weight W of the lth network layerl The structural weight after quantization by the quantization function Q, Q(W l ) represents the quantization function Q, is the quantization step size, W l The maximum value of W l The minimum value of , where n represents the number of quantization bits.

[0083] When the pruning ratio is P and the number of quantization bits is n, the objective function is as follows:

[0084]

[0085] In the formula, E(P,n) represents the total error of the model when the pruning ratio is P and the number of quantization bits is n, T(P,n) represents the inference time of the model when the pruning ratio is P and the number of quantization bits is n, γ represents the balance parameter, It represents the minimum value when the pruning ratio is P and the number of quantization bits is n.

[0086] Where E(P,n)=∑ l (|W l -W l pruned | 2 +E quant )

[0087] Where W l pruned represents the structural weight of the lth network layer after pruning, E quant represents the quantization error,

[0088] E quant =|W l -W l q | 2

[0089] Where W l q Represents the structural weight of the lth network layer after quantization.

[0090] In some embodiments of the present application, the optimization method of model configuration is based on gradient descent; initializing the pruning ratio P and the number of quantization bits n; calculating the objective function E(P,n); updating P and n according to the device state S(t). The neural network model is adjusted based on the updated P and n, the adjusted neural network model is loaded, and the reasoning task is performed.

[0091] It should be noted that the optimization of the embedded system model proposed in the embodiment of the present application, by real-time evaluation of the importance of weights, dynamically prunes the low-contribution parts of the model, and reduces the amount of calculation and storage occupancy. Effectively reduce the inference time, while retaining the model performance to the greatest extent, and improve the real-time response capability of embedded devices. Dynamically adjust the quantization accuracy of the model based on the load status of the device, use high-precision representation for the key layer, and use low-precision representation for the non-key layer, so as to balance the performance and resource consumption of the model. Greatly reduce storage and computing overhead, further reduce energy consumption when the device is under high load, and improve the operating efficiency of the embedded system. Combined with pruning and quantization technology, a full balance of performance, resource utilization and inference speed is achieved through joint optimization, and the optimal pruning and quantization configuration is automatically selected according to the real-time status of the device. Provide a more flexible optimization mechanism to meet the needs of different embedded scenarios while reducing development complexity and deployment costs. Design a mechanism for real-time monitoring of device status, and dynamically adjust the pruning ratio and quantization accuracy according to parameters such as memory usage, task complexity and power consumption. Intelligent matching of model resources and device capabilities is achieved, extending device battery life, and improving the adaptability of the model in multi-task scenarios.

[0092] Figure 3 A method for optimizing a visual model is shown, including: determining pruning and quantization parameters according to a device load state, pruning and quantizing the visual model, and then deploying the visual model.

[0093] Figure 4 An optimization device for an embedded system model according to an embodiment of the present application includes:

[0094] An acquisition module 40 is used to acquire a state function of a device for running a neural network model, wherein the state function is used to represent a current load state of the device;

[0095] A determination module 42, configured to determine the pruning ratio and the number of quantization bits required for the neural network model according to the state function;

[0096] The optimization module 44 is used to optimize the pruning ratio and the quantization bit number according to the preset optimization target and the load condition of the device to obtain an optimized neural network model configuration, and the neural network model configuration at least includes: the pruning ratio and the quantization bit number.

[0097] Through the optimization device of the above-mentioned embedded system model, the state function of the device used to run the neural network model is obtained, and the state function is used to represent the current load state of the device; the pruning ratio and the number of quantization bits required for the neural network model are determined according to the state function; the pruning ratio and the number of quantization bits are optimized according to the preset optimization target and the load condition of the device to obtain the optimized neural network model configuration, and the neural network model configuration at least includes: the pruning ratio and the number of quantization bits, thereby achieving the purpose of adjusting the pruning ratio and the number of quantization bits of the neural network model according to the operating state of the device, thereby achieving the technical effect of flexibly determining the pruning and quantization strategy of the model, and then solving the technical problem in the related technology that the model stability is poor due to the single model pruning and quantization strategy of the embedded system model.

[0098] The determination module 42 includes: a determination submodule, which is used to determine the pruning ratio and the number of quantization bits required for the neural network model according to the state function, including: obtaining the maximum value of the pruning ratio and the minimum value of the pruning ratio; determining the pruning ratio required for the neural network model based on the maximum value of the pruning ratio, the minimum value of the pruning ratio and the state function, wherein the state function is the ratio of the current load of the device to the maximum load of the device; determining the number of quantization bits required for the neural network model according to the state function.

[0099] The determination submodule includes: a determination unit and an acquisition unit, the determination unit is used to determine the number of quantization bits required for the neural network model according to the state function, including: acquiring a device state value corresponding to the state function, the device state value including at least one of the following: memory occupancy, task complexity and power consumption value; when the device state value is in a first interval, determining the number of quantization bits to be a first preset value; when the device state value is in a second interval, determining the number of quantization bits to be a second preset value; when the device state value is in a third interval, determining the number of quantization bits to be a third preset value, wherein the first preset value, the second preset value and the third preset value increase in sequence, the maximum value of the first interval is less than the minimum value of the second interval, and the maximum value of the second interval is less than the minimum value of the third interval.

[0100] An acquisition unit is used to acquire multiple structural weights in the neural network model and the loss function of the neural network model; determine the importance of each structural weight according to each structural weight in the neural network model and the loss function of the neural network model; acquire the total number of the multiple structural weights; determine the number of structural weights to be eliminated according to the total number and the pruning ratio; eliminate a target number of structural weights from the multiple structural weights, the importance of the target number of structural weights is less than any structural weight among the multiple structural weights except the target number of structural weights.

[0101] The determination unit also includes: a strategy subunit, which is used to obtain multiple network layers in the neural network model; when the network layer belongs to a preset key layer, the structural weights in the network layer are quantized according to a preset number of quantization bits; when the network layer does not belong to the preset key layer, the structural weights in the network layer are quantized according to the number of quantization bits required by the neural network model.

[0102] The optimization module 44 includes: an optimization submodule, which is used to optimize the pruning ratio and the number of quantization bits according to a preset optimization target and the load status of the device to obtain an optimized neural network model configuration, including: obtaining an objective function corresponding to the preset optimization target, the objective function is used to optimize the neural network model configuration according to the goal of minimizing the error after pruning and quantization operations on the neural network model and minimizing the inference time of the neural network model to obtain the optimized neural network model configuration; obtaining the current load condition of the device, and updating the optimized neural network model configuration according to the current load condition of the device.

[0103] The optimization submodule includes: an objective function unit, which is used to obtain the objective function corresponding to the preset optimization target, including: obtaining the pruning error after the neural network model is pruned and the quantization error after the neural network model is quantized; obtaining the inference time and a balance parameter, and the balance parameter is used to balance the weight between the pruning error and the quantization error and the inference time; determining the objective function according to the pruning error, the quantization error, the inference time and the balance parameter.

[0104] It should be noted that Figure 4 The optimized device of the embedded system model shown is used to perform Figure 2 The optimization method of the embedded system model shown in the figure, therefore the relevant explanations in the above-mentioned optimization method of the embedded system model are also applicable to the optimization device of the embedded system model, and will not be repeated here.

[0105] An embodiment of the present application also provides a computer device, including: a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the optimization method of the above-mentioned embedded system model.

[0106] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned embedded system model optimization method by running the computer program.

[0107] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the embedded system model optimization method in the present application.

[0108] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0109] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0110] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0111] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0112] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0113] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0114] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for optimizing an embedded system model, characterized in that: include: Obtaining a state function of a device for running a neural network model, wherein the state function is used to represent a current load state of the device; Determine the pruning ratio and quantization bit number required for the neural network model according to the state function; The pruning ratio and the number of quantization bits are optimized according to a preset optimization target and the load condition of the device to obtain an optimized neural network model configuration, wherein the neural network model configuration at least includes: the pruning ratio and the number of quantization bits.

2. The method according to claim 1, characterized in that Determining the pruning ratio and the number of quantization bits required for the neural network model according to the state function includes: Obtaining a maximum value of the pruning ratio and a minimum value of the pruning ratio; Determining a pruning ratio required for the neural network model based on the maximum value of the pruning ratio, the minimum value of the pruning ratio and the state function, wherein the state function is a ratio of a current load of the device to a maximum load of the device; The number of quantization bits required for the neural network model is determined according to the state function.

3. The method according to claim 2, characterized in that Determining the number of quantization bits required for the neural network model according to the state function includes: Obtaining a device state value corresponding to the state function, wherein the device state value includes at least one of the following: memory occupancy rate, task complexity, and power consumption value; When the device state value is in a first interval, determining the number of quantization bits to be a first preset value; When the device state value is in a second interval, determining the number of quantization bits to be a second preset value; When the device status value is in a third interval, the number of quantization bits is determined to be a third preset value, wherein the first preset value, the second preset value and the third preset value increase sequentially, the maximum value of the first interval is smaller than the minimum value of the second interval, and the maximum value of the second interval is smaller than the minimum value of the third interval.

4. The method according to claim 2, characterized in that: The method further comprises: Acquire a plurality of structural weights in the neural network model and a loss function of the neural network model; Determining the importance of each structural weight according to each structural weight in the neural network model and the loss function of the neural network model; Obtaining a total number of the plurality of structural weights; Determine the number of structural weights to be removed according to the total number and the pruning ratio; A target number of structural weights are eliminated from the multiple structural weights, and the importance of the target number of structural weights is less than any structural weight among the multiple structural weights except the target number of structural weights.

5. The method according to claim 3, characterized in that: The method further comprises: Acquire multiple network layers in the neural network model; In the case where the network layer belongs to a preset key layer, quantizing the structural weights in the network layer according to a preset number of quantization bits; In the case that the network layer does not belong to the preset key layer, the structural weights in the network layer are quantized according to the number of quantization bits required by the neural network model.

6. The method according to claim 1, characterized in that The pruning ratio and the quantization bit number are optimized according to a preset optimization target and the load state of the device to obtain an optimized neural network model configuration, including: Obtaining an objective function corresponding to the preset optimization target, wherein the objective function is used to optimize the configuration of the neural network model according to the goal of minimizing the error after pruning and quantization operations on the neural network model and minimizing the inference time of the neural network model, to obtain the optimized neural network model configuration; Obtain the current load condition of the device, and update the optimized neural network model configuration according to the current load condition of the device.

7. The method according to claim 6, characterized in that Obtaining the objective function corresponding to the preset optimization objective includes: Obtaining a pruning error after a pruning operation is performed on the neural network model and a quantization error after a quantization operation is performed on the neural network model; Acquire the inference duration and a balance parameter, where the balance parameter is used to balance the weight between the pruning error and the quantization error and the inference duration; The objective function is determined according to the pruning error, the quantization error, the inference duration, and the balance parameter.

8. An optimization device for an embedded system model, characterized in that: include: An acquisition module, used for acquiring a state function of a device for running a neural network model, wherein the state function is used for representing a current load state of the device; A determination module, used to determine the pruning ratio and quantization bit number required for the neural network model according to the state function; The optimization module is used to optimize the pruning ratio and the number of quantization bits according to the preset optimization target and the load condition of the device to obtain an optimized neural network model configuration, and the neural network model configuration at least includes: the pruning ratio and the number of quantization bits.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory is used to store program instructions; The processor is connected to the memory and is used to execute the embedded system model optimization method described in any one of claims 1-7.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the optimization method of the embedded system model described in any one of claims 1 to 7 is implemented.

Images