Quantization precision selection method and device for deep network model

By determining the quantization sensitivity of each layer of the deep network model and adjusting the quantization accuracy layer by layer, the problem of insufficient selection of quantization accuracy by existing tools is solved, and the effect of improving quantization efficiency and reducing inference time is achieved on the basis of ensuring accuracy.

WO2024230170A1PCT designated stage expired Publication Date: 2024-11-14SHENZHEN ORBBEC CO LTD

Patent Information

Application Number
PCT/CN2023/139026
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-11
Filing Date
2023-12-15
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

The hybrid precision quantization tools of existing deep network models do not pay enough attention to the selection of quantitative accuracy, resulting in long inference time and complex tool structure, which makes it impossible to effectively balance model accuracy and quantization efficiency.

Method used

By obtaining the quantization sensitivity of each layer of the deep network model, adjusting the quantization accuracy to optimize the inference time, determining the quantization sensitivity by combining the calculation error value and the mean ratio, and adjusting the quantization bit width layer by layer to meet the preset target time.

Benefits of technology

While ensuring model accuracy, it improves quantitative efficiency, reduces inference time and storage consumption, and enables the deep network model to run on limited resource devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023139026_14112024_PF_FP_ABST
    Figure CN2023139026_14112024_PF_FP_ABST
Patent Text Reader

Abstract

The present application relates to the technical field of deep learning, and in particular, to a quantization precision selection method and device for a deep network model. The quantization precision selection method comprises: S110, acquiring a deep network model to be processed, and letting the quantization precision of each layer in the deep network model be first precision; S120, determining the quantization sensitivity of each layer in the deep network model; S130, traversing, in the deep network model, layers the quantization precision of which is not changed, so as to select the layer having the lowest quantization sensitivity, and changing the quantization precision of the layer to be second precision, wherein the second precision is smaller than the first precision; and S140, performing inference by using the deep network model the quantization precision of which has been changed, and if the inference time is shorter than or equal to a preset target time, ending to obtain an expected deep network model. According to the present embodiment, the model quantization efficiency is improved while the precision of a network model is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

A method and device for selecting quantization accuracy of deep network models

[0001] This application claims priority to a Chinese patent application filed with the Patent Office of China on May 11, 2023, with application number 2023105342377, entitled “A method and device for selecting quantization accuracy of a deep network model”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present application relates to the field of deep learning technology, and in particular to a method and device for selecting quantization accuracy of a deep network model. Background Art

[0003] Model quantization is an essential step in the deployment of existing deep networks. Common sense suggests that higher quantization bit widths result in better quantization, while lower bit widths result in lower quantization accuracy. However, higher bit widths increase inference time, while lower bit widths often result in poor model accuracy after quantization. This has naturally led to research into mixed-precision quantization tools. Currently, most mixed-precision quantization methods use reinforcement learning to automatically determine the quantization accuracy of each layer.

[0004] However, existing tools with mixed-precision quantization capabilities pay too little attention to mixed precision. These tools often focus on selecting the quantization parameter and then determining the quantization bit width; or they select the quantization bit width and quantization parameter together, clearly not paying enough attention to the choice of precision. Deep learning-based methods also face the problems of long quantization times and complex tool structures.

[0005] Summary of the Invention

[0006] In view of this, an embodiment of the present application provides a method and device for selecting quantization accuracy of a deep network model, which can solve at least one technical problem in the related art.

[0007] In a first aspect, an embodiment of the present application provides a method for selecting quantization precision for a deep network model, comprising: S110, obtaining an input image and a deep network model to be processed, quantizing the parameters of the deep network model, and setting the quantization precision of each layer in the quantized deep network model to a first precision; S120, inputting the input image into the quantized deep network model to obtain a feature map output by each layer of the deep network model, and determining the quantization sensitivity of each layer in the deep network model using the feature map output by each layer of the deep network model; S130, traversing the layers in the deep network model whose quantization precision has not been changed to select a layer with the lowest quantization sensitivity, and changing the quantization precision of the layer to a second precision, which is less than the first precision; S140, performing inference using the deep network model after the quantization precision has been changed. If the inference time is greater than a preset target time, returning to step S130; if the inference time is less than or equal to the preset target time, terminating the process to obtain a deep network model that meets the expectations. This embodiment adjusts the quantization precision of the network model based on the quantization sensitivity of each layer of the deep network model, thereby improving the model quantization efficiency while ensuring the accuracy of the network model.

[0008] In some embodiments, determining the quantization sensitivity of each layer in a deep network model includes: obtaining a quantized value and a floating-point value of a feature map output by each layer in the deep network model; and determining the quantization sensitivity of each layer based on the quantized value and the floating-point value. This embodiment calculates the quantization sensitivity of each layer by combining the quantized value and the floating-point value of the feature map output by each layer, thereby obtaining an accurate quantization sensitivity value and further ensuring the accuracy of the network model.

[0009] In some embodiments, the quantization sensitivity of each layer is determined based on the quantization value and the floating-point value of the layer, including: calculating the error value between the quantization value and the floating-point value of the feature map output by each layer as a first value; calculating the mean of the floating-point values ​​of the feature map output by each layer as a second value; calculating the ratio of the first value to the second value of each layer as the quantization sensitivity of the layer; wherein the error value includes: mean square error, cosine value, root mean square error, or mean absolute error.

[0010] This embodiment calculates the ratio of the error value (or first value) to the mean value (or second value) to obtain the quantization sensitivity of each layer. This can quickly obtain the quantization sensitivity, further improving the quantization efficiency of the network model. In addition, this embodiment also obtains the quantization sensitivity based on statistical data, which can obtain a more accurate quantization sensitivity value. The calculation is simple and easy to implement in this application.

[0011] In the second aspect, an embodiment of the present application provides a quantization accuracy selection device for a deep network model, including: an acquisition module for acquiring an input image and a deep network model to be processed, quantizing the parameters of the deep network model and making the quantization accuracy of each layer in the quantized deep network model a first accuracy; a determination module for inputting the input image into the quantized deep network model to obtain a feature map output by each layer of the deep network model, and determining the quantization sensitivity of each layer in the deep network model using the feature map output by each layer of the deep network model; a selection module for traversing the layers in the deep network model whose quantization accuracy has not been changed to select a layer with the smallest quantization sensitivity, and change the quantization accuracy of the layer to a second accuracy, which is less than the first accuracy; an inference module for performing inference using the deep network model after the quantization accuracy has been changed. If the inference time is greater than the preset target time, the process returns to the selection module; if the inference time is less than or equal to the preset target time, the process ends and a deep network model that meets expectations is obtained.

[0012] In some embodiments, the determination module includes a first acquisition submodule and a second determination submodule. The first acquisition submodule is used to obtain the quantized value and floating-point value of the feature map output by each layer in the deep network model; the second determination submodule is used to calculate the error value between the quantized value and floating-point value of the feature map output by each layer as the first value; calculate the mean of the floating-point values ​​of the feature map output by each layer as the second value; and calculate the ratio of the first value to the second value of each layer as the quantization sensitivity of the layer; wherein the error value includes: mean square error, cosine value, root mean square error, or mean absolute error.

[0013] In a third aspect, an embodiment of the present application provides a transplantation system, which includes a data processing device and an electronic device. The data processing device is used to implement the quantization accuracy selection method of the deep network model as described in any embodiment of the first aspect when executing a computer program to obtain a deep network model that meets expectations. The deep network model is transplanted to the electronic device for use in the electronic device.

[0014] In a fourth aspect, an embodiment of the present application provides an electronic device comprising a processor, wherein the transplantation system transplants a deep network model, such as the transplantation system in the third aspect, to the processor of the electronic device, and the processor processes input data through the carried deep network model to perform a specific task.

[0015] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a data processing device, it implements a method for selecting quantization accuracy of a deep network model as in any embodiment of the first aspect.

[0016] In a sixth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a data processing device, the data processing device executes the method for selecting quantization accuracy of a deep network model as described in any embodiment of the first aspect.

[0017] It should be understood that the beneficial effects of the second to sixth aspects can be found in the relevant description of the embodiment of the first aspect and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] FIG1 is a diagram of a transplantation system for a deep network model provided by the present application;

[0020] FIG2 is a schematic diagram of an implementation flow of a method for selecting quantization accuracy of a deep network model provided by an embodiment of the present application;

[0021] FIG3 is a process diagram of step S120 provided in one embodiment of the present application;

[0022] FIG4 is a process diagram of step S122 provided in one embodiment of the present application;

[0023] FIG5 is a schematic diagram of the structure of a device for selecting quantization accuracy of a deep network model provided by an embodiment of the present application;

[0024] FIG6 is a schematic structural diagram of a determination module provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0026] The term "and / or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0027] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of the present application include a particular feature, structure, or characteristic described in conjunction with that embodiment. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0028] In addition, in the description of this application, "a plurality of" means two or more. The terms "first" and "second" are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0029] In order to illustrate the technical solution described in this application, specific embodiments are provided below.

[0030] FIG1 is a diagram of a transplantation system for a deep network model provided in accordance with the present application. The system includes electronic devices and data processing devices. The data processing devices are used to process the deep network model according to the quantization accuracy selection method provided in one or more embodiments of the present application to obtain a quantized deep network model; the quantized model is transplanted to an electronic device so that the user can perform specific tasks through the electronic device, such as image processing, speech recognition, liveness detection, and face recognition. Specifically, the data processing devices include but are not limited to electronic devices with computing capabilities, such as computers, tablets, servers, or wearable devices. The servers include but are not limited to independent servers or cloud servers; the electronic devices include depth cameras, mobile devices, payment terminals, and robots.

[0031] In one embodiment, the deep network model is composed of multiple layers, which can be functionally divided into an input layer, a hidden layer, and an output layer. The input layer is used to receive input data, the hidden layer is used to perform a series of nonlinear changes and feature extraction on the input data to better complete a specific task, and the output layer is used to output the results of performing the specific task. It should be noted that this application does not limit the number of input layers, hidden layers, and output layers. They can be multiple layers or a single layer, depending on the specific tasks performed by the deep network model.

[0032] In this embodiment, the depth in the deep network model refers to the number of layers of the network model, which is more than the number of layers of the traditional neural network and has powerful feature extraction and representation capabilities, but it also means that the deep network model requires higher reasoning costs and larger storage. In order to reduce the reasoning cost and storage of the deep network model, the deep network model needs to be quantized during its deployment, and quantization will lose the accuracy of the model. Based on this, the present application provides a method for selecting the quantization accuracy of a deep network model, which reduces the reasoning cost and storage consumption of the deep network model while ensuring high quantization accuracy, so that the deep network model can also run on devices with limited resources and capabilities such as low computing power and low memory.

[0033] FIG2 is a schematic diagram of an implementation flow of a method for selecting quantization accuracy of a deep network model provided in an embodiment of the present application, the method comprising steps S110 to S140.

[0034] S110, obtaining an input image and a deep network model to be processed, quantizing the parameters of the deep network model and setting the quantization accuracy of each layer in the quantized deep network model to a first accuracy.

[0035] In one embodiment, the input image may include one or more images, which may be face images, body images, gesture images, or other images. The specific content of the image depends on the specific task performed by the deep network model.

[0036] In one embodiment, the parameters in the deep network model are high-precision floating-point numbers. In order to reduce the size of the model, improve the speed of inference, and reduce memory and power consumption, they need to be converted into low-precision fixed-point numbers. That is, the parameters of the deep network model are quantized and the quantization accuracy of each layer of the quantized deep network model is set to the first accuracy (denoted as Q1). Correspondingly, the quantization bit width of each layer of the quantized deep network model is the first bit width. In some possible implementations, the first bit width can be, for example, 32 bits.

[0037] S120, inputting the input image into the quantized deep network model to obtain a feature map output by each layer of the deep network model, and using the feature map output by each layer of the deep network model to determine the quantization sensitivity of each layer in the deep network model.

[0038] Among them, quantization sensitivity indicates the sensitivity of each layer in the deep network model to the approximation of the original floating-point operations with fixed-point operations; the greater the quantization sensitivity of a layer in the deep network model, the more sensitive the layer is to quantization; conversely, the smaller the quantization sensitivity of a layer in the deep network model, the less sensitive the layer is to quantization.

[0039] In some embodiments of the present application, as shown in Figure 3, step S120 may include: S121, obtaining the quantization value and floating-point value of the feature map output by each layer in the deep network model; S122, determining the quantization sensitivity of the layer based on the quantization value and floating-point value of each layer.

[0040] Assume that the feature map output by layer j in the deep network model is F j , based on the feature map, obtain the feature map F of the j-th layer output j The quantized value F quan tj and the floating point value F floatj ; According to the feature map F output by the jth layer j The quantized value F quan tj and the floating point value F floatj Compute the quantization sensitivity of the jth layer.

[0041] This embodiment combines the quantization value and the floating-point value to determine the quantization sensitivity, and subsequently selects the layer with the smaller quantization sensitivity to use the low bit width for quantization, so as to reduce the inference time of the network model. The cycle is repeated until the inference time meets the target value, thereby improving the efficiency of quantization while ensuring the network accuracy.

[0042] As a possible implementation method of an embodiment of the present application, as shown in Figure 4, step S122 may include: S1221, calculating the error value of the quantization value and the floating-point value of the feature map output by each layer as the first value; S1222, calculating the mean of the floating-point values ​​of the feature map output by each layer as the second value; S1223, calculating the ratio of the first value to the second value of each layer as the quantization sensitivity of the layer.

[0043] In some examples of this application, the error value between the quantized value and the floating point value may include but is not limited to the mean square error, cosine value, root mean square error, or mean absolute error, etc., preferably the mean square error is used as the error value. Specifically, the feature map F output by the jth layer of the deep network model j The quantitative value is F quan tj , the floating point value is F floatj , the first value is equal to mse(F quan tj , F floatj ),in, The second value is equal to That is F floatj The mean of the quantization sensitivity of the jth layer is This embodiment uses the statistical mean square error to measure the quantization sensitivity, which can obtain a more accurate value of the quantization sensitivity and further ensure the accuracy of the network model.

[0044] S130, traverse the layers in the deep network model whose quantization precision has not been changed to select a layer with the smallest quantization sensitivity, and change the quantization precision of the layer to a second precision; wherein the second precision is less than the first precision.

[0045] In some embodiments of the present application, the second precision can be denoted as Q2, where Q2 is less than Q1. Correspondingly, its quantization bit width is the second bit width. In some possible implementations, the second bit width can be, for example, 16 bits or 8 bits. This embodiment reduces the inference time of the network model by selecting layers with low quantization sensitivity and using a low bit width for quantization, that is, changing these layers with low quantization sensitivity from the first precision to the second precision.

[0046] S140, use the deep network model with changed quantization accuracy to perform inference. If the inference time is greater than the preset target time, return to step S130; if the inference time is less than or equal to the preset target time, end and obtain the expected deep network model.

[0047] In an embodiment of the present application, the quantization sensitivity of each layer is first determined, and then the layer with the smallest quantization sensitivity is selected from the layers whose quantization accuracy has not been changed, and quantization is performed using a low bit width. This cycle is repeated until the inference time meets the preset target time. The process is then terminated to obtain a deep network model that meets the expectations, thereby improving the efficiency of quantization while ensuring the accuracy of the network model.

[0048] It should be noted that, in some other embodiments, when the inference time is equal to the preset target time, the step of returning to step S130 may also be executed. Those skilled in the art may make a choice based on actual conditions, and the embodiments of the present application do not impose any specific restrictions on this.

[0049] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0050] One embodiment of the present application provides a quantization precision selection device for a deep network model. For details not described in detail in the quantization precision selection device, please refer to the relevant description in the aforementioned quantization precision selection method embodiment, and no further details will be given here.

[0051] FIG5 is a schematic diagram of the structure of a quantization precision selection device for a deep network model provided by an embodiment of the present application. The quantization precision selection device includes: an acquisition module 41 for acquiring an input image and a deep network model to be processed, quantizing the parameters of the deep network model, and setting the quantization precision of each layer in the quantized deep network model to a first precision; a determination module 42 for inputting the input image into the quantized deep network model to obtain a feature map output by each layer of the deep network model, and determining the quantization sensitivity of each layer in the deep network model using the feature map output by each layer of the deep network model; a selection module 43 for traversing the layers of the deep network model whose quantization precision has not been changed to select a layer with the lowest quantization sensitivity, and changing the quantization precision of the layer to a second precision, which is less than the first precision; and an inference module 44 for performing inference using the deep network model after the quantization precision has been changed. If the inference time is greater than a preset target time, the process returns to the selection module 43; if the inference time is less than or equal to the preset target time, the process ends and a deep network model that meets the expectations is obtained.

[0052] In some embodiments, as shown in FIG6 , determination module 42 includes a first acquisition submodule 421 and a second determination submodule 422. The first acquisition submodule 421 is configured to acquire the quantized values ​​and floating-point values ​​of the feature maps output by each layer in the deep network model. The second determination submodule 422 is configured to determine the quantization sensitivity of each layer based on the quantized values ​​and floating-point values ​​of the layer.

[0053] In some embodiments, the second determination submodule is specifically used to: calculate the error value between the quantization value and the floating-point value of the feature map output by each layer as the first value; calculate the mean of the floating-point values ​​of the feature map output by each layer as the second value; calculate the ratio of the first value to the second value of each layer as the quantization sensitivity of the layer.

[0054] An embodiment of the present application also provides an electronic device, which may include one or more processors. A deep network model is transplanted to the processor of the electronic device through a transplantation system as shown in Figure 1, so that the processor can process input data through the onboard deep network model to perform specific tasks.

[0055] In one embodiment, the electronic device further includes a camera configured to capture two-dimensional images or depth images and transmit them to a processor; the processor processes the input images based on a deep network model to perform specific tasks, such as facial recognition, liveness detection, or payment. It should be noted that the camera and electronic device may be designed as an integrated or independent system, and the two may communicate data via wired or wireless means, without limitation herein.

[0056] In one embodiment, the processor may be a central processing unit (CPU), or other general-purpose processors, neural network processing chips, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0057] Those skilled in the art will appreciate that the above are merely examples of electronic devices and do not constitute a limitation on electronic devices. An electronic device may include more or fewer components than shown, or may combine certain components or different components. For example, an electronic device may also include input and output devices, network access devices, buses, etc.

[0058] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0059] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the steps in the embodiment of the method for selecting quantization accuracy of a deep network model can be implemented.

[0060] An embodiment of the present application provides a computer program product. When the computer program product is run, the steps in the embodiment of the method for selecting quantization accuracy of a deep network model can be implemented.

[0061] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0062] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0063] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0064] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and the computer program that can be completed by instructing the relevant hardware through a computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0065] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for selecting quantization accuracy of a deep network model, characterized in that: include: S110, obtaining an input image and a deep network model to be processed, quantizing parameters of the deep network model and setting the quantization accuracy of each layer in the quantized deep network model to a first accuracy; S120, inputting the input image into the quantized deep network model to obtain a feature map output by each layer of the deep network model, and determining the quantization sensitivity of each layer in the deep network model using the feature map output by each layer of the deep network model; S130, traversing the layers of the deep network model whose quantization precision has not been changed to select a layer with the smallest quantization sensitivity, and changing the quantization precision of the layer to a second precision, where the second precision is less than the first precision; S140, performing reasoning using the deep network model after changing the quantization accuracy, and if the reasoning time is less than or equal to the preset target time, then the process ends to obtain a deep network model that meets expectations.

2. The quantization accuracy selection method according to claim 1, characterized in that: Determining the quantization sensitivity of each layer in the deep network model by using the feature map output by each layer of the deep network model includes: Obtaining quantized values ​​and floating-point values ​​of feature maps output by each layer in the deep network model; The quantization sensitivity of each layer is determined according to the quantization value and the floating point value of the layer.

3. The quantization accuracy selection method according to claim 2, characterized in that: The determining the quantization sensitivity of each layer according to the quantization value and the floating point value of the layer includes: Calculate the error value between the quantized value and the floating point value of the feature map output by each layer as the first value; Calculate the mean of the floating-point values ​​of the feature map output by each layer as the second value; The ratio of the first value to the second value of each layer is calculated as the quantization sensitivity of the layer.

4. The quantization accuracy selection method according to claim 3, characterized in that: The error value includes: mean square error, cosine value, root mean square error, or mean absolute error.

5. The quantization accuracy selection method according to claim 4, characterized in that: When the error value is the mean square error, the quantization sensitivity is calculated as follows: Assuming that the feature map F output by the jth layer of the deep network model j The quantitative value is F quan tj , the floating point value is F floatj , the first value is equal to mse(F quan tj , F floatj ),in, The second value is equal to That is F floatj The mean of the quantization sensitivity of the jth layer is 6. The quantization accuracy selection method according to any one of claims 1 to 5, characterized in that: After performing reasoning using the neural network model with changed quantization accuracy, the method further includes: if the reasoning time is greater than the preset target time, returning to execute step S130.

7. A device for selecting quantization accuracy of a deep network model, characterized in that: include: An acquisition module, used to acquire an input image and a deep network model to be processed, quantize the parameters of the deep network model and set the quantization accuracy of each layer in the quantized deep network model to be a first accuracy; A determination module, configured to input the input image into the quantized deep network model to obtain a feature map output by each layer of the deep network model, and determine the quantization sensitivity of each layer in the deep network model using the feature map output by each layer of the deep network model; A selection module is used to traverse the layers of the deep network model whose quantization precision is not changed to select a layer with the smallest quantization sensitivity, and changing the quantization accuracy of the layer to a second accuracy, where the second accuracy is less than the first accuracy; The inference module is used to perform inference using the deep network model after the quantization accuracy is changed. If the inference time is less than or equal to the preset target time, the deep network model that meets the expectations is obtained.

8. A deep network model transplantation system, characterized in that: It comprises a data processing device and an electronic device, wherein the data processing device is used to execute the quantization accuracy selection method of the deep network model as described in any one of claims 1 to 6 to obtain a deep network model that meets the expectations, and the deep network model is transplanted to the electronic device for use by the electronic device.

9. An electronic device, comprising a processor, characterized in that: The transplantation system as described in claim 8 transplants the deep network model to the processor of the electronic device, and the processor processes the input data by carrying the deep network model to perform a specific task.

10. A computer storage medium storing a computer program, characterized in that: When the computer program is executed, the quantization accuracy selection method according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Quantization precision selection method and device for deep network model

    CN116579399A

  • Compression for deep neural network

    US20210266565A1

  • Method and system for generating a predictive model

    US20230037498A1

  • Model quantization method and apparatus, device, storage medium, computer program product, and computer program

    WO2023029349A1

  • Neural network training method and apparatus, character recognition method and apparatus, device, storage medium, and computer program product

    WO2023040626A1

Cited By

  • Model quantification method, electronic equipment and storage medium

    CN120430434A

  • Multi-algorithm collaborative industrial simulation agent model construction method

    CN120805435A

  • Adaptive quantization language interaction method and related equipment

    CN121214929A