Reasoning method, device, equipment and storage medium of deep learning model

By dynamically generating output scaling factors, map floating-point data to int8 type, it solves the error problem caused by the quantization of deep learning models, improves the inference efficiency and quality, and realizes the efficient application of the model.

CN114819159BActive Publication Date: 2024-11-26BEIJING QIYI CENTURY SCI & TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210404848.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-18
Publication Date
2024-11-26
Estimated Expiration
2042-04-18

AI Technical Summary

Technical Problem

The deep learning model has large errors due to quantization during the inference process, which affects the performance optimization and application efficiency of the model.

Method used

By dynamically generating the output scaling factor matching the data set to be inferred, the floating-point data inference results are mapped to the int8 type to achieve a balance of quantization speed and mass.

Benefits of technology

It improves the inference efficiency of deep learning network models, ensures the quality of inference, and effectively alleviates the problem of large overall errors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114819159B_ABST
    Figure CN114819159B_ABST
Patent Text Reader

Abstract

The present application relates to a method, device, equipment and storage medium for reasoning a deep learning model, wherein the method comprises: obtaining a data set to be inferred; inputting the data set to be inferred into an inference convolution kernel to obtain a floating-point data reasoning result; obtaining an output scaling factor corresponding to the data set to be inferred; wherein the output scaling factor is determined according to the maximum value of an element in the data reasoning result; calculating the product of the data reasoning result and the output scaling factor to obtain a quantization result; and continuing the reasoning of the deep learning model according to the quantization result. The present application is used to solve the technical problem that quantization leads to a large overall error in the process of optimizing model performance in the prior art.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of deep learning network technology, and in particular to an inference method, device, equipment and storage medium for a deep learning model. Background Art

[0002] At present, deep learning has been widely used in all walks of life and has achieved great results in areas that are difficult to solve with traditional algorithms. However, one problem with the current promotion of deep learning applications is that its operating costs are huge. Even with the significant improvement in the computing power of GPUs (graphics processing units), the deep learning models with increasing parameters year by year have also eaten up the benefits of GPU performance improvement. Therefore, optimizing the performance of the model itself is a key to whether deep learning can be applied in large-scale production processes.

[0003] In the process of model performance optimization, quantization is one of the methods. One of the keys to quantization is to convert the input of floating point float32 precision into the form of integer int8. This conversion process will introduce corresponding errors. Most of the current solutions cannot adjust the errors in real time, resulting in a large overall error, which affects the final product instructions. Summary of the invention

[0004] The present application provides an inference method, device, equipment and storage medium for a deep learning model, which are used to solve the technical problem of large overall error caused by quantization in the model performance optimization process of the prior art.

[0005] In a first aspect, the present application provides a method for reasoning a deep learning model, comprising:

[0006] Get the data set to be inferred;

[0007] Inputting the data set to be inferred into the inference convolution kernel to obtain a floating-point data inference result;

[0008] Obtaining an output scaling factor corresponding to the data set to be inferred; wherein the output scaling factor is determined according to the maximum value of the elements in the data inference result;

[0009] Calculating the product of the data inference result and the output scaling factor to obtain a quantized result;

[0010] The deep learning model is continued to be inferred according to the quantization result.

[0011] Optionally, obtaining the output scaling factor corresponding to the data set to be inferred includes:

[0012] Determine the maximum element from each element of the data inference result;

[0013] The quotient of a preset value and the maximum element is calculated to obtain the output scaling factor; wherein the preset value is an upper limit value of a value range corresponding to the data type of the data set to be inferred.

[0014] Optionally, the step of inputting the data set to be inferred into an inference convolution kernel to obtain a floating-point data inference result includes:

[0015] Obtaining a first scaling factor corresponding to the data set to be inferred;

[0016] Obtaining a weight corresponding to the inference convolution kernel and a second scaling factor corresponding to the weight;

[0017] Obtaining a bias for the inference convolution kernel;

[0018] The floating-point data inference result is calculated according to the data set to be inferred, the first scaling factor, the weight, the second scaling factor and the bias.

[0019] Optionally, calculating the floating-point data inference result according to the data set to be inferred, the first scaling factor, the weight, the second scaling factor, and the bias includes:

[0020] Calculating the product of the data set to be inferred and the weight to obtain a first intermediate result;

[0021] Dividing the first intermediate result by the first scaling factor and the second scaling factor to obtain the second intermediate result;

[0022] The sum of the second intermediate result and the bias is calculated to obtain the floating-point data inference result.

[0023] Optionally, obtaining a first scaling factor corresponding to the data set to be inferred includes:

[0024] Obtaining a floating-point input data set corresponding to the data set to be inferred;

[0025] The first scaling factor is obtained by dividing the data set to be inferred by the floating-point input data set.

[0026] Optionally, obtaining the data set to be inferred includes:

[0027] From the target video, determine the data set to be inferred corresponding to the current video frame;

[0028] The obtaining the output scaling factor corresponding to the data set to be inferred includes:

[0029] Determine whether a scene switch occurs in the current video frame relative to the previous video frame, and obtain a determination result;

[0030] The output scaling factor corresponding to the data set to be inferred is determined according to the judgment result.

[0031] Optionally, determining the output scaling factor corresponding to the data set to be inferred according to the judgment result includes:

[0032] If the judgment result indicates that a scene switch occurs, then determine the maximum element from the elements of the data inference result; calculate the quotient of a preset value and the maximum element to obtain the output scaling factor; wherein the preset value is the upper limit value of the value range corresponding to the data type of the data set to be inferred;

[0033] If the judgment result indicates that no scene switching occurs, the output scaling factor corresponding to the previous input data set of the data set to be inferred is used as the output scaling factor of the data set to be inferred.

[0034] In a second aspect, the present application provides an inference device for a deep learning model, comprising:

[0035] A first acquisition module is used to acquire a data set to be inferred;

[0036] A first reasoning module, used for inputting the data set to be inferred into the reasoning convolution kernel to obtain a floating-point data reasoning result;

[0037] A second acquisition module, used to acquire an output scaling factor corresponding to the data set to be inferred; wherein the output scaling factor is determined according to the maximum value of the elements in the data inference result;

[0038] A quantization module, used for calculating the product of the data inference result and the output scaling factor to obtain a quantization result;

[0039] The second reasoning module is used to continue the reasoning of the deep learning model according to the quantization result.

[0040] In a third aspect, the present application provides an electronic device comprising: a processor, a memory and a communication bus, wherein the processor and the memory communicate with each other through the communication bus; the memory is used to store computer programs; the processor is used to execute the programs stored in the memory to implement the inference method of the deep learning model described in the first aspect.

[0041] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the reasoning method of the deep learning model described in the first aspect.

[0042] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the method provided by the embodiment of the present application can obtain an output scaling factor that matches the data set to be inferred for different data sets to be inferred, and can dynamically generate an output scaling factor according to the data set to be inferred input during actual application, and then multiply the output scaling factor by the floating-point data inference result obtained by inputting the data set to be inferred into the inference convolution kernel, thereby achieving a balance between int8 integer quantization speed and quality, that is, the method provided by the embodiment of the present application can not only improve the inference efficiency of the deep learning network model, but also ensure the inference quality, and can effectively alleviate the problem of relatively large overall errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0044] Figure 1 A schematic diagram of mapping floating point data to integer data provided in an embodiment of the present application;

[0045] Figure 2 A flowchart of a method for reasoning a deep learning model provided in an embodiment of the present application;

[0046] Figure 3 A flowchart of a method for obtaining floating-point data inference results provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of the structure of the int8 integer convolution kernel provided in an embodiment of the present application;

[0048] Figure 5 A schematic diagram of the structure of an inference device for a deep learning model provided in an embodiment of the present application;

[0049] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0051] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0052] In the process of deep learning training, for the overall target performance, the current mainstream method is to use floating point float32 data for training, where float represents the floating point data type, and the number 32 after float represents the number of bits occupied. However, when it comes to model deployment, for production efficiency, methods such as int8 integer quantization will be used to speed up the corresponding reasoning; where int represents the integer data type, and the number 8 after int represents the number of bits occupied. For the mapping of floating point to integer, the mainstream method is to use a scaling factor, such as Figure 1 As shown, the floating-point data is multiplied by the scaling factor to expand it to the integer range of [-128, 127], and then the expanded floating-point data is further rounded using the rounding method to obtain the final int8 type integer value, for example: the int8 type integer value after the floating-point data -0.5 is mapped is -128; the int8 type integer value after the floating-point data 1.5 is mapped is 128; the int8 type integer value after the floating-point data 0 is mapped is -Z; the int8 type integer value after the floating-point data SZ is mapped is 0.

[0053] Subsequently, this int8 type integer value is used as input and input into the inference convolution kernel (inferenceconv kernel) for inference to improve the inference efficiency of the model.

[0054] However, in actual applications, in order to ensure the overall speed, although the input data type is mapped to the int8 type, in order to ensure a certain level of accuracy, in the inference process inside the inference convolution kernel, for example, in the addition operation after the multiplication calculation and the offset addition operation, the floating-point data result is still retained. In order to ensure that the output result is int8 type, it is necessary to perform the relu function and round operation on the basis of the floating-point data inference result. If the value is between 0 and 1, the result is 0 after the round operation. Therefore, the method provided by the prior art sacrifices accuracy in order to improve speed, resulting in a large overall error.

[0055] In order to solve the problem of large overall error caused by quantization in the process of model performance optimization in the prior art, an embodiment of the present application provides an inference method for a deep learning model, which can be applied to a server or a cloud; wherein the server or the cloud stores a deep learning network model, and the deep learning network model includes an inference convolution kernel.

[0056] like Figure 2As shown, the inference method of a deep learning model provided in an embodiment of the present application specifically includes the following steps:

[0057] Step 201, obtaining a data set to be inferred;

[0058] The data set to be inferred may be a matrix mapped from floating-point data to an integer data type. The specific method for mapping from floating-point data to an integer data type may refer to the mapping method in the above-mentioned prior art. On the basis of each floating-point element of the floating-point input matrix, the first scaling factor is multiplied, and each floating-point element is mapped to an integer element to obtain an integer type matrix as the data set to be inferred. The data set to be inferred is the input of the inference convolution kernel.

[0059] For example, taking the int8 integer convolution kernel running on NVIDIA GPU as an example, the components of the convolution kernel include: input (activation), weight (weight) and bias (bias), where the input (activation) is the data set to be inferred in actual applications.

[0060] Step 202, input the data set to be inferred into the inference convolution kernel to obtain a floating-point data inference result;

[0061] Specifically, Figure 3 As shown in FIG. 1 , the method of inputting the data set to be inferred into the inference convolution kernel and obtaining the floating-point data inference result mainly includes the following steps:

[0062] Step 301, obtaining a first scaling factor corresponding to the data set to be inferred;

[0063] The actual input data set is of floating-point type. In order to improve the inference speed, the floating-point input data set is converted into an integer type input data set as the data set to be inferred. The floating-point input data set is multiplied by the first scaling factor to obtain the data set to be inferred. Conversely, when the floating-point input data set corresponding to the data set to be inferred is known, the data set to be inferred is divided by the floating-point input data set to obtain the first scaling factor.

[0064] Step 302, obtaining a weight corresponding to the inference convolution kernel and a second scaling factor corresponding to the weight;

[0065] Each inference convolution kernel corresponds to a weight, and the weight is usually constant for a certain inference convolution kernel, and the second scaling factor corresponding to the weight is also fixed.

[0066] Step 303, obtaining the bias of the inference convolution kernel;

[0067] Bias is also an important parameter in the inference convolution kernel. For a certain inference convolution kernel, the bias is also fixed.

[0068] Step 304 , calculating a floating-point data inference result according to the data set to be inferred, the first scaling factor, the weight, the second scaling factor and the bias.

[0069] Specifically, given the data set to be inferred, the first scaling factor, the weight, the second scaling factor and the bias, the floating-point data inference result is calculated according to the operation inside the inference convolution kernel. Taking the int8 integer convolution kernel running on the NVIDIA GPU as an example, the floating-point data inference result is calculated according to the data set to be inferred, the first scaling factor, the weight, the second scaling factor and the bias, including:

[0070] Calculate the product of the data set to be inferred and the weight to obtain a first intermediate result; divide the first intermediate result by the first scaling factor and the second scaling factor to obtain a second intermediate result; calculate the sum of the second intermediate result and the bias to obtain a floating-point data inference result.

[0071] Assume the input is I Q , with weight W Q , the first scaling factor corresponding to the input is S I , the second scaling factor corresponding to the weight is S W , the bias is B, then the floating point data inference result is I Q *W Q / S I / S W +B, at this time, the output inference result is floating point type.

[0072] Step 203, obtaining an output scaling factor corresponding to the data set to be inferred; wherein the output scaling factor is determined according to the maximum value of the elements in the data inference result;

[0073] Specifically, a maximum element is determined from each element of the data inference result of the data set to be inferred; and a quotient of a preset value and the maximum element is calculated to obtain an output scaling factor.

[0074] In specific implementation, the preset value can be set to 127. Taking the input data set as an input matrix as an example, the data inference result is also a matrix. Suppose the maximum value of each matrix element in the data inference result is MAX output , then the output scaling factor is So = 127 / MAX output .

[0075] Calculating the output scaling factor with the maximum value of the matrix elements can ensure that other matrix elements can also meet the needs.

[0076] In the embodiment of the present application, the output scaling factor matches the data set to be inferred, rather than reusing the same coefficient for each inference as in the prior art. In the embodiment of the present application, the floating-point data inference result is remapped to the int8 type by outputting the scaling factor, which not only ensures the inference efficiency of the deep learning model, but also ensures the inference quality.

[0077] Step 204, calculating the product of the data inference result and the output scaling factor to obtain a quantized result;

[0078] Based on the floating-point data inference result, the output scaling factor is multiplied to map the floating-point output result to the int8 type. The int8 type data is faster than the float32 type inference speed.

[0079] Step 205, continue the reasoning of the deep learning model according to the quantization result.

[0080] The subsequent reasoning process of different deep learning models may be different. For example, the subsequent reasoning process may be through the relu function and round operation.

[0081] In order to further highlight the difference between the inference method of the deep learning model provided in the embodiment of the present application and the prior art, the inference method of the deep learning model provided in the prior art is described as follows:

[0082] Take a data set, and then use this data set to fine-tune the entire quantization process to obtain a coefficient that performs well on the data set. Later, when the model is deployed, this coefficient is reused for each inference. However, the problem with this approach is that the data encountered in the actual model inference process may be very different in data distribution from the data set used for fine-tuning. If this happens, the coefficient obtained based on the data set used for fine-tuning can no longer be used. If it is forcibly used, it will directly affect the quality of inference.

[0083] In the embodiment of the present application, for different data sets to be inferred, an output scaling factor that matches the data set to be inferred can be obtained, and the output scaling factor can be dynamically generated according to the data set to be inferred during actual application, and then multiplied by the floating-point data inference result obtained by inputting the data set to be inferred into the inference convolution kernel, so as to achieve a balance between int8 integer quantization speed and quality. That is, the method provided in the embodiment of the present application can not only improve the inference efficiency of the deep learning network model, but also ensure the inference quality, and can effectively alleviate the problem of relatively large overall errors.

[0084] In order to facilitate understanding of the inventive concept of the embodiment of the present application, the int8 integer convolution kernel running on NVIDIA GPU is also taken as an example. For the deep learning network model, the components of the convolution kernel include: input (activation), weight (weight) and bias (bias), wherein the input (activation) is the data set to be inferred in practical applications, that is, the target input matrix. In order to ensure the overall speed while ensuring a considerable level of accuracy, the int8 kernel implementation process of NVIDIA retains the floating point float form in the final addition process with the bias.

[0085] Since the input and weight are of type int8, they each have a scaling factor, defined here as S I (first scaling factor) and S W (second scaling factor), and for int8 input and weights, they are defined as I Q and W Q , the floating point bias is defined as B, and for the final output in int8 form, it also has a corresponding scaling factor (output scaling factor), defined as S O . After Figure 4 The reasoning process described above can obtain an output in the form of int8. The reasoning process for the convolution kernel can be expressed as: (I Q *W Q / S I / S W +B)*S O (1); can be expanded to: I Q *W Q *S O / S I / S W +B*S O (2).

[0086] However, the scaling factor corresponding to the final output cannot be known in advance. So here we change the idea, if the output is not in the form of int8, but in the form of floating point, the scaling factor of the output is no longer needed, and dynamic calculation of the scaling factor becomes possible.

[0087] In order to achieve this goal, So is first set to 1, then formula (1) is expressed as:

[0088] I Q *W Q / S I / S W +B(3)

[0089] The result obtained by formula (3) is used as the floating-point data inference result. Among them, the scaling factor S corresponding to the weight W , the int8 integer convolution kernel is unchanged, and the corresponding input scaling factor S I It can be calculated.

[0090] After obtaining the floating-point data inference result, the output scaling factor is obtained by determining the maximum matrix element among the matrix elements and calculating the quotient of the preset value and the maximum matrix element.

[0091] In the specific implementation, the preset value can be set to 127. Assume that the maximum value of each matrix element in the data inference result is MAX output , then the output scaling factor is So = 127 / MAX output .

[0092] In actual applications, for the same video, if the characteristics of the input data before and after do not change much (for example, the color does not change much and the style is similar), the output scaling factor can continue to be reused, and there is no need to recalculate the scaling factor for each inference, which greatly shortens the time required for inference.

[0093] In actual application, in order to further improve the accuracy, it can be determined whether a scene switch occurs based on the two frames of video before and after the video to decide whether to reuse the output scaling factor.

[0094] Specifically, if a scene switch occurs, the maximum element is determined from each element of the data inference result; the quotient of the preset value and the maximum element is calculated to obtain the output scaling factor; wherein the preset value is the upper limit of the value range corresponding to the data type of the data set to be inferred; for example, the value range of the int data type is -128 to 127, and the preset value is 127; if a scene switch does not occur, the output scaling factor corresponding to the previous input data set of the data set to be inferred is used as the output scaling factor of the data set to be inferred. wherein the previous input data set is an input data set determined according to the previous frame of the current video frame.

[0095] In addition, it should be noted that in the specific implementation, each frame of video corresponds to an input matrix, which is the input data set. The input matrix can be a matrix that represents the characteristics of the video frame, and the data type of the input matrix is ​​converted. The converted result is used as the input of the inference convolution kernel.

[0096] The method for determining whether a scene switch occurs based on two adjacent video frames mentioned in the embodiment of the present application can adopt any method for determining whether a scene switch occurs in the prior art, and is not limited here.

[0097] In an embodiment of the present application, the output scaling factor is determined based on whether the scene is switched. If the scene is switched, the scaling factor is re-determined based on the data set to be inferred. If the scene is not switched, in order to minimize the time used for inference, the scaling factor of the previous input data set can be reused.

[0098] Based on the same concept, an inference device for a deep learning model is provided in the embodiment of the present application. The specific implementation of the device can be found in the description of the method embodiment part, and the repeated parts will not be repeated. Figure 5 As shown, the device mainly includes:

[0099] A first acquisition module 501 is used to acquire a data set to be inferred;

[0100] A first reasoning module 502, used for inputting the data set to be inferred into the reasoning convolution kernel to obtain a floating-point data reasoning result;

[0101] A second acquisition module 503 is used to acquire an output scaling factor corresponding to the data set to be inferred; wherein the output scaling factor is determined according to the maximum value of the elements in the data inference result;

[0102] A quantization module 504, configured to calculate the product of the data inference result and the output scaling factor to obtain a quantization result;

[0103] The second reasoning module 505 is used to continue the reasoning of the deep learning model according to the quantization result.

[0104] In an embodiment of the present application, for different data sets to be inferred, an output scaling factor matching the data set to be inferred can be obtained, and the output scaling factor can be dynamically generated according to the data set to be inferred during actual application, and then multiplied by the floating-point data inference result obtained by inputting the data set to be inferred into the inference convolution kernel, so as to achieve a balance between int8 integer quantization speed and quality, that is, the method provided in the embodiment of the present application can not only improve the inference efficiency of the deep learning network model, but also ensure the inference quality, and can effectively alleviate the problem of relatively large overall errors.

[0105] In a specific embodiment, the second acquisition module 503 is used to determine the maximum element from the elements of the data inference result of the data set to be inferred; calculate the quotient of a preset value and the maximum element to obtain the output scaling factor.

[0106] In a specific embodiment, the first reasoning module 502 is used to obtain a first scaling factor corresponding to the data set to be inferred; obtain a weight corresponding to the reasoning convolution kernel and a second scaling factor corresponding to the weight; obtain a bias of the reasoning convolution kernel; and calculate the floating-point data inference result based on the data set to be inferred, the first scaling factor, the weight, the second scaling factor and the bias.

[0107] In a specific embodiment, the first reasoning module 502 is used to calculate the product of the data set to be inferred and the weight to obtain a first intermediate result; divide the first intermediate result by the first scaling factor and the second scaling factor to obtain the second intermediate result; calculate the sum of the second intermediate result and the bias to obtain the floating-point data inference result.

[0108] In a specific embodiment, the first reasoning module 502 is used to obtain a floating-point input data set corresponding to the data set to be inferred; and obtain the first scaling factor by dividing the data set to be inferred by the floating-point input data set.

[0109] In a specific embodiment, the first acquisition module 501 is used to determine the data set to be inferred corresponding to the current video frame from the target video; the second acquisition module 503 is used to obtain the output scaling factor corresponding to the data set to be inferred; determine whether the current video frame has a scene switch relative to the previous video frame to obtain a judgment result; and determine the output scaling factor corresponding to the data set to be inferred based on the judgment result.

[0110] In a specific embodiment, the second acquisition module 503 is used to determine the maximum element from the various elements of the data inference result if the judgment result indicates that a scene switch has occurred; calculate the quotient of a preset value and the maximum element to obtain the output scaling factor; wherein the preset value is the upper limit value of the value range corresponding to the data type of the data set to be inferred; if the judgment result indicates that no scene switch has occurred, the output scaling factor corresponding to the previous input data set of the data set to be inferred is used as the output scaling factor of the data set to be inferred.

[0111] In an embodiment of the present application, the output scaling factor is determined based on whether the scene is switched. If the scene is switched, the scaling factor is re-determined based on the data set to be inferred. If the scene is not switched, in order to minimize the time used for inference, the scaling factor of the previous input data set can be reused.

[0112] Based on the same concept, an electronic device is also provided in the embodiment of the present application, such as Figure 6As shown, the electronic device mainly includes: a processor 601, a memory 602 and a communication bus 603, wherein the processor 601 and the memory 602 communicate with each other through the communication bus 603. The memory 602 stores a program executable by the processor 601, and the processor 601 executes the program stored in the memory 602 to implement the following steps:

[0113] Get the data set to be inferred;

[0114] Inputting the data set to be inferred into the inference convolution kernel to obtain a floating-point data inference result;

[0115] Obtaining an output scaling factor corresponding to the data set to be inferred; wherein the output scaling factor is determined according to the maximum value of the elements in the data inference result;

[0116] Calculate the product of the floating point data inference result and the output scaling factor to obtain a quantized result;

[0117] The deep learning model is continued to be inferred according to the quantization result.

[0118] The communication bus 603 mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus 603 can be divided into an address bus, a matrix bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0119] The memory 602 may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the processor 601.

[0120] The above-mentioned processor 601 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., and can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0121] In another embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program. When the computer program runs on a computer, the computer executes the reasoning method of a deep learning model described in the above embodiment.

[0122] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instruction can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instruction is transmitted from a website site, a computer, a server or a matrix center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, microwave, etc.) mode to another website site, computer, server or matrix center. The computer-readable storage medium can be any available medium that a computer can access or a matrix storage device such as a server, a matrix center, etc. that contains one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape, etc.), an optical medium (e.g., a DVD) or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0123] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0124] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for reasoning a deep learning model, characterized in that: include: Get the data set to be inferred; Inputting the data set to be inferred into the inference convolution kernel to obtain a floating-point data inference result; Obtaining an output scaling factor corresponding to the data set to be inferred; wherein the output scaling factor is determined according to the maximum value of the elements in the data inference result; Calculating the product of the data inference result and the output scaling factor to obtain a quantized result; Continue reasoning of the deep learning model according to the quantization result; Wherein, obtaining the data set to be inferred includes: From the target video, determine the data set to be inferred corresponding to the current video frame; The obtaining the output scaling factor corresponding to the data set to be inferred includes: Determine whether a scene switch occurs in the current video frame relative to the previous video frame, and obtain a determination result; If the judgment result indicates that a scene switch occurs, then determine the maximum element from the elements of the data inference result; calculate the quotient of a preset value and the maximum element to obtain the output scaling factor; wherein the preset value is the upper limit value of the value range corresponding to the data type of the data set to be inferred; If the judgment result indicates that no scene switching occurs, the output scaling factor corresponding to the previous input data set of the data set to be inferred is used as the output scaling factor of the data set to be inferred.

2. The inference method of the deep learning model according to claim 1, characterized in that: The step of inputting the data set to be inferred into the inference convolution kernel to obtain a floating-point data inference result includes: Obtaining a first scaling factor corresponding to the data set to be inferred; Obtaining a weight corresponding to the inference convolution kernel and a second scaling factor corresponding to the weight; Obtaining a bias for the inference convolution kernel; The floating-point data inference result is calculated according to the data set to be inferred, the first scaling factor, the weight, the second scaling factor and the bias.

3. The inference method of the deep learning model according to claim 2, characterized in that: The calculating the floating-point data inference result according to the data set to be inferred, the first scaling factor, the weight, the second scaling factor and the bias includes: Calculating the product of the data set to be inferred and the weight to obtain a first intermediate result; Dividing the first intermediate result by the first scaling factor and the second scaling factor to obtain a second intermediate result; The sum of the second intermediate result and the bias is calculated to obtain the floating-point data inference result.

4. The inference method of the deep learning model according to claim 2, characterized in that: The obtaining a first scaling factor corresponding to the data set to be inferred includes: Obtaining a floating-point input data set corresponding to the data set to be inferred; The first scaling factor is obtained by dividing the data set to be inferred by the floating-point input data set.

5. A reasoning device for a deep learning model, characterized in that: include: A first acquisition module is used to acquire a data set to be inferred; A first reasoning module, used for inputting the data set to be inferred into the reasoning convolution kernel to obtain a floating-point data reasoning result; A second acquisition module, used to acquire an output scaling factor corresponding to the data set to be inferred; wherein the output scaling factor is determined according to the maximum value of the elements in the data inference result; A quantization module, used for calculating the product of the data inference result and the output scaling factor to obtain a quantization result; A second reasoning module, used to continue reasoning of the deep learning model according to the quantization result; The first acquisition module is used to determine the data set to be inferred corresponding to the current video frame from the target video; The second acquisition module is used to obtain the output scaling factor corresponding to the data set to be inferred; determine whether the current video frame has a scene switch relative to the previous video frame to obtain a determination result; and determine the output scaling factor corresponding to the data set to be inferred according to the determination result; The second acquisition module is specifically used for: if the judgment result indicates that a scene switch has occurred, determining the maximum element from the various elements of the data inference result; calculating the quotient of a preset value and the maximum element to obtain the output scaling factor; wherein the preset value is the upper limit value of the value range corresponding to the data type of the data set to be inferred; if the judgment result indicates that no scene switch has occurred, using the output scaling factor corresponding to the previous input data set of the data set to be inferred as the output scaling factor of the data set to be inferred.

6. An electronic device, characterized in that: include: A processor, a memory and a communication bus, wherein the processor and the memory communicate with each other via the communication bus; the memory is used to store computer programs; the processor is used to execute the programs stored in the memory to implement the inference method of the deep learning model described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the inference method of the deep learning model described in any one of claims 1 to 4 is implemented.