Data processing method and device, electronic equipment and readable storage medium

Abstract

The invention discloses a data processing method and device, electronic equipment and a readable storage medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining a to-be-quantized tensor of a neural network model disposed at a mobile terminal side; the tensor to be quantized is converted into a binary number representation mode, a first floating point tensor is obtained, and the first floating point tensor comprises a symbol part, an index part and a decimal part; determining a quantized tensor after the tensor to be quantized is quantized through the symbol part, the index part and the decimal part; and storing the quantized tensor. The demand of a neural network model on a mobile terminal side for computing resources can be reduced, and the power consumption of equipment on the mobile terminal side is reduced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a data processing method, device, electronic device, and readable storage medium. Background Art

[0002] With the development of artificial intelligence (AI) technology, neural network models can be deployed on mobile devices to solve specific problems. However, these models currently require significant storage and computing resources, resulting in high power consumption on mobile devices. Summary of the Invention

[0003] The present application proposes a data processing method, device, electronic device and readable storage medium.

[0004] In a first aspect, an embodiment of the present application provides a data processing method, including: obtaining a tensor to be quantized of a neural network model deployed on a mobile terminal side; converting the tensor to be quantized into a binary number representation to obtain a first floating-point tensor, wherein the first floating-point tensor includes a sign part, an exponent part, and a decimal part; determining a quantized tensor after quantizing the tensor to be quantized through the sign part, the exponent part, and the decimal part; and storing the quantized tensor.

[0005] In a second aspect, an embodiment of the present application further provides a data processing device, comprising: an acquisition unit, a first conversion unit, a quantization unit, and a storage unit. The acquisition unit is used to acquire a tensor to be quantized of a neural network model deployed on a mobile terminal; the first conversion unit is used to convert the tensor to be quantized into a binary representation to obtain a first floating-point tensor, wherein the first floating-point tensor includes a sign part, an exponent part, and a decimal part; the quantization unit is used to determine a quantized tensor after quantizing the tensor to be quantized using the sign part, the exponent part, and the decimal part; and the storage unit is used to store the quantized tensor.

[0006] In a third aspect, an embodiment of the present application further provides an electronic device comprising: one or more processors; a memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method described in the first aspect.

[0007] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which program code is stored. The program code can be called by a processor to execute the method described in the first aspect above.

[0008] The data processing method, device, electronic device and readable storage medium provided by the present application first obtain the tensor to be quantized of the neural network model deployed on the mobile side; then convert the tensor to be quantized into a binary number representation to obtain a first floating-point tensor; then determine the quantized tensor after quantizing the tensor to be quantized through the sign part, exponent part and decimal part; and store the quantized tensor. By storing the quantized tensor, not only can the storage space occupied by the neural network model on the mobile side be compressed to reduce the demand for storage space on the mobile side; but also when performing calculations through the mobile neural network model, the demand for computing resources of the neural network model can be further reduced, thereby reducing the power consumption of the device on the mobile side.

[0009] Other features and advantages of the embodiments of the present application will be described in the following description and, in part, will become apparent from the description or be understood by practicing the embodiments of the present application. The objectives and other advantages of the embodiments of the present application can be achieved and obtained through the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 An application scenario diagram of the data processing method provided in an embodiment of the present application is shown;

[0012] Figure 2 A flow chart of a data processing method according to an embodiment of the present invention is shown;

[0013] Figure 3 A flow chart showing a data processing method provided by another embodiment of the present application is shown;

[0014] Figure 4 A schematic diagram showing a first floating-point number represented in binary format according to an embodiment of the present application is shown;

[0015] Figure 5 A flow chart showing a data processing method provided by another embodiment of the present application is shown;

[0016] Figure 6 A structural block diagram of a data processing device provided in an embodiment of the present application is shown;

[0017] Figure 7A structural block diagram of an electronic device provided in an embodiment of the present application is shown;

[0018] Figure 8 A structural block diagram of a computer-readable storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work fall within the scope of protection of the present application.

[0020] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0021] With the development of artificial intelligence (AI) technology, neural network models can be deployed on mobile devices to solve specific problems. However, these models currently require significant storage and computing resources, resulting in high power consumption on mobile devices. Reducing the storage and computing resource requirements of mobile neural network models, and thus reducing power consumption, is a pressing issue.

[0022] Currently, model quantization can be used to lightweight deploy neural network models deployed on mobile devices. Mobile devices can be electronic devices such as smartphones, smart tablets, laptops, and in-vehicle computers.

[0023] However, the inventors found in their research that after the neural network model is processed using the existing quantization method, when the neural network model is needed for subsequent calculations, the inverse quantization process still requires a large amount of computing resources and storage space, resulting in higher power consumption of the device on the mobile side.

[0024] Therefore, in order to overcome the above-mentioned defects, the present application provides a data processing method, device, electronic device and readable storage medium.

[0025] See also Figure 1 , Figure 1 An application scenario diagram of a data processing method is shown, namely a data processing scenario 100 , which includes an electronic device 110 and a user 120 .

[0026] exist Figure 1 The electronic device 110 shown in the figure is a smart phone. The user 120 can use the electronic device 110. The electronic device 110 can store a neural network model, which is a quantized model. For the relevant introduction of quantization, please refer to the subsequent embodiments of this application.

[0027] When the user 120 uses the electronic device 110, the stored neural network model can be run to perform corresponding operations through the neural network model. Similarly, a detailed description of the corresponding operations performed through the neural network model can also be found in the subsequent embodiments.

[0028] See also Figure 2 , Figure 2 A flow chart of a data processing method provided by an embodiment of the present application is shown. The data processing method can be applied to Figure 1 In the electronic device 110 shown in FIG, the processor in the electronic device 110 can be used as the execution subject. The data processing method specifically includes steps S110 to S140.

[0029] Step S110: Obtain the tensor to be quantized of the neural network model deployed on the mobile terminal side.

[0030] It should be noted that the neural network model on the mobile side is a neural network model deployed in an electronic device on the mobile side, and the tensor to be quantized can be a tensor corresponding to the data that needs to be compressed in the neural network model. For example, the data can be the weight value of the neural network model, or it can be the activation value of the neural network model. Therefore, the tensor to be quantized can be a tensor corresponding to the weight value of the neural network model, or it can be a tensor corresponding to the activation value, etc.

[0031] A tensor is a multidimensional array, generally having two basic attributes: shape and data type (dtype). In a neural network model, the data type of the tensors corresponding to weight values ​​and activation values ​​is generally a floating-point data type. That is, in some embodiments, the tensor to be quantized is a tensor obtained by combining data of floating-point data types.

[0032] For example, the tensor to be quantized can be [1.1, 1.2, 1.3, 1.4]. That is, the tensor to be quantized can be integrated into a set using "[]", which specifically includes four floating-point numbers to be quantized, namely 1.1, 1.2, 1.3 and 1.4, and each floating-point number to be quantized is separated by ",".

[0033] The neural network model deployed on the mobile side can be a pre-trained model, such as a deep learning model, specifically a large language model or a CV model. It is understandable that different initial models can be determined according to different specific needs in actual applications, and then the initial models can be trained through different training samples to obtain different neural network models that can meet various needs. Exemplarily, the neural network model can be used to process images in electronic devices on the mobile side, such as image recognition, image enhancement, image generation, etc.; for another example, the neural network model can also be used to process text in electronic devices on the mobile side, such as text recognition, text semantic understanding, text semantic summary extraction, etc.

[0034] Step S120: converting the tensor to be quantized into a binary number representation to obtain a first floating-point tensor, wherein the first floating-point tensor includes a sign part, an exponent part, and a decimal part.

[0035] It is understandable that if the tensor of the floating-point data type is directly stored, the storage space occupied is large. Therefore, the tensor to be quantized can be quantized to reduce the storage space required to store the tensor to be quantized.

[0036] Floating-point numbers can be represented by binary numbers, so each floating-point number to be quantized in the tensor to be quantized can be represented by binary numbers.

[0037] For some implementations, the manner in which the binary number is represented may be determined by a preset formula.

[0038] Specifically, if r is used to represent a floating point number, the following formula (1) is the preset formula, and the floating point number r can be represented by the following formula (1).

[0039] r=―1 s ×2 x―bias ×1.y (1)

[0040] It can be seen that the floating point number represented by formula (1) consists of three parts: the sign part, the specified part and the decimal part. s Used to represent the symbol part, 2 x―biasThe value 1.y represents the exponent, while the value 1.y represents the decimal. S, X, and y are all binary integers. Therefore, the value of S determines the sign, X combined with the bias value determines the exponent, and the value of y determines the decimal. Bias is the offset specified in IEEE-754. The specific value of bias can vary for different floating-point types. For example, for a 32-bit floating-point number, the bias can be 127.

[0041] It should be noted that the method of converting a floating point number into a binary number representation is to use formula (1) to represent the floating point number with three binary integer data S, X and y.

[0042] That is, the tensor to be quantized can be converted into a binary representation using the above formula (1) to obtain a first floating-point tensor. For example, each floating-point number to be quantized in the tensor to be quantized can be converted into the binary representation to obtain the first floating-point tensor. Thus, the first floating-point tensor can correspond to a signed part, a specified part, and a decimal part.

[0043] Step S130: determining a quantized tensor obtained by quantizing the tensor to be quantized through the sign part, the exponent part, and the decimal part.

[0044] It can be understood that the first floating-point tensor represented as a binary number representation includes the binary number representation corresponding to each floating point to be quantized, that is, each floating-point tensor to be quantized represented by a binary number can correspond to a signed part, a specified part, and a decimal part.

[0045] Exemplarily, the first floating-point tensor may include the sign part A1, the specified part A2, and the decimal part A3 corresponding to the floating point A to be quantized; also include the sign part B1, the specified part B2, and the decimal part B3 corresponding to the floating point B to be quantized; and also include the sign part C1, the specified part C2, and the decimal part C3 corresponding to the floating point C to be quantized.

[0046] Therefore, in some implementations, the sign part, the specified part, and the decimal part corresponding to each floating-point number to be quantized in the first floating-point tensor can be directly used as a quantized tensor after quantizing the tensor to be quantized.

[0047] Optionally, in the process of converting the tensor to be quantized into a binary representation to obtain the first floating-point tensor, the first floating-point tensor can be first processed to obtain a third floating-point tensor, and then the three floating-point tensors can be converted into a binary representation to obtain the first floating-point tensor. As a result, the sign part and the specified part corresponding to each floating-point number to be quantized in the first floating-point tensor may be the same. In this case, only one identical sign part and specified part can be stored, thereby further reducing the storage space required by the neural network model. For a detailed description, please refer to the subsequent embodiments.

[0048] Step S140: storing the quantized tensor.

[0049] After obtaining the quantized tensor, the quantized tensor can be stored in a mobile terminal, where the mobile terminal can be an electronic device on the mobile side. Since the tensor to be quantized is the tensor corresponding to the data in the neural network model deployed on the mobile side, the quantized tensor obtained after quantizing the tensor to be quantized can reduce the storage space requirement of the neural network model deployed on the mobile side.

[0050] Optionally, the quantized tensor can be dequantized to perform related calculations through the neural network model. Among them, the neural network model on the mobile side is a neural network model deployed in the electronic device on the mobile side. Since the mobile side generally has relatively limited computing resources, storage space, etc., the demand for storage space of the neural network model can be reduced by quantization, and then combined with dequantization to reduce the demand for computing resources for the use of the neural network model. However, traditional quantization and dequantization methods still have high demands on computing resources.

[0051] Therefore, in an embodiment of the present application, a method is proposed that can reduce the size of the storage space occupied by the neural network model. At the same time, when the neural network model is used for subsequent calculations, the model's demand for computing resources can be further reduced. The method first obtains the tensor to be quantized of the neural network model deployed on the mobile side; then converts the tensor to be quantized into a binary number representation to obtain a first floating-point tensor; then determines the quantized tensor after quantizing the tensor to be quantized through the sign part, the exponent part and the decimal part; and stores the quantized tensor. By storing the quantized tensor, not only can the storage space occupied by the neural network model on the mobile side be compressed to reduce the demand for storage space on the mobile side; subsequently, when the mobile neural network model is used for calculations, the demand for computing resources of the neural network model can be further reduced, thereby reducing the power consumption of the device on the mobile side.

[0052] See also Figure 3 , Figure 3A flow chart of a data processing method provided by an embodiment of the present application is shown. The data processing method can be applied to Figure 1 In the electronic device 110 shown in FIG, the processor in the electronic device 110 can be used as the execution subject. The data processing method specifically includes steps S210 to S270.

[0053] Step S210: Obtain the tensor to be quantized of the neural network model deployed on the mobile terminal side.

[0054] Among them, step S210 has been described in detail in the above embodiment and will not be repeated here.

[0055] Step S220: respectively determining the maximum floating point number and the minimum floating point number among the plurality of floating point numbers to be quantized.

[0056] As can be seen from the above description, the tensor to be quantized may include multiple floating-point numbers to be quantized. Therefore, the maximum floating-point number and the minimum floating-point number among the multiple floating-point numbers to be quantized can be determined at this time.

[0057] In some embodiments, the max To represent the maximum floating point number, through r min To represent the minimum floating point number. For example, if the tensor to be quantized includes four floating point numbers to be quantized, namely 1.1, 2.1, 3.1 and 4.1. Then the maximum floating point number r can be determined max is 4.1, the minimum floating point number r min is 1.1.

[0058] Step S230: Convert the tensor to be quantized into a third floating-point tensor based on the maximum floating-point number and the minimum floating-point number.

[0059] Then, the tensor to be quantized can be converted into a third floating-point tensor based on the maximum floating-point number and the minimum floating-point number. The third floating-point tensor can be obtained by performing data processing on each floating-point number to be quantized in the tensor to be quantized. In other words, the number of floating-point numbers included in the third floating-point tensor is the same as the number of floating-point numbers to be quantized included in the tensor to be quantized.

[0060] In some implementations, step S230 may include steps S231 to S233.

[0061] Step S231: Obtain a scaling factor based on the maximum floating-point number, the minimum floating-point number, the pre-constructed first intermediate number, and the pre-constructed second intermediate number.

[0062] Step S232: Obtain a bias coefficient based on the second intermediate number, the scaling factor, and the maximum floating-point number.

[0063] Step S233: Obtain a third floating-point tensor based on the tensor to be quantized, the scaling coefficient, and the bias coefficient.

[0064] In some implementations, the quantized tensor can be processed using a scaling factor and a bias factor to obtain a third floating-point tensor. The scaling factor can be obtained based on the maximum floating-point number, the minimum floating-point number, a pre-constructed first intermediate number, and a pre-constructed second intermediate number; and the bias factor can be obtained using the second intermediate number, the scaling factor, and the maximum floating-point number. For a detailed description, please refer to the subsequent steps.

[0065] Therefore, after obtaining the scaling coefficient and the bias coefficient, the third floating-point tensor can be further determined.

[0066] Among them, step S233 can also include step S2331 to step S2332.

[0067] Step S2331: taking the quotient of the tensor to be quantized and the scaling coefficient as the eighth intermediate number.

[0068] Step S2332: taking the sum of the eighth intermediate number and the bias coefficient as the third floating-point tensor.

[0069] In some embodiments, if r represents the tensor to be quantized, S represents the scaling factor, and Z represents the bias factor, then the quotient of the tensor to be quantized and the scaling factor can be used as the eighth intermediate number.

[0070] It is understandable that there can be multiple eighth intermediate numbers, that is, each floating-point number to be quantized in the tensor to be quantized can be divided by the scaling coefficient, so that the number obtained by dividing each floating-point number to be quantized by the scaling coefficient is used as the eighth intermediate number.

[0071] Furthermore, the sum of the eighth intermediate number and the bias coefficient can be used as the third floating-point tensor. If r1 represents the third floating-point tensor, the third floating-point tensor can be represented by the following formula (2).

[0072]

[0073] It should be noted that, each floating-point number to be quantized in the quantized tensor can be processed by the above formula (2), thereby obtaining a plurality of processed floating-point numbers to be quantized as the third floating-point tensor.

[0074] In some implementations, before executing step S231 , steps S2311 to S2313 may be further executed to obtain a second intermediate number.

[0075] Step S2311: Use 2 as the base and a first specified number as the exponent to obtain the first intermediate number, where the first specified number is an integer.

[0076] Step S2312: Use 2 as the base and the second specified number as the exponent to obtain a third intermediate number, where the second specified number is the first specified number plus 1.

[0077] Step S2313: taking the difference between the third intermediate number and the fourth intermediate number as the second intermediate number, where the fourth intermediate number is greater than 0 and less than a specified value.

[0078] For some embodiments, q min To characterize the first intermediate number, wherein the first intermediate number can be represented by q min =2 N To determine the first intermediate number, 2 can be used as the base and the first specified number as the exponent to obtain the first intermediate number. Where N is the first specified number, which is an integer. For example, the first specified number can be 1, 2, 3, -1, or -2.

[0079] Furthermore, the third middle number can be determined. Specifically, it can be determined by 2 N+1 To calculate the third intermediate number, use 2 as the base and the second specified number as the exponent to obtain the third intermediate number. Here, N+1 represents the second specified number, which is the first specified number plus 1.

[0080] Then, the difference between the third intermediate number and the fourth intermediate number can be used as the second intermediate number, and the fourth intermediate number is greater than 0 and less than the specified value. The fourth intermediate number is used to make the third floating-point tensor as close to the third intermediate number as possible and less than the third intermediate number. Therefore, the specified value can be a smaller number greater than 0 and close to 0, such as 0.0001, 0.001, etc. In some embodiments, the fourth intermediate number can be represented by eps, and by q max Characterize the second intermediate number. Thus, the second intermediate number can be obtained by q max =2 N+1 That is, the difference between the third intermediate number and the fourth intermediate number is used as the second intermediate number.

[0081] Therefore, the scaling factor and the bias factor can be determined subsequently in combination with the first intermediate number and the second intermediate number.

[0082] Further, for some implementations, step S230 may include steps S234 to S236.

[0083] Step S234: taking the difference between the maximum floating-point number and the minimum floating-point number as the fifth intermediate number.

[0084] Step S235: taking the difference between the second intermediate number and the first intermediate number as the sixth intermediate number.

[0085] Step S236: Using the ratio of the fifth intermediate number to the sixth intermediate number as the scaling factor.

[0086] Specifically, the difference between the maximum floating point number and the minimum floating point number can be first taken as the fifth intermediate number. That is, the fifth intermediate number can be represented as r max -r min .

[0087] Furthermore, the difference between the second intermediate number and the first intermediate number is taken as the sixth intermediate number. In other words, the sixth intermediate number can be represented as q max -q min .

[0088] Thus, the ratio of the fifth intermediate number to the sixth intermediate number is used as the scaling factor. Specifically, the scaling factor can be determined by the following formula (3).

[0089]

[0090] Furthermore, step S232 may include step S2321 and step S2322.

[0091] Step S2321: taking the ratio of the maximum floating-point number to the scaling factor as the seventh intermediate number.

[0092] Step S2322: taking the difference between the second intermediate number and the seventh intermediate number as the bias coefficient.

[0093] Specifically, the ratio of the maximum floating point number to the scaling factor can be used as the seventh intermediate number. That is, the seventh intermediate number can be represented as

[0094] Furthermore, the difference between the second intermediate number and the seventh intermediate number can be used as the bias coefficient. The bias coefficient can be represented by Z, so that the bias coefficient can be determined by formula (4).

[0095]

[0096] In some embodiments, the scaling factor determined by the above formula (3) may be 0. This may cause the bias coefficient determined in formula (4) to be abnormal. Therefore, optionally, after determining the scaling factor by formula (3), it is possible to further determine whether the scaling factor is 0. If the scaling factor is not 0, the bias coefficient can be determined by formula (4). If the scaling factor is 0, the scaling factor can be set to 1, and the bias coefficient can be determined by the following formula (5).

[0097] Z=r max +const0 (5)

[0098] Here, const0 can be any floating-point number between the first intermediate number and the second intermediate number.

[0099] It is understood that the bias coefficient and scaling coefficient obtained in the embodiment of the present application are both floating-point types. Thus, by calculating the tensor to be quantized based on the bias coefficient and scaling coefficient, the third floating-point tensor obtained is still a floating-point type tensor.

[0100] Furthermore, by performing the above data processing on the quantized tensor, the numerical range of the obtained third floating-point tensor can be made to be within the specified numerical interval. The specified numerical interval is greater than or equal to the first intermediate number and less than or equal to the second intermediate number, that is, the specified numerical interval is [2 N , qmax=2 N+1 -eps].

[0101] Step S240: Convert the third floating-point tensor into a binary number representation to obtain a first floating-point tensor, wherein the sign part and the exponent part corresponding to each first floating-point number in the first floating-point tensor are the same.

[0102] Thus, after data processing of the tensor to be quantized, a third floating-point tensor is obtained, and the third floating-point tensor can be further converted into a binary number representation to obtain the first floating-point tensor. As can be seen from the above introduction, the first floating-point tensor can include a sign part, an exponent part, and a decimal part. Specifically, each floating-point number in the third floating-point tensor can correspond to a sign part, an exponent part, and a decimal part. It should be noted that the second floating-point number in the third floating-point tensor is obtained after data processing of the corresponding floating-point number to be quantized in the tensor to be quantized.

[0103] Thus, the third floating-point tensor is converted into a binary representation to obtain a first floating-point tensor. For details, refer to formula (1) for conversion. The method for converting the third floating-point tensor into a binary representation is similar to the method for converting the tensor to be quantized into a binary representation, and will not be repeated here. Thus, the third floating-point tensor is converted into a binary representation to obtain a first floating-point tensor.

[0104] The first floating-point tensor may include multiple first floating-point numbers. The first floating-point number is a number obtained by converting the second floating-point number in the corresponding third floating-point tensor into a binary representation. Each first floating-point number has a corresponding signed portion, an exponent portion, and a fractional portion.

[0105] For example, if the third floating-point tensor includes the second floating-point number X and the second floating-point number Y, the first floating-point tensor may include the first floating-point number X1 and the first floating-point number Y1, wherein the first floating-point number X1 is the number obtained by converting the second floating-point number X into a binary representation; and the first floating-point number Y1 is the number obtained by converting the second floating-point number Y into a binary representation.

[0106] Since the value range of the third floating point tensor is within the specified value interval, the second floating point number in the third floating point tensor is also within the value range of the specified value interval. N , qmax=2 N+1 -eps]. Since the first floating-point number and the corresponding second floating-point number differ only in their data representation, each first floating-point number in the first floating-point tensor also falls within the specified numerical range. Consequently, the sign and integer parts of each first floating-point number in the first floating-point tensor are identical.

[0107] Step S250: Determine an integer tensor corresponding to each of the first floating-point numbers based on the decimal part corresponding to each of the first floating-point numbers.

[0108] As can be seen from the above description, each first floating-point number in the first floating-point tensor is also within a specified numerical range, so the decimal part corresponding to each first floating-point number may be different. Therefore, the integer tensor corresponding to each first floating-point number can be determined based on the decimal part corresponding to each first floating-point number. Subsequently, the quantized tensor after quantizing the tensor to be quantized is determined by combining the identical sign part and exponent part.

[0109] See also Figure 4 , Figure 4 A schematic diagram of a first floating-point number represented in binary format provided in an embodiment of the present application is shown. Figure 4The first floating point number 400 shown in FIG. 4 includes a plurality of binary bits, wherein each box represents a bit, and the value corresponding to each bit can be 1 or 0. Figure 4 The first floating point number 400 shown in FIG. 4 includes 16 bits, specifically bits 401 to 4016. Bit 401 is used to represent the sign portion corresponding to the first floating point number 400, bits 402 to 406 represent the exponent portion of the first floating point number 400, and bits 407 to 416 represent the decimal portion of the first floating point number 400. As can be seen, Figure 4 The quantization bit width of the first floating-point number shown in is 4.

[0110] It should be noted that the quantization bit width of the first floating-point number may also be a value required in other practical applications, such as 8 or 2. The above example is only an illustration and does not constitute a specific limitation of the embodiments of the present application.

[0111] Herein, step S250 may include step S251.

[0112] Step S251: extract the value of the highest first digit from the decimal part corresponding to each of the first floating-point numbers as the integer tensor corresponding to each of the first floating-point numbers.

[0113] In some embodiments, the first digit may be a quantization bit width for quantizing the first floating-point number, for example, 4. Thus, the value of the highest first digit may be truncated from the decimal part corresponding to each first floating-point number as the integer tensor corresponding to each first floating-point number.

[0114] For example, please see Figure 4 ,according to Figure 4 For the first floating-point number 400 shown in FIG, the first digit can be determined to be 4, and thus the highest first digit is truncated from the decimal portion corresponding to each first floating-point number as bits 407 to 410. Therefore, the values ​​corresponding to bits 407 and 410 can be used as the integer tensor corresponding to the first floating-point number 400. For example, if bits 407 to 410 are 1, 0, 1, and 0, respectively, the integer tensor corresponding to the first floating-point number 400 is 1010.

[0115] From the above introduction, it can be seen that the first floating-point tensor may include multiple first floating-point numbers, so that the value of the highest first digit can be intercepted from the decimal part corresponding to each of the first floating-point numbers as the integer tensor corresponding to each of the first floating-point numbers.

[0116] Optionally, in some embodiments, to further improve the quantization accuracy of the first floating-point number, the value of the second digit may be truncated from the decimal portion corresponding to the first floating-point number, and then combined with the value of the highest first digit to determine the integer tensor corresponding to the first floating-point number. Specifically, step S251 may further include steps S2511 to S2515.

[0117] Step S2511: extract the highest first digit from the decimal part corresponding to each of the first floating-point numbers as the ninth intermediate number.

[0118] The introduction to extracting the highest first digit from the decimal portion corresponding to each first floating-point number can be found in the previous steps and will not be repeated here. Thus, the highest first digit from the decimal portion corresponding to each first floating-point number can be used as the ninth intermediate number. In other words, the ninth intermediate number includes the first integer values ​​corresponding to the plurality of first floating-point numbers.

[0119] Step S2512: extracting the value of the second digit from the decimal part corresponding to each of the first floating-point numbers as the tenth intermediate number, where the second digit is the first digit plus 1.

[0120] The second digit is the first digit plus 1, for example, Figure 4 For example, when the first digit is 4 and the second digit is 5, it can be determined that the value corresponding to the bit 411 in the first floating point number 400 is the value of the second digit in the decimal part.

[0121] Then, the value of the second digit can be truncated from the decimal part corresponding to each of the first floating-point numbers to serve as a tenth intermediate number, wherein the tenth intermediate number includes the second integer values ​​corresponding to multiple first floating-point numbers.

[0122] Step S2513: Perform an OR operation on the ninth intermediate number and the tenth intermediate number to obtain an eleventh intermediate number.

[0123] Furthermore, in order to improve the accuracy of the integer tensor corresponding to each first floating-point number in the determined first floating-point tensor, the ninth intermediate number and the tenth intermediate number may be combined to determine the integer tensor corresponding to each first floating-point number.

[0124] In some embodiments, the ninth intermediate number can be ORed with the tenth intermediate number to obtain an eleventh intermediate number. It should be noted that the eleventh intermediate number includes the third integer value corresponding to each first floating-point number. Each third integer value is obtained by ORing the corresponding first integer value in the ninth intermediate number with the corresponding second integer value in the tenth intermediate number. In other words, ORing the ninth intermediate number with the tenth intermediate number is essentially ORing each first integer value in the ninth intermediate number with the corresponding second integer value in the tenth intermediate number.

[0125] For example, if the ninth intermediate number includes the first integer value K1 corresponding to the first floating-point number X1 and the first integer value L1 corresponding to the first floating-point number Y1, and the tenth intermediate number includes the second integer value K2 corresponding to the first floating-point number X1 and the second integer value L2 corresponding to the first floating-point number Y1, then the first integer value K1 and the second integer value K2 can be ORed together to obtain the third integer value M1 corresponding to the first floating-point number X1 included in the eleventh intermediate number; and the first integer value L1 and the second integer value L2 can be ORed together to obtain the third integer value M2 corresponding to the first floating-point number Y1 included in the eleventh intermediate number.

[0126] The OR operation performed on the first integer value and the second integer value may be a bitwise OR operation of the first integer value and the second integer value. It can be understood that the OR operation can be regarded as a "rounding" operation of the least significant bit of the first integer value.

[0127] Step S2514: Use 2 as the base and the first digit as the exponent to obtain the twelfth intermediate number.

[0128] Step S2515: The smaller number between the eleventh intermediate number and the difference between the twelfth intermediate number and 1 is used as the integer tensor corresponding to the first floating-point number.

[0129] Furthermore, the numerical range of the eleventh intermediate number can be limited. Specifically, 2 can be used as the base and the first digit as the exponent to obtain the twelfth intermediate number. The first digit is the quantization bit width. If M represents the quantization bit width, the twelfth intermediate number can be represented as 2 M .

[0130] Thus, the smaller number between the eleventh intermediate number and the difference between the twelfth intermediate number and 1 can be used as the integer tensor corresponding to the first floating-point number. For example, if D represents the eleventh intermediate number, D q As the integer tensor corresponding to the first floating-point number, it can be obtained through D q =min(D, 2 M -1) to determine the integer tensor corresponding to the first floating-point number.

[0131] The corresponding integer tensor is determined for each first floating-point number using the above method, thereby obtaining the integer tensor corresponding to each first floating-point number in the first floating-point tensor.

[0132] Step S260: determining a quantized tensor after quantizing the tensor to be quantized through the sign part, the exponent part, and the integer tensor corresponding to each first floating-point number.

[0133] Step S270: storing the quantized tensor.

[0134] As can be seen from the foregoing introduction, in the embodiment provided by the present application, the sign part and the exponent part corresponding to each first floating-point number in the first floating-point tensor are the same. Therefore, in order to further save storage space and reduce the storage space requirement of the neural network model on the mobile device, only one identical sign part and one specified part may be stored, and then the integer tensor corresponding to each first floating-point number may be stored. Storing an identical sign part and one specified part is essentially reusing the identical sign part and one specified part, thereby providing higher quantization accuracy while keeping the size of the used storage space unchanged.

[0135] That is, the quantized tensor after the tensor to be quantized includes a numerical value corresponding to a sign part and a numerical value corresponding to a specified part, and also includes an integer tensor corresponding to each first floating-point number.

[0136] Optionally, for the case where the quantization bit width is small, such as the case where the quantization bit width is 4, since the minimum bit width of the storage unit in the electronic device on the mobile side may be 8, if the integer tensor corresponding to each first floating-point number is stored separately in a different storage unit, it will cause a waste of storage space. Therefore, in some embodiments, the integer tensors corresponding to multiple first floating-point numbers can be combined into a number with a larger bit width through bit operations to obtain a combined integer tensor, which is then stored, thereby improving the utilization rate of the storage space of the storage unit and further reducing the demand for storage space on the mobile side. For example, the integer tensors corresponding to four first floating-point numbers with a quantization bit width of 4 can be combined into a 16-bit number through operations to obtain a combined integer tensor.

[0137] It should be noted that the data processing method provided in the embodiments of the present application can realize the quantization of different quantization strategies, for example, it can realize the quantization of per-tensor quantization strategy, per-channel quantization strategy or per-group quantization strategy.

[0138] The data processing method provided in this application first processes a tensor to be quantized and converts the tensor to be quantized into a third floating-point tensor. The third floating-point tensor is then converted into a binary representation to obtain a first floating-point tensor, wherein the sign and exponent corresponding to each first floating-point number in the first floating-point tensor are identical. Based on the decimal portion corresponding to each first floating-point number, the integer tensor corresponding to each first floating-point number is determined; thereby, a quantized tensor after quantizing the tensor to be quantized is determined based on the sign and exponent portions, and the integer tensors corresponding to each first floating-point number, and the quantized tensor is then stored. Because the sign and exponent portions corresponding to each first floating-point number in the first floating-point tensor are identical, by storing only one identical sign and designated portion, and then storing the integer tensors corresponding to each first floating-point number, the storage space requirements of the neural network model can be reduced. Furthermore, the integer tensors corresponding to multiple first floating-point numbers can be combined into a number with a larger bit width through bit operations to obtain the combined integer tensor, which is then stored. This improves the storage space utilization of the storage unit and further reduces the storage space requirements on the mobile terminal side. In addition, since the data stored in the exponential part and the sign part in the embodiment of the present application are reused, higher quantization accuracy can be provided while the size of the used storage space remains unchanged.

[0139] See also Figure 5 , Figure 5 A flow chart of a data processing method provided by an embodiment of the present application is shown. The data processing method can be applied to Figure 1 In the electronic device 110 shown in FIG, the processor in the electronic device 110 can be used as the execution subject. The data processing method specifically includes steps S310 to S370.

[0140] Step S310: Obtain the tensor to be quantized of the neural network model deployed on the mobile terminal side.

[0141] Step S320: converting the tensor to be quantized into a binary number representation to obtain a first floating-point tensor, wherein the first floating-point tensor includes a sign part, an exponent part, and a decimal part.

[0142] Step S330: determining a quantized tensor obtained by quantizing the tensor to be quantized through the sign part, the exponent part, and the decimal part.

[0143] Step S340: storing the quantized tensor.

[0144] Among them, steps S310 to S340 have been described in detail in the above embodiments and will not be repeated here.

[0145] Step S350: When the target data request is obtained, the stored quantized tensor is obtained.

[0146] Step S360: Convert the quantized tensor into a second floating-point tensor.

[0147] When the target data request is obtained, the stored quantized tensor can be obtained, and then the target data request can be completed based on the quantized tensor.

[0148] The target data request is a request to obtain the target data. The target data may be data required for computations performed by a neural network model. For example, if the neural network model is used to process image data on a mobile electronic device, the target data may be data required for image processing. For example, a smartphone, as a mobile electronic device, may be deployed with a neural network model for image processing. Thus, a user can directly use the neural network model deployed on the smartphone to perform super-resolution processing on the image being processed. For another example, a user can directly use the neural network model deployed on the smartphone to adjust shooting parameters for the image being processed, such as exposure time, white balance, sensitivity, aperture size, etc. For another example, if the neural network model is used to process text data on a mobile electronic device, the target data may be data required for text processing. For example, a smartphone, as a mobile electronic device, may be deployed with a neural network model for text processing. Thus, a user can directly use the neural network model deployed on the smartphone to perform semantic understanding on the text being processed. For another example, a user can directly use the neural network model deployed on the smartphone to extract a summary of the text being processed.

[0149] The data required to perform operations on the neural network model may include the neural network weights and activation values, and thus the neural network weights and activation values ​​may be used as target data. Therefore, when a user initiates a request to process data to be processed by performing operations on the neural network model deployed in a smartphone, a target data request is generated, i.e., a request to obtain the target data is generated. The data to be processed may be an image or text to be processed.

[0150] It is understandable that the performance parameters of electronic devices on the mobile side are generally low, for example, the storage space and processing resources of electronic devices on the mobile side are low. Therefore, if the tensors corresponding to the data in the neural network model are not compressed and the neural network model is directly deployed to the electronic device on the mobile side, the storage space occupied will be too large, which may cause abnormal problems such as the operation of the electronic device on the mobile side to be stuck. If the neural network model is directly deployed on a cloud device, such as a cloud server, although the cloud device has higher performance parameters, the data processing of the electronic device on the mobile side, such as image super-resolution processing or text semantic understanding, is completed by the neural network model deployed on the cloud device, which will cause high delays and inability to respond to users' data processing needs in a timely manner.

[0151] The data processing method provided in the embodiments of the present application quantizes and stores the tensors corresponding to the data in the neural network model deployed on the mobile terminal, thereby reducing the storage space requirements of the neural network model and, to a certain extent, reducing the pressure of performing operations using the neural network model on the mobile terminal. In addition, since the neural network model can be deployed on the mobile terminal, when performing operations using the neural network model, there is no need to transmit the obtained data to the mobile terminal after running the neural network model deployed on the cloud device. This greatly reduces the delay in performing operations using the neural network model and improves the speed of responding to user needs.

[0152] Furthermore, if the target data request is obtained, the stored quantized tensor can be obtained. It is understandable that although the quantized tensor occupies less storage space, if the quantized tensor is directly used for calculation, the accuracy is poor. Therefore, the obtained quantized tensor can be dequantized to obtain the target floating-point tensor, and then the target floating-point tensor can be used for calculation in the neural network model.

[0153] As can be seen from the introduction of the aforementioned embodiment, the quantized tensor includes a value corresponding to a sign portion and a value corresponding to a specified portion, as well as an integer tensor corresponding to each first floating-point number. Thus, the quantized tensor can first be converted into a second floating-point tensor, where the second floating-point tensor includes a binary representation corresponding to each first floating-point number. In other words, each first floating-point number in the second floating-point tensor corresponds to a sign portion, an exponent portion, and a fractional portion.

[0154] For some implementations, step S360 may include steps S361 to S364.

[0155] Step S361: Use 2 as the base and the first specified number as the exponent to obtain the thirteenth intermediate number.

[0156] Step S362: Perform an OR operation on the thirteenth intermediate number and the quantized tensor to obtain a fourteenth intermediate number.

[0157] Step S363: Perform a shift operation on the fourteenth intermediate number to obtain the second floating-point tensor.

[0158] First, 2 can be used as the base and the first specified number as the exponent to obtain the thirteenth intermediate number. Among them. If the first digit is represented by N, the thirteenth intermediate number can be represented as 2 N , where the first specified number is an integer. For example, the first specified number can be 1, 2, 3, -1, or -2, etc.

[0159] It can be understood that the thirteenth intermediate number can be a number represented in binary integer form. Thus, an OR operation can be performed on the thirteenth intermediate number and the quantized tensor to obtain the fourteenth intermediate number.

[0160] Exemplarily, if const1 represents the thirteenth intermediate number and q represents the quantized tensor, the fourteenth intermediate number can be represented as const1|q.

[0161] Further, perform a shift operation on the fourteenth intermediate number to obtain the second floating-point tensor. Among them, the specific value of the displacement can be represented by const2. It can be understood that the quantized tensor includes multiple numbers, so the fourteenth intermediate number still includes multiple numbers. Each number in the fourteenth intermediate number corresponds to a first floating-point number. Thus, the value of the first digit before the highest decimal part of the first floating-point number corresponding to each number in the fourteenth intermediate number can be used to shift each number in the fourteenth intermediate number. Exemplarily, const2 can be used to represent the specific value of the displacement operation. Thus, Q = const1|q << const2, where q1 is a number in the second floating-point tensor. That is, after performing a shift operation on each number in the fourteenth intermediate number, the obtained numbers can form the second floating-point tensor.

[0162] Step S370: Perform data reinterpretation on the second floating-point tensor to obtain the target floating-point tensor after dequantizing the second floating-point tensor.

[0163] It can be understood that each number in the second floating-point tensor is still a floating-point number stored in binary form.

[0164] Perform data reinterpretation on the second floating-point tensor again, so that the target floating-point tensor after dequantizing the second floating-point tensor can be obtained. Furthermore, the target data required in the target data request can be obtained through the target floating-point tensor.

[0165] Data reinterpretation is generally used to interpret a binary number to obtain a different value. Unlike data type conversion, data reinterpretation does not require data type conversion calculations, so the amount of computation required is relatively small.

[0166] For example, the second floating-point tensor can be reinterpreted using the as_float function. If R represents the target floating-point tensor and q1 represents the second floating-point tensor, the target floating-point tensor can be obtained by R=as_float(q1). The as_float function can be a predefined function for reinterpreting data.

[0167] The obtained target floating-point tensor can be used for the calculation of the neural network model on the mobile side.

[0168] Optionally, in some implementations, after obtaining the quantized tensor, related operations of the neural network model can also be directly implemented through the quantized tensor.

[0169] It should be noted that the data processing method provided in the embodiments of the present application can also be applied to a cloud server, where the cloud server is deployed with a neural network model.

[0170] Currently, even if model compression methods are used to reduce the storage space required for neural network models deployed on mobile devices, subsequent calculations using the neural network model require data type conversion to convert the stored data into the required data. This data type conversion requires high computing resources, which can cause lags in mobile electronic devices.

[0171] The data processing method provided by this application first deploys the neural network model to the mobile terminal, avoiding computing on cloud devices; then quantizes the neural network model to reduce storage space requirements; and finally obtains the target data required for the target data request through data reinterpretation, avoiding data type conversion and reducing the pressure on the processing resources of the electronic device on the mobile terminal. This allows users to directly complete data processing, such as image super-resolution processing or text semantic understanding, through the neural network model deployed on the mobile terminal.

[0172] Specifically, the method first obtains the tensor to be quantized of the neural network model deployed on the mobile side; then converts the tensor to be quantized into a binary number representation to obtain a first floating-point tensor; then determines the quantized tensor after quantizing the tensor to be quantized through the sign part, the exponent part and the decimal part; and stores the quantized tensor. When a target data request is obtained, the stored quantized tensor is obtained; the quantized tensor is converted into a second floating-point tensor; the second floating-point tensor is reinterpreted to obtain a target floating-point tensor after dequantization of the second floating-point tensor. The embodiment of the present application does not involve data type conversion calculations during the dequantization process, and the target floating-point tensor can be obtained only through data reinterpretation, which reduces the amount of calculation required for operations through the neural network model. At the same time, higher neural network model computing performance can also be achieved in mobile terminal devices with limited computing power.

[0173] See also Figure 6 , Figure 6 1 shows a structural block diagram of a data processing device 600 provided in an embodiment of the present application. The data processing device 600 includes: an acquisition unit 610, a first conversion unit 620, a quantization unit 630 and a storage unit 640.

[0174] The acquisition unit 610 is used to obtain the tensor to be quantized of the neural network model deployed on the mobile terminal side.

[0175] The first conversion unit 620 is configured to convert the tensor to be quantized into a binary number representation to obtain a first floating-point tensor, wherein the first floating-point tensor includes a sign part, an exponent part, and a decimal part.

[0176] Optionally, the first conversion unit 620 can also be used to respectively determine the maximum floating-point number and the minimum floating-point number among multiple floating-point numbers to be quantized; convert the tensor to be quantized into a third floating-point tensor based on the maximum floating-point number and the minimum floating-point number; convert the third floating-point tensor into a binary number representation to obtain a first floating-point tensor, and the sign part and exponent part corresponding to each first floating-point number in the first floating-point tensor are the same.

[0177] Optionally, the first conversion unit 620 can also be used to obtain a scaling factor based on the maximum floating-point number, the minimum floating-point number, a pre-constructed first intermediate number, and a pre-constructed second intermediate number; obtain a bias coefficient based on the second intermediate number, the scaling factor, and the maximum floating-point number; and obtain a third floating-point tensor based on the tensor to be quantized, the scaling factor, and the bias coefficient.

[0178] Optionally, the first conversion unit 620 can also be used to use 2 as the base and the first specified number as the exponent to obtain the first intermediate number, where the first specified number is an integer; use 2 as the base and the second specified number as the exponent to obtain a third intermediate number, where the second specified number is the first specified number plus 1; and use the difference between the third intermediate number and the fourth intermediate number as the second intermediate number, where the fourth intermediate number is greater than 0 and less than the specified value.

[0179] Optionally, the first conversion unit 620 can also be used to use the difference between the maximum floating-point number and the minimum floating-point number as the fifth intermediate number; use the difference between the second intermediate number and the first intermediate number as the sixth intermediate number; and use the ratio of the fifth intermediate number to the sixth intermediate number as the scaling factor.

[0180] Optionally, the first conversion unit 620 may be further configured to use a ratio of the maximum floating-point number to the scaling factor as a seventh intermediate number; and use a difference between the second intermediate number and the seventh intermediate number as the bias coefficient.

[0181] Optionally, the first conversion unit 620 may also be configured to take a quotient of the tensor to be quantized and the scaling coefficient as an eighth intermediate number; and take a sum of the eighth intermediate number and the bias coefficient as the third floating-point tensor.

[0182] The quantization unit 630 is configured to determine a quantized tensor after quantizing the tensor to be quantized through the sign part, the exponent part, and the fractional part.

[0183] Optionally, the quantization unit 630 can also be used to determine the integer tensor corresponding to each first floating-point number based on the decimal part corresponding to each first floating-point number; and determine the quantized tensor after quantizing the tensor to be quantized through the sign part, the exponent part and the integer tensor corresponding to each first floating-point number.

[0184] Optionally, the quantization unit 630 may also be configured to extract the value of the highest first digit from the decimal part corresponding to each of the first floating-point numbers as the integer tensor corresponding to each of the first floating-point numbers.

[0185] Optionally, the quantization unit 630 may be further configured to extract the value of the highest first digit from the decimal part corresponding to each of the first floating-point numbers as the ninth intermediate number;

[0186] Extract the value of the second digit from the decimal part corresponding to each of the first floating-point numbers as the tenth intermediate number, where the second digit is the first digit plus 1; perform an OR operation on the ninth intermediate number and the tenth intermediate number to obtain an eleventh intermediate number; use 2 as the base and the first digit as the exponent to obtain a twelfth intermediate number; and use the smaller of the eleventh intermediate number and the difference between the twelfth intermediate number and 1 as the integer tensor corresponding to the first floating-point number.

[0187] The storage unit 640 is configured to store the quantized tensor.

[0188] Optionally, the data processing device 600 may further include an inverse quantization unit ( Figure 6 (not shown). The dequantization unit is configured to, upon receiving a target data request, obtain the stored quantized tensor; convert the quantized tensor into a second floating-point tensor; and reinterpret data of the second floating-point tensor to obtain a target floating-point tensor that is dequantized from the second floating-point tensor.

[0189] Optionally, the dequantization unit can also be used to use 2 as the base and the first specified number as the exponent to obtain a thirteenth intermediate number; perform an OR operation on the thirteenth intermediate number and the quantized tensor to obtain a fourteenth intermediate number; and perform a shift operation on the fourteenth intermediate number to obtain the second floating-point tensor.

[0190] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0191] In the several embodiments provided in this application, the coupling between the units can be electrical, mechanical, or other forms of coupling. In addition, the functional units in the various embodiments of this application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or software functional units.

[0192] See also Figure 7 , Figure 7 The following is a block diagram of an electronic device 110 provided in an embodiment of the present application. The electronic device 110 may be a smartphone, a laptop computer, a desktop computer, a tablet computer, etc. The electronic device 110 in the present application may include one or more of the following components: a processor 111, a memory 112, and one or more application programs, wherein the processor 111 is electrically connected to the memory 112, and the one or more application programs are configured to execute the methods described in the aforementioned embodiments of the data processing method.

[0193] The processor 111 may include one or more processing cores. The processor 111 utilizes various interfaces and circuits to connect various components within the electronic device 110. It executes instructions, programs, code sets, or instruction sets stored in the memory 112, and accesses data stored in the memory 112 to perform various functions and process data within the electronic device 110. Optionally, the processor 111 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 111 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and computer programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 111 and may be implemented separately via a communication chip. Specifically, the method described in the above embodiments may be executed by one or more processors 111 .

[0194] For some embodiments, the memory 112 may include a random access memory (RAM) or a read-only memory (ROM). The memory 112 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 112 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described below, and the like. The data storage area may also store data created by the electronic device 110 during use.

[0195] See also Figure 8 , which shows a block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 800 stores program code, which can be called by a processor to execute the method described in the above method embodiment.

[0196] The computer-readable storage medium 800 can be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 800 has storage space for program code 810 that executes any of the method steps in the above method. These program codes can be read from or written into one or more computer program products. The program code 810 can, for example, be compressed in an appropriate form.

[0197] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, characterized in that: include: Obtain the tensors to be quantized of the neural network model deployed on the mobile side; Converting the tensor to be quantized into a binary representation to obtain a first floating-point tensor, wherein the first floating-point tensor includes a sign part, an exponent part, and a decimal part; Determine a quantized tensor after quantizing the tensor to be quantized through the sign part, the exponent part, and the decimal part; The quantized tensor is stored.

2. The method according to claim 1, characterized in that The tensor to be quantized includes a plurality of floating-point numbers to be quantized, and converting the tensor to be quantized into a binary number representation to obtain a first floating-point tensor includes: respectively determining a maximum floating point number and a minimum floating point number among a plurality of floating point numbers to be quantized; Converting the tensor to be quantized into a third floating-point tensor based on the maximum floating-point number and the minimum floating-point number; The third floating-point tensor is converted into a binary number representation to obtain a first floating-point tensor, wherein the sign part and the exponent part corresponding to each first floating-point number in the first floating-point tensor are the same.

3. The method according to claim 2, characterized in that The converting the to-be-quantized tensor into a third floating-point tensor based on the maximum floating-point number and the minimum floating-point number includes: Obtaining a scaling factor based on the maximum floating-point number, the minimum floating-point number, the pre-constructed first intermediate number, and the pre-constructed second intermediate number; Obtaining a bias coefficient based on the second intermediate number, a scaling factor, and a maximum floating-point number; A third floating-point tensor is obtained based on the tensor to be quantized, the scaling coefficient, and the bias coefficient.

4. The method according to claim 3, characterized in that Before obtaining the scaling factor based on the maximum floating-point number, the minimum floating-point number, the pre-constructed first intermediate number, and the pre-constructed second intermediate number, the method further includes: Using 2 as the base and a first designated number as the exponent to obtain the first intermediate number, wherein the first designated number is an integer; Using 2 as the base and the second specified number as the exponent, a third intermediate number is obtained, where the second specified number is the first specified number plus 1; The difference between the third intermediate number and the fourth intermediate number is used as the second intermediate number, and the fourth intermediate number is greater than 0 and less than a specified value.

5. The method according to claim 3, characterized in that The obtaining of a scaling factor based on the maximum floating point number, the minimum floating point number, the pre-constructed first intermediate number, and the pre-constructed second intermediate number comprises: The difference between the maximum floating point number and the minimum floating point number is used as the fifth intermediate number; The difference between the second intermediate number and the first intermediate number is used as the sixth intermediate number; The ratio of the fifth intermediate number to the sixth intermediate number is used as the scaling factor.

6. The method according to claim 3, characterized in that Obtaining a bias coefficient based on the second intermediate number, the scaling factor, and the maximum floating-point number includes: Taking the ratio of the maximum floating-point number to the scaling factor as a seventh intermediate number; The difference between the second intermediate number and the seventh intermediate number is used as the bias coefficient.

7. The method according to claim 3, characterized in that The obtaining of a third floating-point tensor based on the tensor to be quantized, the scaling coefficient, and the bias coefficient includes: taking the quotient of the tensor to be quantized and the scaling coefficient as an eighth intermediate number; The sum of the eighth intermediate number and the bias coefficient is used as the third floating-point tensor.

8. The method according to claim 2, characterized in that The determining, by using the sign part, the exponent part, and the decimal part, a quantized tensor after quantizing the tensor to be quantized, includes: Determine, based on a decimal part corresponding to each of the first floating-point numbers, an integer tensor corresponding to each of the first floating-point numbers; A quantized tensor after quantizing the tensor to be quantized is determined through the sign part, the exponent part, and the integer tensor corresponding to each first floating-point number.

9. The method according to claim 8, characterized in that The determining, based on the decimal part corresponding to each of the first floating-point numbers, an integer tensor corresponding to each of the first floating-point numbers includes: The value of the highest first digit is truncated from the decimal part corresponding to each of the first floating-point numbers as the integer tensor corresponding to each of the first floating-point numbers.

10. The method according to claim 9, characterized in that The extracting the value of the highest third specified digit from the decimal part corresponding to each of the first floating-point numbers as the integer tensor corresponding to each of the first floating-point numbers includes: truncating the highest first digit from the decimal part corresponding to each of the first floating-point numbers as the ninth intermediate number; truncating the value of the second digit from the decimal part corresponding to each of the first floating-point numbers as the tenth intermediate number, where the second digit is the first digit plus 1; ORing the ninth intermediate number with the tenth intermediate number to obtain an eleventh intermediate number; Using 2 as the base and the first digit as the exponent, we get the twelfth middle number; The smaller number between the eleventh intermediate number and the difference between the twelfth intermediate number and 1 is used as the integer tensor corresponding to the first floating-point number.

11. The method according to claim 1, wherein After storing the quantized tensor, the method further includes: When a target data request is obtained, obtaining the stored quantized tensor; Convert the quantized tensor into a second floating-point tensor; Data is reinterpreted on the second floating-point tensor to obtain a target floating-point tensor obtained by dequantizing the second floating-point tensor.

12. The method according to claim 11, characterized in that The converting the quantized tensor into a second floating-point tensor includes: Using 2 as the base and the first specified number as the exponent, we get the thirteenth middle number; Performing an OR operation on the thirteenth intermediate number and the quantized tensor to obtain a fourteenth intermediate number; A shift operation is performed on the fourteenth intermediate number to obtain the second floating-point tensor.

13. A data processing device, characterized in that: include: An acquisition unit, used to acquire the tensor to be quantized of the neural network model deployed on the mobile terminal side; A first conversion unit, configured to convert the tensor to be quantized into a binary number representation to obtain a first floating-point tensor, wherein the first floating-point tensor includes a sign part, an exponent part, and a decimal part; a quantization unit, configured to determine a quantized tensor after quantizing the tensor to be quantized, using the sign part, the exponent part, and the decimal part; A storage unit, configured to store the quantized tensor.

14. An electronic device, characterized in that: include: one or more processors; Memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claim 12.

Images