Data processing method and device, storage medium and electronic equipment
By converting the data in the first floating-point format into a second floating-point format with a smaller bit width, and storing and reading the second data in the memory, the problem of excessive data access pressure is solved and a more efficient Data storage and access.
Patent Information
- Application Number
- CN202411475440.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-21
- Publication Date
- 2025-01-24
AI Technical Summary
In the prior art, the pressure of data access is too high, resulting in an increase in memory storage pressure and an extended data reading time, which cannot effectively meet actual needs.
By converting the data in the first floating -point format to be stored to the second floating -point format in the first floating -point format as small as the first floating -point format, and store the second data in the memory, the second data is converted to the first data when it needs One floating -point format for operation.
The storage pressure and data reading time of the memory are reduced, and the data access pressure is effectively reduced, and the data access efficiency is improved.
Smart Images
Figure CN119358614A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to data processing technology, and in particular to a data processing method, device, storage medium and electronic device. Background Art
[0002] Data is closely related to information. Specifically, data can be the representation and carrier of information, and information can be the connotation of data.
[0003] In many scenarios, data access is required, that is, writing data into a memory and reading data from a memory. In the related art, it is easy to encounter a situation where the pressure of data access is too large. How to reduce the pressure of data access is a problem worthy of attention for those skilled in the art. Summary of the invention
[0004] In order to solve the above technical problems, the present disclosure provides a data processing method, device, storage medium and electronic device.
[0005] According to one aspect of an embodiment of the present disclosure, there is provided a data processing method, including:
[0006] determining first data in a first floating point format to be stored;
[0007] Converting the first data into second data in a second floating point format; wherein the bit width of the second floating point format is smaller than the bit width of the first floating point format;
[0008] storing the second data in a memory;
[0009] In response to a data read instruction, the second data is read and converted into third data in the first floating point format.
[0010] According to another aspect of an embodiment of the present disclosure, there is provided a data processing device, including:
[0011] A determination module, used for determining first data in a first floating point format to be stored;
[0012] a conversion module, configured to convert the first data determined by the determination module into second data in a second floating point format; wherein a bit width of the second floating point format is smaller than a bit width of the first floating point format;
[0013] A storage module is used to store the second data obtained by the conversion module in a memory; a processing module is used to read the second data stored in the storage module in response to a data read instruction, and convert the second data into third data in the first floating point format.
[0014] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the above data processing method.
[0015] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, the electronic device comprising:
[0016] processor;
[0017] a memory for storing instructions executable by the processor;
[0018] The processor is used to read the executable instructions from the memory and execute the instructions to implement the above data processing method.
[0019] According to another aspect of an embodiment of the present disclosure, a computer program product is provided. When instructions in the computer program product are executed by a processor, a data processing method is performed.
[0020] Based on the data processing method, device, storage medium, electronic device and program product provided by the above-mentioned embodiments of the present disclosure, after determining the first data in the first floating-point format to be stored, the first data is converted into the second data in the second floating-point format, and the second data is stored in the memory. When it is necessary to perform calculations based on the first data, the second data is read from the memory, and the second data is converted into the third data in the first floating-point format, so as to perform calculations through the third data. Since the bit width of the second floating-point format is smaller than the bit width of the first floating-point format, the storage space required to store the second data is smaller than the storage space required to store the first data, which is conducive to reducing the storage pressure of the memory, and the time required to read the second data is shorter than the time required to read the first data, which is conducive to using the required data as soon as possible. Therefore, the embodiments of the present disclosure can effectively reduce the pressure of data access, thereby better meeting actual needs. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 is a schematic diagram of data in floating point format in some exemplary embodiments of the present disclosure.
[0022] Figure 2 It is a flowchart of a data processing method provided by some exemplary embodiments of the present disclosure.
[0023] Figure 3 is a schematic diagram of data in floating point format in some other exemplary embodiments of the present disclosure.
[0024] Figure 4 The present invention is a flowchart of a method for converting first data into second data in a second floating point format provided by some exemplary embodiments of the present disclosure.
[0025] Figure 5 It is a flowchart of a method for determining second data in a second floating point format based on a high N-bit value in the first data and a target value provided by some exemplary embodiments of the present disclosure.
[0026] Figure 6 It is a flowchart of a method for converting first data into second data in a second floating point format provided by some other exemplary embodiments of the present disclosure.
[0027] Figure 7 It is a flowchart of a method for determining second data in a second floating point format based on a high N-bit value in first data provided by some exemplary embodiments of the present disclosure.
[0028] Figure 8 It is a flowchart of a method for converting first data into second data in a second floating point format provided by some further exemplary embodiments of the present disclosure.
[0029] Fig. 9 It is a flowchart of a method for converting second data into third data in a first floating point format provided by some exemplary embodiments of the present disclosure.
[0030] Fig.10 is a schematic diagram of first data, second data and third data in some exemplary embodiments of the present disclosure.
[0031] Fig.11 It is a schematic diagram of the structure of a data processing device provided by some exemplary embodiments of the present disclosure.
[0032] Fig.12 is a schematic diagram of the structure of the conversion module in some exemplary embodiments of the present disclosure.
[0033] Fig.13 It is a schematic diagram of the structure of the conversion module in some other exemplary embodiments of the present disclosure.
[0034] Fig.14 is a schematic diagram of the structure of the conversion module in some further exemplary embodiments of the present disclosure.
[0035] Fig.15 It is a schematic diagram of the structure of the processing module in some exemplary embodiments of the present disclosure.
[0036] Fig.16 It is a schematic diagram of the structure of an electronic device provided by some exemplary embodiments of the present disclosure. DETAILED DESCRIPTION
[0037] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.
[0038] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.
[0039] Application Overview
[0040] In many scenarios, data access is required, that is, writing data into the memory and reading data from the memory.
[0041] For example, in the training phase or inference phase of the neural network model, data access is required. Optionally, the neural network model may include a first network layer and a second network layer, and the second network layer may be the next network layer of the first network layer. Through the operation of the first network layer, an intermediate tensor may be generated, the intermediate tensor may be written into the memory, and then the intermediate tensor may be read from the memory, and the read intermediate tensor may be used as input data of the second network layer, and the second network layer may operate on the input data. Here, the intermediate tensor may be in the form of a feature map.
[0042] For another example, when the vehicle is in motion, data access is required. Optionally, sensors (such as cameras, radars, etc.) provided on the vehicle can collect environmental data, which can be written into a memory, and then the environmental data can be read out of the memory for calculation to assist the vehicle's automatic or semi-automatic driving.
[0043] It should be noted that no matter what scenario requires data access, if the amount of data to be accessed is large, data storage will take up a large storage space, which will bring heavy storage pressure to the memory, and it will take a long time to read the data, and the required data cannot be used as soon as possible. Therefore, the related technology has the problem of excessive data access pressure and cannot meet actual needs well.
[0044] Exemplary Systems
[0045] In computer systems, there are two common data representation formats, fixed-point format and floating-point format. In fixed-point format, the decimal point position in the data is fixed. In floating-point format, the decimal point position in the data is not fixed.
[0046] Optionally, the fixed-point format may include but is not limited to INT4, INT8, INT16, INT32, etc. The floating-point format may include but is not limited to FP8, FP16, FP32, etc. INT4 uses 4 bits to represent a data, and the bit width of INT4 may be 4 bits. INT8 and FP8 both use 8 bits to represent a data, and the bit width of INT8 and the bit width of FP8 may both be 8 bits. INT16 and FP16 both use 16 bits to represent a data, and the bit width of INT16 and the bit width of FP16 may both be 16 bits. INT32 and FP32 both use 32 bits to represent a data, and the bit width of INT32 and the bit width of FP32 may both be 32 bits.
[0047] Optionally, the data in floating point format may include three parts, namely, a sign bit, an exponent bit, and a mantissa bit. The sign bit may occupy 1 bit and is used to indicate the positive or negative of the data. For example, if the sign bit is 0, it indicates that the data is a positive number. If the sign bit is 1, it indicates that the data is a negative number. The exponent bit may correspond to the exponent part in scientific notation. The mantissa bit may correspond to the base part in scientific notation.
[0048] As an example, Figure 1 As shown, the floating point format may be FP16, in which the sign bit may occupy 1 bit, the exponent bit may occupy 5 bits, and the mantissa bit may occupy 10 bits. Alternatively, the floating point format may be FP8, specifically FP8 (E5M2), in which the sign bit may occupy 1 bit, the exponent bit may occupy 5 bits, and the mantissa bit may occupy 2 bits. Alternatively, the floating point format may be FP8, specifically FP8 (E4M3), in which the sign bit may occupy 1 bit, the exponent bit may occupy 4 bits, and the mantissa bit may occupy 3 bits.
[0049] Exemplary Methods
[0050] Figure 2 It is a flowchart of a data processing method provided by some exemplary embodiments of the present disclosure. Figure 2 The method shown may include step 210 , step 220 , step 230 , and step 240 .
[0051] Step 210: determine first data in a first floating point format to be stored.
[0052] Optionally, the first data in the first floating point format may be data related to a neural network model. As an example, the neural network model may be a large language model (LLM). Of course, the type of the neural network model is not limited thereto, and the present disclosure does not limit this. The number of first data in the first floating point format may be one or more, and the following mainly introduces the processing method of a single first data.
[0053] In some optional embodiments of the present disclosure, the first data may be data to be stored determined from intermediate activation data or key-value pair data generated during the operation of the neural network model.
[0054] Optionally, the neural network model may include an intermediate nonlinear activation operator. During the operation of the neural network model, the intermediate nonlinear activation operator may generate intermediate activation data. For example, in the face of a binary classification task, the intermediate activation data generated by the intermediate nonlinear activation operator may include two activation values, one of which is used to represent the probability corresponding to one classification, and the other is used to represent the probability corresponding to another classification. Then, any one of the two activation values may be used as a first data.
[0055] Optionally, the neural network model can be a model based on the attention mechanism. For example, the neural network model can be a Transformer model. The Transformer model can generate key-value pair data during operation. The "key" in the key-value pair data can represent a key tensor (Key), and the "value" in the key-value pair data can represent a value tensor (Value). The key tensor and the value tensor can respectively include multiple eigenvalues, and any eigenvalue of the multiple eigenvalues included in the key tensor can be used as a first data, and any eigenvalue of the multiple eigenvalues included in the value tensor can also be used as a first data.
[0056] In some other optional embodiments of the present disclosure, the first data may be data to be stored determined from model parameters of the neural network model.
[0057] Optionally, the model parameters of the neural network model may include but are not limited to weight parameters, bias parameters, etc. Any weight value in the weight parameters may be used as a first data. Any bias value in the bias parameters may also be used as a first data.
[0058] Referring to the above embodiments, the first data may be a single activation value, a single weight value, a single bias value, etc. Of course, the type of the first data is not limited thereto. For example, the entire intermediate activation data generated by the neural network model during operation may be used as the first data. For another example, the entire key tensor in the key-value pair data generated by the neural network model during operation may be used as the first data. For another example, the entire value tensor in the key-value pair data generated by the neural network model during operation may be used as the first data. For ease of understanding, the following mainly uses the case where the first data is a single activation value, a single weight value, a single bias value, etc. as an example for explanation.
[0059] Step 220: convert the first data into second data in a second floating point format; wherein the bit width of the second floating point format is smaller than the bit width of the first floating point format.
[0060] It should be noted that the first floating point format and the second floating point format may be two different floating point formats, and the first floating point format and the second floating point format may have at least the following difference: the bit width of the second floating point format is smaller than the bit width of the first floating point format. That is, the precision of the second data is lower than the precision of the first data.
[0061] Optionally, the first floating point format and the second floating point format may satisfy the following conditions:
[0062] The bit width of the sign bit of the second floating point format is equal to the bit width of the sign bit of the first floating point format;
[0063] The bit width of the exponent bits of the second floating point format is equal to the bit width of the exponent bits of the first floating point format;
[0064] The bit width of the mantissa bits of the second floating point format is smaller than the bit width of the mantissa bits of the first floating point format.
[0065] Assuming that the bit width of the sign bit of the first floating-point format is expressed as A1, the bit width of the exponent bit of the first floating-point format is expressed as B1, the bit width of the mantissa bit of the first floating-point format is expressed as C1, the bit width of the sign bit of the second floating-point format is expressed as A2, the bit width of the exponent bit of the second floating-point format is expressed as B2, and the bit width of the mantissa bit of the second floating-point format is expressed as C2, then: A2=A1, B2=B1, C2<C1.
[0066] In one example, the first floating point format may be Figure 1 , Figure 3 In FP16, the second floating point format can be Figure 1 , Figure 3 FP8(E5M2) in, then, A2=A1=1, B2=B1=5, 2=C2<C1=10.
[0067] After determining the first data in the first storage format by executing step 210, the data representation format of the first data may be converted to obtain second data in the second floating point format. The data representation format conversion of the first data may include: obtaining second data with lower precision by intercepting part of the bit width of the first data or by rounding off part of the bit width of the first data.
[0068] Step 230: Store the second data in the memory.
[0069] Optionally, the memory may be a module in the chip for storing data. As an example, the chip may include, but is not limited to, a smart driving chip, a smart cockpit chip, etc. The smart driving chip may be specifically used to support the smart driving function of the vehicle. The smart cockpit chip may be specifically used to support the smart cockpit function of the vehicle. The memory may include, but is not limited to, a synchronous dynamic random access memory (SDRAM), a register file, a flash memory, etc.
[0070] Optionally, an unoccupied space may be searched in the memory, and the second data may be written into the searched space. Since the bit width of the second floating point format is smaller than the bit width of the first floating point format, the precision of the second data is lower than that of the first data, and the storage space required to store the first data in the memory is smaller than that of directly storing the first data in the memory.
[0071] Step 240 , in response to the data read instruction, read the second data, and convert the second data into third data in the first floating point format.
[0072] Optionally, a large amount of data may be stored in the memory. Certain hardware units in the chip may initiate a data read instruction for part of the data stored in the memory, and the part of the data may include the second data. The hardware unit that initiates the data read instruction may be, for example, a tensor operation unit, a vector operation unit, etc. The tensor operation unit may be used to perform tensor operations, which may include, for example, convolution operations. The vector operation unit may be used to perform vector operations, which may include, for example, point-to-point element-wise operations.
[0073] In response to the data read instruction, the second data can be obtained by reading data from the memory. Generally speaking, the hardware unit that initiates the data read instruction has requirements for the data representation format of the data to be operated, for example, requiring the data representation format of the data to be operated to be a first floating point format. In view of this, after obtaining the second data, the data representation format of the second data can be converted to obtain third data in the first floating point format. Among them, converting the data representation format of the second data can include: obtaining third data with a larger bit width by expanding the bit width of the second data. Since the third data is in the first floating point format, it meets the requirements of the hardware unit that initiates the data read instruction for the data representation format of the data to be operated, and the hardware unit that initiates the data read instruction can successfully operate on the third data, such as successfully performing tensor operations or vector operations on the third data.
[0074] In an embodiment of the present disclosure, after determining the first data in the first floating-point format to be stored, the first data is converted into the second data in the second floating-point format, and the second data is stored in the memory. When it is necessary to perform an operation based on the first data, the second data is read from the memory, and the second data is converted into the third data in the first floating-point format, so as to perform the operation through the third data. Since the bit width of the second floating-point format is smaller than the bit width of the first floating-point format, the storage space required to store the second data is smaller than the storage space required to store the first data, which is conducive to reducing the storage pressure of the memory, and the time required to read the second data is shorter than the time required to read the first data, which is conducive to using the required data as soon as possible. Therefore, the embodiment of the present disclosure can effectively reduce the pressure of data access, thereby better meeting the actual needs.
[0075] As described above, the first data may be data to be stored determined from intermediate activation data or key-value pair data generated during the operation of the neural network model. That is, for the intermediate activation data and key-value pair data generated during the operation of the neural network model, these data can be converted from higher precision to lower precision before storage, which is beneficial to reduce the required storage space and reduce the storage pressure of the memory. Moreover, when calculations need to be performed based on these data, these data can be read in a shorter time, which is beneficial to ensure the operation speed of the neural network model and realize the calculation acceleration of the neural network model.
[0076] As described above, the first data may also be data to be stored determined from the model parameters of the neural network model. That is, for the model parameters of the neural network model, these model parameters can be converted from higher precision to lower precision before storage, which is beneficial to reduce the required storage space and reduce the storage pressure of the memory. Moreover, when calculations need to be performed based on these model parameters, these model parameters can be read in a shorter time, which is beneficial to ensure the running speed of the neural network model and realize the calculation acceleration of the neural network model.
[0077] Figure 4 The present invention is a flowchart of a method for converting first data into second data in a second floating point format provided by some exemplary embodiments of the present disclosure. Figure 4 The illustrated method may include step 410 and step 430 .
[0078] Figure 4 In the method shown, the bit width of the second floating point format may be N. N may be an integer greater than 0. As described above, the bit width of the sign bit of the second floating point format may be represented as A2, the bit width of the exponent bit of the second floating point format may be represented as B2, and the bit width of the mantissa bit of the second floating point format may be represented as C2, then: N=A2+B2+C2. In an example, the second floating point format may be FP8 (E5M2) or FP8 (E4M3), then N may be 8.
[0079] Optionally, the second data in the second floating-point format can be regarded as a first numerical sequence formed by sequentially arranging N numerical values from left to right. Each of the N numerical values can be 0 or 1. In the first numerical sequence, the sign bit numerical value is sorted first, the exponent bit numerical value is sorted after the sign bit numerical value, and the mantissa bit numerical value is sorted after the exponent bit numerical value, and the sign bit numerical value is composed of A2 numerical values, the exponent bit numerical value is composed of B2 numerical values, and the mantissa bit numerical value is composed of C2 numerical values.
[0080] Step 410, determining the high-N-bit value in the first data and the target value; wherein the target value is a value adjacent to the high-N-bit value in the first data.
[0081] As introduced above, the bit width of the sign bit of the first floating-point format can be expressed as A1, the bit width of the exponent bit of the first floating-point format can be expressed as B1, and the bit width of the mantissa bit of the first floating-point format can be expressed as C1. If the bit width of the first floating-point format is expressed as M, then: M=A1+B1+C1, M>N.
[0082] Optionally, the first data can be regarded as a second numerical sequence formed by sequentially arranging M numerical values from left to right. Each of the M numerical values can be 0 or 1. In the second numerical sequence, the sign bit numerical value is sorted first, the exponent bit numerical value is sorted after the sign bit numerical value, and the mantissa bit numerical value is sorted after the exponent bit numerical value, and the sign bit numerical value is composed of A1 numerical values, the exponent bit numerical value is composed of B1 numerical values, and the mantissa bit numerical value is composed of C1 numerical value.
[0083] It should be noted that the high N bits in the first data may refer to the first N bits in the first data. For example, in order from left to right, the first N values in the second numerical sequence may be used as the high N bit values in the first data (in the high N bit values, the later the order, the lower the number of bits), then, the high N bit values in the first data may include the sign bit value, the exponent bit value, and part of the mantissa bit value in the first data. In addition, in order from left to right, the N+1th value in the second numerical sequence may be used as the target value in the first data.
[0084] Step 430: Determine second data in a second floating point format based on the high N-bit value in the first data and the target value.
[0085] Optionally, the high N-bit values in the first data may be updated based on the target value, and the second data may be determined based on the updated high N-bit values. As an example, the high N-bit values in the first data may be updated based on the target value by rounding the mantissa.
[0086] In some optional embodiments of the present disclosure, Figure 5 As shown, step 430 may include step 4301 , step 4303 , step 4305 and step 4307 .
[0087] Step 4301, add the lowest bit value in the high N bit values in the first data to the target value, and update the high N bit values in the first data.
[0088] In an example, Figure 3As shown, the first floating point format is FP16, the second floating point format is FP8 (E5M2), the first data can be regarded as a second numerical sequence formed by 16 numerical values arranged in order from left to right, the lowest bit value in the high N-bit numerical value in the first data can be the 8th numerical value in the second numerical sequence, and the target numerical value in the first data can be the 9th numerical value in the second numerical sequence. Then, according to the binary addition rule, the 9th numerical value in the second numerical sequence can be added to the 8th numerical value in the second numerical sequence to update the first 8 numerical values in the second numerical sequence, and the updated first 8 numerical values are the updated high N-bit numerical values. Generally speaking, according to the binary addition rule, we can have: 0+0=0, 0+1=1, 1+0=1, 1+1=10 (carry to the high bit).
[0089] Step 4303, determining the sign bit value in the second data based on the sign bit value in the updated high N-bit value.
[0090] Step 4305, determining the exponent bit value in the second data based on the exponent bit value in the updated high N-bit value.
[0091] Step 4307, determining the mantissa value in the second data based on the mantissa value in the updated high N-bit value.
[0092] Optionally, the updated high N-bit value may include a sign bit value, an exponent bit value and a mantissa bit value. Then, the sign bit value in the updated high N-bit value may be determined as the sign bit value in the second data, the exponent bit value in the updated high N-bit value may be determined as the exponent bit value in the second data, and the mantissa bit value in the updated high N-bit value may be determined as the mantissa bit value in the second data.
[0093] In this way, by adding the target value to the lowest bit value of the high N-bit value in the first data, the high N-bit value in the first data can be updated, and based on the sign bit value, exponent bit value and mantissa bit value in the updated high N-bit value, the sign bit value, exponent bit value and mantissa bit value in the second data can be reasonably determined. In this way, the complete second data can be obtained efficiently and quickly.
[0094] It should be noted that, in the above-mentioned implementation manner, the definition rules of the sign bit may be the same in the first floating-point format and the second floating-point format. For example, both the first floating-point format and the second floating-point format satisfy: the sign bit is 0, indicating that the data is a positive number, and the sign bit is 1, indicating that the data is a negative number. In some embodiments, the definition rules of the sign bit may also be different in the first floating-point format and the second floating-point format. For example, the first floating-point format satisfies: the sign bit is 0, indicating that the data is a positive number; the second floating-point format satisfies: the sign bit is 1, indicating that the data is a positive number. In this case, if the sign bit value in the updated high N-bit value is 0, the sign bit value (i.e., 0) in the updated high N-bit value may not be determined as the sign bit value in the second data, but 1 may be determined as the sign bit value in the second data.
[0095] In the embodiment of the present disclosure, the high N-bit value and the target value can be intercepted from the first data, and the second data in the second floating point format can be determined efficiently and quickly based on the intercepted high N-bit value and the target value. Since the second data is determined based on several values in the high bits of the first data, the precision loss in the process of converting the first data into the second data is small.
[0096] Figure 6 It is a flowchart of a method for converting first data into second data in a second floating point format provided by some other exemplary embodiments of the present disclosure. Figure 6 The illustrated method may include step 610 and step 630 . Figure 6 In the method shown, the bit width of the second floating point format may be N.
[0097] Step 610, determine the high-order N-bit value of the first data.
[0098] Optionally, the specific implementation of step 610 may refer to the above description of step 410 and will not be repeated here.
[0099] Step 630: Determine second data in a second floating point format based on the high N-bit value of the first data.
[0100] Optionally, the second data may be determined directly based on the sign bit value, exponent bit value and mantissa bit value in the high N-bit value of the first data. Alternatively, the high N-bit value in the first data may be updated in a certain manner, and then the second data may be determined based on the sign bit value, exponent bit value and mantissa bit value in the updated high N-bit value. For details, please refer to Figure 5 The relevant introduction in the illustrated implementation manner will not be repeated here.
[0101] In some optional embodiments of the present disclosure, Figure 7As shown, step 630 may include step 6301 , step 6303 and step 6305 .
[0102] Step 6301, based on the sign bit value in the high N-bit value in the first data, determine the sign bit value in the second data.
[0103] Step 6303, determine the exponent bit value in the second data based on the exponent bit value in the high N-bit value in the first data.
[0104] Step 6305, determine the mantissa value in the second data based on the mantissa value in the high N-bit value in the first data.
[0105] Optionally, the sign bit value in the high N-bit value in the first data can be determined as the sign bit value in the second data, the exponent bit value in the high N-bit value in the first data can be determined as the exponent bit value in the second data, and the mantissa bit value in the high N-bit value in the first data can be determined as the mantissa bit value in the second data.
[0106] Figure 7 In the illustrated embodiment, the sign bit value, exponent bit value and mantissa bit value in the second data can be reasonably determined based on the sign bit value, exponent bit value and mantissa bit value in the high N-bit value in the first data, so that the complete second data can be obtained efficiently and quickly.
[0107] It should be noted that, in the above implementation, the definition rules of the sign bit may be the same in the first floating-point format and the second floating-point format. In some embodiments, the definition rules of the sign bit may also be different in the first floating-point format and the second floating-point format. For example, the first floating-point format satisfies: the sign bit is 0, indicating that the data is a positive number; the second floating-point format satisfies: the sign bit is 1, indicating that the data is a positive number. In this case, if the sign bit value in the high-N-bit value of the first data is 0, the sign bit value (i.e., 0) in the high-N-bit value of the first data may not be determined as the sign bit value in the second data, and 1 may be determined as the sign bit value in the second data.
[0108] In the embodiment of the present disclosure, the high N-bit value can be intercepted from the first data, and the second data in the second floating point format can be determined efficiently and quickly based on the intercepted high N-bit value. Since the second data is determined based on several values in the high bits of the first data, the precision loss in the process of converting the first data into the second data is small.
[0109] Figure 8 It is a flowchart of a method for converting first data into second data in a second floating point format provided by some further exemplary embodiments of the present disclosure. Figure 8The method shown may include step 810 , step 820 , and step 830 . Figure 8 In the method shown, the bit width of the second floating point format may be N.
[0110] Step 810, determining the high-N-bit value in the first data and the target value; wherein the target value is a value adjacent to the high-N-bit value in the first data.
[0111] Optionally, the specific implementation of step 810 may refer to the above description of step 410 and will not be repeated here.
[0112] Step 820 , in response to the target value being a preset value, determining second data in a second floating point format based on the high N-bit value in the first data.
[0113] Step 830 , in response to the target value not being the preset value, determining second data in a second floating point format based on the high N-bit value in the first data and the target value.
[0114] Optionally, the preset value may be 0.
[0115] It should be noted that according to the binary addition rule, adding a certain value to 0 still results in the original value, and the value will not change. Then, when the target value is 0, if the lowest bit value of the high N bits in the first data is added to the target value to update the high N bits in the first data, the updated high N bits are exactly the same as the unupdated high N bits, and the addition operation here can be considered an unnecessary operation.
[0116] In view of this, in the embodiment of the present disclosure, it can be determined whether the target value is 0. If the target value is 0, the second data in the second floating point format can be determined based on the high N-bit value in the first data, for example, Figure 7 In the embodiment shown, the second data in the second floating point format is determined to achieve the conversion of the data representation format as quickly as possible. If the target value is not 0, the second data in the second floating point format can be determined based on the high N-bit value in the first data and the target value. For example, Figure 5 In the illustrated implementation, the second data in the second floating point format is determined to implement the conversion of the data representation format by rounding the mantissa, thereby avoiding unnecessary binary addition operations and saving resources and power consumption.
[0117] Fig. 9 It is a flowchart of a method for converting second data into third data in a first floating point format provided by some exemplary embodiments of the present disclosure. Fig. 9 The method shown may include step 910 , step 920 , and step 930 .
[0118] Step 910, determining a sign bit value in the third data based on the sign bit value in the second data.
[0119] Step 920, determining the value of the exponent bit in the third data based on the value of the exponent bit in the second data.
[0120] Step 930, based on the mantissa value in the second data and the preset value, determine the mantissa value in the third data; wherein, in the third data, the preset value is sorted after the mantissa value in the second data.
[0121] Optionally, the preset value may be 0.
[0122] Optionally, the second data may include a sign bit value, an exponent bit value and a mantissa bit value. Then, the sign bit value in the second data may be determined as the sign bit value in the third data, the exponent bit value in the second data may be determined as the exponent bit value in the third data, and several preset values may be added after the mantissa bit value in the second data, so that the total number of values included in the mantissa bit value in the second data and the several preset values is C1.
[0123] In the embodiment of the present disclosure, based on the second data, the sign bit value, exponent bit value and part of the mantissa bit value in the third data can be reasonably determined, and then by filling the low bits with 0 based on the second data, the complete third data can be obtained efficiently and quickly.
[0124] In an example, Fig.10 As shown, the first floating point format is FP16, the second floating point format is FP8 (E5M2), and the first data can be in the following numerical sequence: 0011000001000001. Then, the high 8-bit value can be intercepted from the first data. Obviously, the intercepted high 8-bit value can be in the following numerical sequence: 00110000. In addition, the target value in the first data is 0. Adding 0 to the lowest bit value in the intercepted high 8-bit value can obtain the updated high 8-bit value. The updated high 8-bit value can be in the following numerical sequence: 00110000. Then, the second data can be in the following numerical sequence: 00110000. The second data can be stored in the memory. After the second data is subsequently read out from the memory, 8 zeros can be added to the end of the second data to obtain the third data, and the third data can be in the following numerical form: 001100000000000. It can be easily seen that the data in the embodiment of the present disclosure represents that the precision loss caused by the format conversion is small.
[0125] In this way, by using the embodiments of the present disclosure, the conversion between FP16 and FP8 can be realized very efficiently and quickly, the pressure of data memory access can be reduced (for example, the data memory access bandwidth can be doubled), and the running speed of the neural network model can be guaranteed without loss or small loss of model accuracy, and the calculation acceleration of the neural network model can be realized, thereby greatly improving the performance of the neural network model. In the embodiments of the present disclosure, the operating environment of the neural network model does not need to support FP8 calculations, but only needs to support FP16 calculations to obtain the storage and access efficiency under the FP16 data representation format, and processors that support FP16 calculations are very common. In addition, in the embodiments of the present disclosure, no additional parameters are introduced in the entire floating-point data format conversion process, and the implementation is simple and efficient.
[0126] Exemplary Devices
[0127] Fig.11 It is a schematic diagram of the structure of a data processing device provided by some exemplary embodiments of the present disclosure. Fig.11 The apparatus shown may include:
[0128] A determination module 1110, configured to determine first data in a first floating point format to be stored;
[0129] A conversion module 1120, configured to convert the first data determined by the determination module 1110 into second data in a second floating point format; wherein the bit width of the second floating point format is smaller than the bit width of the first floating point format;
[0130] The storage module 1130 is used to store the second data obtained by the conversion module 1120 after conversion in the memory;
[0131] The processing module 1140 is configured to read the second data stored in the storage module 1130 in response to the data read instruction, and convert the second data into third data in the first floating point format.
[0132] In some optional examples, the bit width of the second floating point format is N;
[0133] like Fig.12 As shown, the conversion module 1120 includes:
[0134] The first determination submodule 1210 is used to determine the high-order N-bit value in the first data and the target value; wherein the target value is a value adjacent to the high-order N-bit value in the first data;
[0135] The second determining submodule 1220 is used to determine second data in a second floating point format based on the high N-bit value and the target value in the first data determined by the first determining submodule 1210.
[0136] In some optional examples, the second determining submodule 1220 includes:
[0137] An updating module, configured to add the lowest bit value of the high-N bit values in the first data to the target value, and update the high-N bit values in the first data;
[0138] A first determining unit, configured to determine a sign bit value in the second data based on a sign bit value in the high-order N-bit value updated by the updating module;
[0139] A second determining unit, configured to determine a value of an exponent bit in the second data based on a value of an exponent bit in the high-N-bit value updated by the updating module;
[0140] The third determining unit is used to determine the mantissa value in the second data based on the mantissa value in the high N-bit value updated by the updating module.
[0141] In some optional examples, the bit width of the second floating point format is N;
[0142] like Fig.13 As shown, the conversion module 1120 includes:
[0143] The third determination submodule 1310 is used to determine the high N-bit value of the first data;
[0144] The fourth determining submodule 1320 is used to determine second data in a second floating point format based on the high N-bit value of the first data determined by the third determining submodule 1310.
[0145] In some optional examples, the fourth determining submodule 1320 includes:
[0146] A fourth determining unit, configured to determine a sign bit value in the second data based on a sign bit value in a high-order N-bit value in the first data determined by the third determining submodule 1310;
[0147] A fifth determining unit, configured to determine a value of an exponent bit in the second data based on a value of an exponent bit in a high-order N-bit value in the first data determined by the third determining submodule 1310;
[0148] The sixth determining unit is used to determine the mantissa value in the second data based on the mantissa value in the high-order N-bit value in the first data determined by the third determining submodule 1310.
[0149] In some optional examples, the bit width of the second floating point format is N;
[0150] like Fig.14 As shown, the conversion module 1120 includes:
[0151] A fifth determination submodule 1410 is used to determine a high-order N-bit value in the first data and a target value; wherein the target value is a value adjacent to the high-order N-bit value in the first data;
[0152] A sixth determining submodule 1420, configured to determine, in response to the target value determined by the fifth determining submodule 1410 being a preset value, second data in a second floating point format based on a high N-bit value in the first data determined by the fifth determining submodule 1410;
[0153] The seventh determination submodule 1430 is used to determine second data in a second floating point format based on the high N-bit value of the first data determined by the fifth determination submodule 1410 and the target value in response to the target value determined by the fifth determination submodule 1410 not being the preset value.
[0154] In some optional examples, such as Fig.15 As shown, the processing module 1140 includes:
[0155] An eighth determination submodule 1510, configured to determine a sign bit value in the third data based on a sign bit value in the second data;
[0156] A ninth determination submodule 1520, configured to determine a value of an exponent digit in the third data based on a value of an exponent digit in the second data;
[0157] The tenth determination submodule 1530 is used to determine the mantissa value in the third data based on the mantissa value in the second data and the preset value; wherein, in the third data, the preset value is sorted after the mantissa value in the second data.
[0158] In some optional examples, the first data is data to be stored determined from intermediate activation data or key-value pair data generated during the operation of the neural network model.
[0159] In some optional examples, the first data is data to be stored determined from model parameters of the neural network model.
[0160] In the device of the present disclosure, the various optional embodiments, optional implementation methods and optional examples disclosed above can be flexibly selected and combined as needed to achieve corresponding functions and effects, and the present disclosure does not list them one by one.
[0161] Exemplary Electronic Devices
[0162] Fig.16 The block diagram of an electronic device according to an embodiment of the present disclosure is illustrated. The electronic device 1600 includes one or more processors 1610 and a memory 1620 .
[0163] The processor 1610 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1600 to perform desired functions.
[0164] The memory 1620 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1610 may execute one or more computer program instructions to implement the methods of the various embodiments of the present disclosure described above and / or other desired functions.
[0165] In one example, the electronic device 1600 may further include: an input device 1630 and an output device 1640 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).
[0166] The input device 1630 may also include, for example, a keyboard, a mouse, etc.
[0167] The output device 1640 can output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.
[0168] Of course, to simplify, Fig.16 Only some of the components related to the present disclosure in the electronic device 1600 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 1600 may also include any other appropriate components.
[0169] Exemplary computer program products and computer-readable storage media
[0170] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.
[0171] The computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0172] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.
[0173] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0174] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations. It cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present disclosure. The specific details disclosed above are only for the purpose of illustration and ease of understanding, not limitation. The above details do not limit the present disclosure to being implemented by adopting the above specific details.
[0175] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present disclosure claims and their equivalents, the present disclosure is also intended to include these modifications and variations.
Claims
1. A data processing method, comprising: determining first data in a first floating point format to be stored; Converting the first data into second data in a second floating point format; wherein the bit width of the second floating point format is smaller than the bit width of the first floating point format; storing the second data in a memory; In response to a data read instruction, the second data is read and converted into third data in the first floating point format.
2. The method according to claim 1, wherein: The bit width of the second floating point format is N; The converting the first data into second data in a second floating point format comprises: Determine a high-order N-bit value and a target value in the first data; wherein the target value is a value adjacent to the high-order N-bit value in the first data; The second data in the second floating point format is determined based on the high N-bit value in the first data and the target value.
3. The method according to claim 2, wherein: The determining, based on the high N-bit value in the first data and the target value, the second data in the second floating point format comprises: Add the lowest bit value in the high-N bit values in the first data to the target value, and update the high-N bit values in the first data; Determine the sign bit value in the second data based on the sign bit value in the updated high-N-bit value; Determine the exponent bit value in the second data based on the exponent bit value in the updated high N-bit value; Based on the mantissa bit value in the updated high-N-bit value, the mantissa bit value in the second data is determined.
4. The method according to claim 1, wherein: The bit width of the second floating point format is N; The converting the first data into second data in a second floating point format comprises: Determine a high-order N-bit value in the first data; The second data in the second floating point format is determined based on the upper N-bit values in the first data.
5. The method according to claim 4, wherein: The determining, based on the high N-bit value in the first data, the second data in the second floating point format comprises: Determine the sign bit value in the second data based on the sign bit value in the high-order N-bit value in the first data; Determine the value of the exponent bit in the second data based on the value of the exponent bit in the high-N-bit value in the first data; The mantissa bit value in the second data is determined based on the mantissa bit value in the upper N-bit value in the first data.
6. The method according to claim 1, wherein: The bit width of the second floating point format is N; The converting the first data into second data in a second floating point format comprises: Determine a high-order N-bit value and a target value in the first data; wherein the target value is a value adjacent to the high-order N-bit value in the first data; In response to the target value being a preset value, determining the second data in the second floating point format based on a value of high N bits in the first data; In response to the target value not being a preset value, the second data in the second floating point format is determined based on a high-order N-bit value in the first data and the target value.
7. The method according to claim 1, wherein: The converting the second data into third data in the first floating point format comprises: determining a sign bit value in the third data based on a sign bit value in the second data; Determining a value of an exponent bit in the third data based on a value of an exponent bit in the second data; Based on the mantissa value in the second data and the preset value, the mantissa value in the third data is determined; wherein, in the third data, the preset value is sorted after the mantissa value in the second data.
8. The method according to any one of claims 1 to 7, wherein: The first data is data to be stored determined from intermediate activation data or key-value pair data generated during the operation of the neural network model; or, The first data is data to be stored that is determined from model parameters of the neural network model.
9. A data processing device, comprising: A determination module, used for determining first data in a first floating point format to be stored; a conversion module, configured to convert the first data determined by the determination module into second data in a second floating point format; wherein a bit width of the second floating point format is smaller than a bit width of the first floating point format; A storage module, used for storing the second data obtained by the conversion module after conversion in a memory; The processing module is used for reading the second data stored in the storage module in response to a data reading instruction, and converting the second data into third data in the first floating point format.
10. A computer-readable storage medium, wherein the storage medium stores a computer program, wherein the computer program is used to execute the data processing method according to any one of claims 1 to 8.
11. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the data processing method described in any one of claims 1-8.