Data processing method and device, computer equipment and storage medium

Abstract

The invention relates to a data processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring input original data and a corresponding data type; under the condition that the data type is a floating point format, obtaining an index and a mantissa of the original data; according to the index, determining an integer significant digit of a fixed-point format corresponding to the original data; according to the index and the mantissa, determining mantissa significant digits of a fixed-point format corresponding to the original data; determining a fixed-point format structure corresponding to the original data according to the integer significant bit and the mantissa significant bit of the fixed-point format; converting the original data into fixed-point format data, and storing the fixed-point format data into a fixed-point format structure to obtain converted data; and under the condition that the data type of the original data is a fixed-point format or the converted data is obtained, inputting the original data and / or the converted data into the normalized exponential function to obtain target data. By adopting the method, the data processing efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a data processing method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of big data technology, a variety of data processing methods have emerged. Among these methods, the most critical is to combine related data (data with a certain correlation, such as distance and time; after combined processing, speed can be obtained, etc.) (for example, normalizing exponential functions to process related data, etc.) to generate more valuable data or information. However, in actual use, many related data are stored in different data types (for example, fixed-point format, floating-point format, etc.), and combining related data requires that the related data must have a unified data type.

[0003] In the current technology, before processing the associated data, the associated data of different data types are usually preprocessed to convert them into a standardized data format for subsequent combined processing. However, this method requires the establishment of a data type conversion module and a data processing module at the same time, which are used to complete the data type conversion and the combined processing of the associated data respectively, and the hardware resource occupancy rate is high. Summary of the invention

[0004] Based on this, it is necessary to provide a data processing method, apparatus, computer device, computer-readable storage medium and computer program product to address the above technical issues.

[0005] In a first aspect, the present application provides a data processing method. The method comprises:

[0006] Get the input raw data and the corresponding data type;

[0007] When the data type is in floating point format, obtaining the exponent and mantissa of the original data;

[0008] Determine the number of significant digits of an integer in a fixed-point format corresponding to the original data according to the exponent; and determine the number of significant digits of a mantissa in a fixed-point format corresponding to the original data according to the exponent and the mantissa;

[0009] Determine a fixed-point format structure corresponding to the original data according to the integer significant digits and the mantissa significant digits of the fixed-point format;

[0010] Converting the original data into fixed-point format data and storing the data into the fixed-point format structure to obtain converted data;

[0011] When the data type of the original data is in fixed-point format, or when the converted data is obtained, the original data and / or the converted data are input into a preset normalized exponential function to obtain the target data.

[0012] In one embodiment, determining the fixed-point format structure corresponding to the original data according to the number of valid digits of the integer and the number of valid digits of the mantissa in the fixed-point format includes:

[0013] Acquire a first standard structure of a fixed-point format; wherein the first standard structure includes a sign bit, an integer reserved bit, a decimal point bit, and a mantissa reserved bit;

[0014] According to the number of significant digits of the integer and the number of significant digits of the mantissa in the fixed-point format, the integer reserved bits and the mantissa reserved bits in the first standard structure are adjusted respectively to obtain a fixed-point format structure corresponding to the original data.

[0015] In one embodiment, establishing a first standard structure of a fixed-point format includes:

[0016] Get the exponent threshold and mantissa threshold of the floating point format data in the input raw data;

[0017] Determining, according to the exponential threshold, the number of integer reserved bits of the first standard structure;

[0018] Determine the number of reserved mantissa bits of the first standard structure according to the exponent threshold and the mantissa threshold;

[0019] A first standard structure of a fixed-point format is established according to the number of integer reserved bits and the number of mantissa reserved bits.

[0020] In one embodiment, the step of obtaining an exponent threshold and a mantissa threshold of floating-point format data in the input original data includes:

[0021] Get the exponent bit width and mantissa bit width of the floating point format data in the input original data;

[0022] Determining an exponent threshold of floating-point format data according to the exponent bit width;

[0023] A mantissa threshold of the floating-point format data is determined according to the mantissa bit width.

[0024] In one embodiment, the data type further includes: integer format, and after obtaining the input original data and the corresponding data type, the method further includes:

[0025] When the data type is integer data, obtaining an integer part and a sign part corresponding to the original data;

[0026] The integer part and the sign part are stored in a second standard structure in a fixed-point format to obtain conversion data.

[0027] In one embodiment, establishing a second standard structure in a fixed-point format includes:

[0028] Get the integer threshold of the integer format data in the input raw data;

[0029] Determine the number of integer digits of the second standard structure according to the integer threshold;

[0030] A second standard structure of the fixed-point format is established according to the number of integer bits.

[0031] In a second aspect, the present application further provides a data processing device. The device comprises:

[0032] A data acquisition module is used to obtain the input raw data and the corresponding data type;

[0033] An exponent number confirmation module, used for obtaining the exponent and mantissa of the original data when the data type is in floating point format;

[0034] A valid digit determination module, used to determine the number of valid digits of an integer in a fixed-point format corresponding to the original data according to the exponent; and to determine the number of valid digits of a mantissa in a fixed-point format corresponding to the original data according to the exponent and the mantissa;

[0035] A data structure confirmation module, used to determine the fixed-point format structure corresponding to the original data according to the integer valid digits and the mantissa valid digits of the fixed-point format;

[0036] A conversion data confirmation module, used for converting the original data into fixed-point format data, and storing the data into the fixed-point format structure to obtain conversion data;

[0037] The data calculation module is used to input the original data and / or the converted data into a preset normalized exponential function to obtain target data when the data type of the original data is in fixed-point format or when the converted data is obtained.

[0038] In one embodiment, the data structure confirmation module includes:

[0039] A standard structure confirmation submodule, used to obtain a first standard structure in a fixed-point format; wherein the first standard structure includes a sign bit, an integer reserved bit, a decimal point bit, and a mantissa reserved bit;

[0040] The data structure confirmation submodule is used to adjust the integer reserved bits and the mantissa reserved bits in the first standard structure according to the integer valid bits and the mantissa valid bits of the fixed-point format, so as to obtain the fixed-point format structure corresponding to the original data.

[0041] In one embodiment, the device further comprises a standard structure establishing module, which is used to:

[0042] Get the exponent threshold and mantissa threshold of the floating point format data in the input raw data;

[0043] Determining, according to the exponential threshold, the number of integer reserved bits of the first standard structure;

[0044] Determine the number of reserved mantissa bits of the first standard structure according to the exponent threshold and the mantissa threshold;

[0045] A first standard structure of a fixed-point format is established according to the number of integer reserved bits and the number of mantissa reserved bits.

[0046] In one embodiment, the standard structure establishment module includes:

[0047] A bit width confirmation submodule is used to obtain the exponent bit width and mantissa bit width of the floating point format data in the input original data;

[0048] A threshold confirmation submodule, used to determine an exponent threshold of floating-point format data according to the exponent bit width;

[0049] The threshold confirmation submodule is further used to determine the mantissa threshold of the floating point format data according to the mantissa bit width.

[0050] In one embodiment, the data type further includes: an integer format, and the device further includes an integer conversion module for:

[0051] When the data type is integer data, obtaining an integer part and a sign part corresponding to the original data;

[0052] The integer part and the sign part are stored in a second standard structure in a fixed-point format to obtain conversion data.

[0053] In one embodiment, the device further comprises a standard structure establishing module, which is used to:

[0054] Get the integer threshold of the integer format data in the input raw data;

[0055] Determine the number of integer digits of the second standard structure according to the integer threshold;

[0056] A second standard structure of the fixed-point format is established according to the number of integer bits.

[0057] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the data processing method as described in any one of the embodiments of the present disclosure is implemented.

[0058] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the data processing method described in any one of the embodiments of the present disclosure is implemented.

[0059] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the data processing method as described in any one of the embodiments of the present disclosure is implemented.

[0060] The above data processing method, device, computer equipment, storage medium and computer program product obtain the input raw data and data type; when the data type is a floating-point format, the exponent and mantissa of the floating-point format are used to determine the fixed-point format corresponding to the raw data. After converting the floating-point format to the fixed-point format, or when the raw data is a fixed-point format, the converted fixed-point format or the corresponding raw data is input into the normalized exponential function to obtain the target function. By determining the fixed-point format structure corresponding to the raw data, it is ensured that the minimum bit width is used to achieve lossless conversion from the floating-point format to the fixed-point format, thereby improving the accuracy of data processing. Through the above data processing method, it is possible to achieve a common set of computing modules for fixed-point format data and floating-point format data, that is, data processing of fixed-point format and floating-point format can be achieved through one hardware resource, thereby avoiding the need to use two hardware resources to achieve data conversion and data processing respectively in the prior art, optimizing the utilization of hardware resources, and improving the efficiency of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] FIG1 is a schematic flow chart of a data processing method in one embodiment;

[0062] FIG2 is a schematic diagram of half-floating point format data in one embodiment;

[0063] FIG3 is a schematic diagram of a flow chart of fixed-point format data confirmation in one embodiment;

[0064] FIG4 is a schematic diagram of a first standard structure in one embodiment;

[0065] FIG5 is a schematic diagram of a process for establishing a first standard structure in one embodiment;

[0066] FIG6 is a schematic diagram of a process for obtaining an exponent threshold and a mantissa threshold in one embodiment;

[0067] FIG7 is a schematic diagram of a flow chart of integer data conversion in one embodiment;

[0068] FIG8 is a schematic diagram of a process for establishing a second standard structure in one embodiment;

[0069] FIG9 is a schematic diagram of a second standard structure in one embodiment;

[0070] FIG10 is a schematic diagram of a flow chart of a data processing method according to an embodiment;

[0071] FIG11 is a block diagram of a data processing device in one embodiment;

[0072] FIG. 12 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0074] In one embodiment, as shown in FIG1 , a data processing method is provided. This embodiment is illustrated by applying the method to a terminal. It is understandable that the method can also be applied to a server, or to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0075] Step S100, obtaining input original data and corresponding data types.

[0076] In an exemplary embodiment, the data type may include a fixed-point format, a floating-point format, an integer format, etc. Wherein, the data type corresponds to the original data, for example, if the original data is input as data in a fixed-point format, the data type is marked as a fixed-point format, etc. In an exemplary embodiment, the original data and the data type may include input by manual means and subsequent calculations. In another exemplary embodiment, the original data and the data type may be obtained by confirming or converting the data type of the original data before collecting, acquiring, and calculating the original data, and directly inputting it into the data processing system, etc. For example, if the cleaned data A needs to be input into the data processing, the output data type may be set after cleaning, and after output, it may be directly input into the data processing system for data processing, etc.

[0077] Step S200, when the data type is in floating point format, obtaining the exponent and mantissa of the original data.

[0078] In an exemplary embodiment, the data type may include a data identifier, and a data identifier is used to identify the floating point format. For example, "identifier 1" indicates a fixed point format, "identifier 2" indicates a floating point format, and "identifier 3" indicates an integer format. It can be understood that the floating point format includes a sign bit, an exponent bit, and a mantissa bit. Specifically, taking the half-precision floating point format as an example, the half-precision floating point format can be shown in FIG. 2, with a sign bit S, five exponent bits, and ten mantissa bits. Specifically, S represents the sign bit, e represents the exponent bit, and m represents the mantissa bit. The data of the half-precision floating point number can be represented as (-1) S *2 e-15 * (1+m), that is, the fixed-point format data corresponding to the half-precision floating-point number can be obtained through the above formula.

[0079] Step S300, determining the number of significant digits of an integer in a fixed-point format corresponding to the original data according to the exponent; and determining the number of significant digits of a mantissa in a fixed-point format corresponding to the original data according to the exponent and the mantissa.

[0080] In an exemplary embodiment, taking the half-precision floating point format as an example, the data in the half-precision floating point format is represented as (-1) S *2 e-15 * (1+m), where S represents the sign bit, e represents the exponent bit, and m represents the mantissa bit; specifically, m is greater than 0 and less than 1, so 1+m must be less than 2; so 2 e-15 * (1+m) must be less than 2 e-14 ; Therefore, in actual use, e determines the power of 2, that is, the number of digits of the integer after conversion to fixed-point format. For example, e-15 is equal to 5, so 2 e-15 * (1+m) must be greater than 2 5 and less than 2 6 , so only five integer bits (binary) are needed to represent the integer bit after conversion to fixed-point format, that is, the integer valid bit is 5; specifically, when the integer bit is greater than 5, the extra bits are all 0, which is an invalid bit; and when the integer bit is less than 5, since the converted fixed-point format contains five integer bits, data loss will occur. Therefore, the exponent bit of the floating-point format can be used to determine the integer valid bit of the converted fixed-point format. Similarly, since m is the decimal place, and 2 e-15 It may also be a decimal, so the number of significant digits of the mantissa in the converted fixed-point format can be determined by m (mantissa) and e (exponent). Specifically, when m is a ten-digit decimal, e-15 is -5, and 2 -5 * m is 15, which is a decimal, so the number of significant digits in the mantissa of the converted fixed-point format is 15, etc.

[0081] Step S400: determining a fixed-point format structure corresponding to the original data according to the number of significant digits of the integer in the fixed-point format and the number of significant digits of the mantissa.

[0082] In an exemplary embodiment, the fixed-point format structure corresponding to the original data may include a sign bit, an integer bit, a decimal point bit and a mantissa bit; and they are the sign bit, the integer bit, the decimal point bit and the mantissa bit in sequence; it can be understood that the sign bit and the decimal point bit can both be represented by a bit width of one bit, so when the number of significant digits of the integer and the number of significant digits of the mantissa are determined, the fixed-point format structure corresponding to the original data can be established and determined.

[0083] Step S500: convert the original data into fixed-point format data, and store the data into the fixed-point format structure to obtain converted data.

[0084] In an exemplary embodiment, the floating point format can be converted into a fixed point format by a corresponding formula, for example, a half-precision floating point format, by (-1) S *2 e-15 * (1+m), the conversion from the original data to the fixed-point format data can be realized. Therefore, the method of converting the floating-point format data into the fixed-point format data is not limited here.

[0085] Step S600, when the data type of the original data is in fixed-point format, or when the converted data is obtained, the original data and / or the converted data are input into a preset normalized exponential function to obtain target data.

[0086] In an exemplary embodiment, after obtaining the converted data, the fixed-point format data converted from the original data is obtained. Therefore, after the original data is in a fixed-point format or the original data is converted to a fixed-point format, the corresponding fixed-point format data can be input into a normalized exponential function to obtain target data, etc. Among them, the normalized exponential function may include a softmax function, etc.; specifically, the Softmax function, as an indispensable component in the attention mechanism, can be used for the calculation of the attention mechanism. It can be understood that after the softmax function is transformed, the analysis value interval is made between [0,1] through subtraction operations, avoiding large fluctuations in values. Due to the addition of subtraction operations, the prior art usually converts floating-point data into fixed-point format for operation. Therefore, the present solution can use fewer hardware resources to implement operations on floating-point data, fixed-point data, etc.

[0087] In an exemplary embodiment, the data processing method can be applied to a hardware, wherein the hardware can realize the conversion of non-fixed-point data into fixed-point format data, and can also perform normalized exponential function processing on the fixed-point format data. It can be understood that when performing normalized exponential function processing on data, it is necessary to ensure that the input data type is consistent. Therefore, before inputting into the normalized exponential function, the non-fixed-point format data needs to be converted into fixed-point format data.

[0088] In the above data processing method, the input raw data and data type are obtained; when the data type is a floating-point format, the exponent and mantissa of the floating-point format are used to determine the fixed-point format corresponding to the raw data. After converting the floating-point format to the fixed-point format, or when the raw data is a fixed-point format, the converted fixed-point format or the corresponding raw data is input into the normalized exponential function to obtain the target function. By determining the fixed-point format structure corresponding to the raw data, it is ensured that the minimum bit width is used to achieve lossless conversion from the floating-point format to the fixed-point format, thereby improving the accuracy of data processing. Through the above data processing method, it is possible to achieve a common set of computing modules for fixed-point format data and floating-point format data, that is, data processing of fixed-point format and floating-point format can be achieved through one hardware resource, thereby avoiding the need to use two hardware resources to achieve data conversion and data processing respectively in the prior art, thereby optimizing the utilization of hardware resources and improving the efficiency of data processing.

[0089] In one embodiment, as shown in FIG3 , determining the fixed-point format structure corresponding to the original data according to the number of valid digits of the integer and the number of valid digits of the mantissa in the fixed-point format includes:

[0090] Step S401, obtaining a first standard structure of a fixed-point format; wherein the first standard structure includes a sign bit, an integer reserved bit, a decimal point bit, and a mantissa reserved bit.

[0091] In an exemplary embodiment, the first standard structure may include a structure for converting a floating-point format into a fixed-point format, etc.; specifically, it may be as shown in FIG. 4 , which includes a sign bit S; fifteen integer reserved bits, a decimal point bit (the 1 here is the implicit 1 before the decimal point in the floating-point format, which is placed at the decimal point), and a mantissa reserved bit, etc.

[0092] Step S402, adjusting the integer reserved bits and the mantissa reserved bits in the first standard structure according to the integer significant digits and the mantissa significant digits of the fixed-point format, respectively, to obtain a fixed-point format structure corresponding to the original data.

[0093] In an exemplary embodiment, the first standard structure has integer reserved bits and mantissa reserved bits. When determining the number of integer significant digits and mantissa significant digits, the integer reserved bits and mantissa reserved bits can be adjusted. For example, if the number of mantissa significant digits is 10 and the number of integer significant digits is 5, then the integer bits of the first standard structure are adjusted to 5 bits, and the mantissa bits are adjusted to 10 bits, and so on.

[0094] In this embodiment, the first standard structure is adjusted by the integer significant digits and the mantissa significant digits to obtain the fixed-point format structure corresponding to the original data. It is possible to achieve lossless conversion from floating-point format to fixed-point format with the minimum bit width by the integer significant digits and the mantissa significant digits, thereby improving the accuracy and practicality of data processing.

[0095] In one embodiment, as shown in FIG5 , establishing a first standard structure of a fixed-point format includes:

[0096] Step S411, obtaining an exponent threshold and a mantissa threshold of floating-point format data in the input original data.

[0097] Step S412: determining the number of integer reserved bits of the first standard structure according to the exponential threshold.

[0098] Step S413: determining the number of reserved mantissa bits of the first standard structure according to the exponent threshold and the mantissa threshold.

[0099] Step S414: establishing a first standard structure of a fixed-point format according to the number of reserved integer bits and the number of reserved mantissa bits.

[0100] In an exemplary embodiment, the exponent threshold of the floating-point format can determine the number of integer digits after the floating-point format is converted to a fixed-point format, for example, the exponent threshold is -14 to 15; then the number of integer digits is at most 15 digits; the exponent threshold and mantissa threshold of the floating-point format can determine the number of mantissa digits after the floating-point format is converted to a fixed-point format, for example, the exponent threshold is -14 to 15; the mantissa threshold is 0 to 0.11111111111 (ten digits after the decimal point), then the mantissa is at most -14 for the exponent, and ten digits for the mantissa, here the number of mantissa digits is 24 bits, therefore, the integer reserved bits and mantissa reserved bits after the floating-point format is converted to a fixed-point format can be obtained through the exponent threshold and the mantissa threshold of the floating-point format.

[0101] In an exemplary embodiment, the first annotation structure may include a sign bit, an integer reserved bit, a decimal point bit, and a mantissa reserved bit. It can be understood that the sign bit and the decimal point bit are both known, so when the integer reserved bit and the mantissa reserved bit are determined, the first standard structure can be established.

[0102] In this embodiment, the integer reserved bits and the mantissa reserved bits of the fixed-point format after conversion are determined by the exponent threshold and the mantissa threshold of the floating-point format, and the first standard structure for converting the floating-point format into the fixed-point format is further established, so that the fixed-point format structure can be dynamically adjusted according to the threshold of the input floating-point format data to ensure its adaptability and flexibility. By determining the exponent threshold and the mantissa threshold, the setting of the integer and mantissa reserved bits can be optimized, thereby maximizing the use of storage space and improving data transmission and calculation efficiency. At the same time, the first standard structure is established by the exponent threshold and the mantissa threshold, avoiding data loss caused by the conversion of the floating-point format into the fixed-point format, and improving the accuracy of data conversion.

[0103] In one embodiment, as shown in FIG6 , the step of obtaining the exponent threshold and mantissa threshold of the floating point format data in the input original data includes:

[0104] Step S421, obtaining the exponent bit width and mantissa bit width of the floating point format data in the input original data.

[0105] In an exemplary embodiment, the exponent bit width and the mantissa bit width can be obtained by setting the original data to the bit width of the floating point format before establishing data processing, wherein the set bit width may include the exponent bit width and the mantissa bit width, and it can be understood that the bit width can represent the number of exponent bits and the number of mantissa bits; specifically, the corresponding exponent threshold and mantissa threshold can be determined by the number of exponent bits and the number of mantissa bits; for example, when the exponent bit is 3 bits, the exponent threshold is 000-111; that is, 0-7, etc.; in general, the mantissa bit width is usually larger than the exponent bit width, for example, the mantissa bit width is 10, the exponent bit width is 5, etc.

[0106] Step S422: determining an exponent threshold of fixed-point format data according to the exponent bit width.

[0107] Step S423: determining a mantissa threshold of the fixed-point format data according to the mantissa bit width.

[0108] In this embodiment, the exponent threshold and mantissa threshold of the fixed-point format are determined respectively by the exponent bit width and mantissa bit width of the floating-point format data in the input original data. The exponent bit width and mantissa bit width can be used to accurately know the exponent threshold and mantissa threshold of the floating-point format, thereby improving the accuracy of the exponent threshold and mantissa threshold, and further improving the accuracy of data processing.

[0109] In one embodiment, as shown in FIG. 7 , the data type further includes: integer format, and after obtaining the input original data and the corresponding data type, the following is further included:

[0110] Step S701, when the data type is integer data, obtain the integer part and the sign part corresponding to the original data.

[0111] Step S702: Store the integer part and the sign part into a second standard structure in a fixed-point format to obtain conversion data.

[0112] In an exemplary embodiment, the integer data may include an integer bit and a sign bit. It is understandable that the integer data does not include decimals. Therefore, when determining the data of the integer part and the sign part, the integer part and the sign part may be directly stored in the second standard structure, and the data converted from the integer data to the fixed-point format may be directly obtained. In an exemplary embodiment, the second standard structure may include a template for converting the integer data to the fixed-point format, that is, including the integer part and the sign part.

[0113] In this embodiment, the integer part and the sign part of the integer data are respectively extracted and stored in the second standard structure of the fixed-point format, which helps to improve the efficiency and accuracy of data processing.

[0114] In one embodiment, as shown in FIG8 , establishing a second standard structure of a fixed-point format includes:

[0115] Step S711, obtaining an integer threshold of integer format data in the input original data.

[0116] Step S712: determining the number of integer digits of the second standard structure according to the integer threshold.

[0117] Step S713: establishing a second standard structure in fixed-point format according to the integer bit number.

[0118] In an exemplary embodiment, the integer threshold may include setting the bit width of the integer data before setting the data processing, and determining the integer threshold of the integer data by the bit width of the integer data; for example, if the bit width is 6, it means that there are five integer bits (one sign bit) in the integer data, then the integer threshold is 00000-11111, that is, 0-31, therefore, the second standard structure also only needs five integer bits to represent the integer part of the integer data, etc.

[0119] In an exemplary embodiment, when the number of integer bits of the second standard structure is determined, a corresponding second standard structure can be established. It can be understood that the second standard structure can include only a sign bit and an integer bit, and the sign bit can be represented by one bit, so when the integer bit is determined, the second standard structure can be easily established. Specifically, for example, when the integer part is fifteen bits, the second standard structure can be as shown in Figure 9, including a 1-bit sign bit S, and 15 integer bits, and can also include a vacant bit, etc., wherein the vacant bit can be used to extend data, identification, placeholders, and ensure data consistency and readability. Specifically, the vacant bit can reserve one bit, which provides convenience for future format changes, making the system easier to expand; it can also be used as a vacant bit or identification bit to distinguish different data types; or as a placeholder, quickly adjust the data structure when needed to adapt to different computing needs, etc.

[0120] In this embodiment, by obtaining the integer threshold of the original integer data and determining the second standard structure accordingly, the storage efficiency and the calculation speed can be optimized, while the numerical accuracy and the specific needs can be enhanced. This method makes the data structure more compact, the operation more efficient, reduces the errors caused by overflow or precision loss, and helps to achieve the best balance between performance, storage and data accuracy, thereby improving the overall performance and reliability of the system.

[0121] In an exemplary embodiment, the data processing method may be applied to a hardware resource, as shown in the flowchart of FIG10 , and specifically includes:

[0122] Step S901, input data and corresponding data type;

[0123] Step S902, determine the data type, if it is integer data or floating point data, proceed to step S903; if it is fixed point format data, proceed to step S905;

[0124] Step S903, inputting the integer data or floating point data into a data conversion module;

[0125] Step S904, the data conversion module converts the integer data or floating point data into fixed point format data;

[0126] Step S905, sending the fixed-point format data converted from the integer data or floating-point format data, or the input fixed-point format data, to the general calculation module;

[0127] Step S906: the general calculation module calculates the data and outputs the calculation result.

[0128] Specifically, it can be understood that the data processing method is applied to one hardware resource, that is, data of different data types can be input into the hardware resource for calculation, etc. This avoids the low utilization of hardware resources caused by the traditional need of one hardware resource for data conversion and one hardware resource for calculation.

[0129] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0130] Based on the same inventive concept, the embodiment of the present application also provides a data processing device for implementing the data processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more data processing device embodiments provided below can refer to the limitations on the data processing method above, and will not be repeated here.

[0131] In one embodiment, as shown in FIG11 , a data processing device 100 is provided, comprising: a data acquisition module 101, an exponent digit confirmation module 102, a valid digit determination module 103, a data structure confirmation module 104, a conversion data confirmation module 105 and a data calculation module 106, wherein:

[0132] A data acquisition module is used to obtain the input raw data and the corresponding data type;

[0133] An exponent number confirmation module, used for obtaining the exponent and mantissa of the original data when the data type is in floating point format;

[0134] A valid digit determination module, used to determine the number of valid digits of an integer in a fixed-point format corresponding to the original data according to the exponent; and to determine the number of valid digits of a mantissa in a fixed-point format corresponding to the original data according to the exponent and the mantissa;

[0135] A data structure confirmation module, used to determine the fixed-point format structure corresponding to the original data according to the integer valid digits and the mantissa valid digits of the fixed-point format;

[0136] A conversion data confirmation module, used for converting the original data into fixed-point format data, and storing the data into the fixed-point format structure to obtain conversion data;

[0137] The data calculation module is used to input the original data and / or the converted data into a preset normalized exponential function to obtain target data when the data type of the original data is in fixed-point format or when the converted data is obtained.

[0138] In one embodiment, the data structure confirmation module includes:

[0139] A standard structure confirmation submodule, used to obtain a first standard structure in a fixed-point format; wherein the first standard structure includes a sign bit, an integer reserved bit, a decimal point bit, and a mantissa reserved bit;

[0140] The data structure confirmation submodule is used to adjust the integer reserved bits and the mantissa reserved bits in the first standard structure according to the integer valid bits and the mantissa valid bits of the fixed-point format, so as to obtain the fixed-point format structure corresponding to the original data.

[0141] In one embodiment, the device further comprises a standard structure establishing module, which is used to:

[0142] Get the exponent threshold and mantissa threshold of the floating point format data in the input raw data;

[0143] Determining, according to the exponential threshold, the number of integer reserved bits of the first standard structure;

[0144] Determine the number of reserved mantissa bits of the first standard structure according to the exponent threshold and the mantissa threshold;

[0145] A first standard structure of a fixed-point format is established according to the number of integer reserved bits and the number of mantissa reserved bits.

[0146] In one embodiment, the standard structure establishment module includes:

[0147] A bit width confirmation submodule is used to obtain the exponent bit width and mantissa bit width of the floating point format data in the input original data;

[0148] A threshold confirmation submodule, used to determine an exponent threshold of floating-point format data according to the exponent bit width;

[0149] The threshold confirmation submodule is further used to determine the mantissa threshold of the floating point format data according to the mantissa bit width.

[0150] In one embodiment, the data type further includes: an integer format, and the device further includes an integer conversion module for:

[0151] When the data type is integer data, obtaining an integer part and a sign part corresponding to the original data;

[0152] The integer part and the sign part are stored in a second standard structure in a fixed-point format to obtain conversion data.

[0153] In one embodiment, the device further comprises a standard structure establishing module, which is used to:

[0154] Get the integer threshold of the integer format data in the input raw data;

[0155] Determine the number of integer digits of the second standard structure according to the integer threshold;

[0156] A second standard structure of the fixed-point format is established according to the number of integer bits.

[0157] Each module in the above data processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0158] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG12. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store target data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a data processing method is implemented.

[0159] Those skilled in the art will understand that the structure shown in FIG. 12 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0160] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0161] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0162] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data processing method, characterized in that: The method comprises: Get the input raw data and the corresponding data type; When the data type is in floating point format, obtaining the exponent and mantissa of the original data; Determine the number of significant digits of an integer in a fixed-point format corresponding to the original data according to the exponent; and determine the number of significant digits of a mantissa in a fixed-point format corresponding to the original data according to the exponent and the mantissa; Determine a fixed-point format structure corresponding to the original data according to the integer significant digits and the mantissa significant digits of the fixed-point format; Converting the original data into fixed-point format data and storing the data into the fixed-point format structure to obtain converted data; When the data type of the original data is in fixed-point format, or when the converted data is obtained, the original data and / or the converted data are input into a preset normalized exponential function to obtain the target data.

2. The method according to claim 1, characterized in that The determining, according to the integer significant digits of the fixed-point format and the mantissa significant digits, the fixed-point format structure corresponding to the original data comprises: Acquire a first standard structure of a fixed-point format; wherein the first standard structure includes a sign bit, an integer reserved bit, a decimal point bit, and a mantissa reserved bit; According to the number of significant digits of the integer and the number of significant digits of the mantissa in the fixed-point format, the integer reserved bits and the mantissa reserved bits in the first standard structure are adjusted respectively to obtain a fixed-point format structure corresponding to the original data.

3. The method according to claim 2, characterized in that Establish the first standard structure of fixed-point format, including: Get the exponent threshold and mantissa threshold of the floating point format data in the input raw data; Determining, according to the exponential threshold, the number of integer reserved bits of the first standard structure; Determine the number of reserved mantissa bits of the first standard structure according to the exponent threshold and the mantissa threshold; A first standard structure of a fixed-point format is established according to the number of integer reserved bits and the number of mantissa reserved bits.

4. The method according to claim 3, characterized in that: The step of obtaining the exponent threshold and mantissa threshold of floating point format data in the input original data includes: Get the exponent bit width and mantissa bit width of the floating point format data in the input original data; Determining an exponent threshold of floating-point format data according to the exponent bit width; A mantissa threshold of the floating-point format data is determined according to the mantissa bit width.

5. The method according to claim 1, characterized in that The data type also includes: integer format, and after obtaining the input original data and the corresponding data type, it also includes: When the data type is integer data, obtaining an integer part and a sign part corresponding to the original data; The integer part and the sign part are stored in a second standard structure in a fixed-point format to obtain conversion data.

6. The method according to claim 4, characterized in that Create a second standard structure for fixed-point format, including: Get the integer threshold of the integer format data in the input raw data; Determine the number of integer digits of the second standard structure according to the integer threshold; A second standard structure of the fixed-point format is established according to the number of integer bits.

7. A data processing device, characterized in that: The device comprises: A data acquisition module is used to obtain the input raw data and the corresponding data type; An exponent number confirmation module, used for obtaining the exponent and mantissa of the original data when the data type is in floating point format; A valid digit determination module, used to determine the number of valid digits of an integer in a fixed-point format corresponding to the original data according to the exponent; and to determine the number of valid digits of a mantissa in a fixed-point format corresponding to the original data according to the exponent and the mantissa; A data structure confirmation module, used to determine the fixed-point format structure corresponding to the original data according to the integer valid digits and the mantissa valid digits of the fixed-point format; A conversion data confirmation module, used for converting the original data into fixed-point format data, and storing the data into the fixed-point format structure to obtain conversion data; The data calculation module is used to input the original data and / or the converted data into a preset normalized exponential function to obtain target data when the data type of the original data is in fixed-point format or when the converted data is obtained.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images