A floating point data processing method, system, medium and device
By directly converting floating-point data into integer data for storage and retrieval, the problems of large errors and slow speed in traditional methods are solved, achieving efficient data processing and real-time reading.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 粤港澳大湾区(广东)国创中心
- Filing Date
- 2026-02-04
- Publication Date
- 2026-06-05
Smart Images
Figure CN122152267A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of floating-point data processing technology, and in particular to a floating-point data processing method, system, medium, and device. Background Technology
[0002] With the explosive growth of fields such as the Internet of Things and industrial monitoring, time-series data is growing exponentially. A single device can generate thousands of timestamped indicator data per second. Based on the characteristics of time-series data, such as strong temporal locality (small changes between adjacent values), high repetition of indicator values (such as steady-state values of sensors), and fixed label dimensions, columnar storage of time-series data is adopted to store all data of the same indicator (column) continuously.
[0003] Traditional floating-point number encoding compression uses differential encoding (such as Delta, Delta of Delta) to convert the original value into a smaller difference, and then quantizes or uses variable-length encoding (such as Gorilla, CHIMP algorithms) to significantly reduce storage redundancy. However, floating-point data needs to undergo multiple differential encoding and decoding processes during storage and retrieval, which can lead to a large error between the read and restored floating-point data and the original stored floating-point data. Furthermore, it can increase computational latency in retrieval scenarios, resulting in slow reading speeds.
[0004] Therefore, traditional floating-point data processing methods rely on multiple differential encoding and decoding to store and retrieve data, resulting in large errors between the retrieved and stored data, as well as slow data retrieval speed, making real-time reading impossible. Summary of the Invention
[0005] In order to overcome the shortcomings of the prior art, the present invention aims to provide a floating-point data processing method, system, medium and device that can realize data storage and retrieval without multiple differential encoding and decoding, reduce the error between the read data and the stored data, improve the data reading speed and realize real-time data reading.
[0006] This invention is implemented according to the following scheme:
[0007] A floating-point data processing method is provided, including: Obtain an initial set of metadata, including symbol space markers, precision features, and order-of-magnitude features; Establish the association between the initial metadata set and the storage device, determine the storage characteristics based on the initial metadata set, and obtain a target metadata set including the symbol space marker, the precision characteristics, the order of magnitude characteristics, and the storage characteristics; In response to a storage instruction, the floating-point data to be stored is converted into integer data according to the target metadata set, and the integer data and corresponding storage parameters are stored in the data sequence of the storage device. The storage parameters include storage features, and the storage instruction is used to store the floating-point data to be stored. In response to a read instruction including read parameters, the read parameters having corresponding integer data to be read, the integer data to be read is inversely converted to floating-point data according to the target metadata set and the read instruction, and then read, the integer data to be read is stored in the data sequence of the storage device.
[0008] Optionally, the symbol space markers include: The first space marker is used to represent a floating-point data type where all floating-point data are negative. The second space marker is used to indicate the floating-point data type, including negative numbers; The third space marker is used to indicate floating-point data types that do not include negative numbers.
[0009] Optionally, the storage features include storage type and storage size; Based on the initial metadata set, storage characteristics are determined, including: The storage type is determined based on the symbol space marker; The storage size is determined based on the accuracy characteristics.
[0010] Optionally, determining the storage type based on the symbol space marker includes: When the symbol space is marked as the first space mark, the storage type is an int type used to indicate signed integers or a uint type used to indicate unsigned integers; When the symbol space is marked as the second space mark, the storage type is an int type used to indicate a signed integer; When the symbol space is labeled as the third space label, the storage type is a uint type used to indicate unsigned integers.
[0011] Optionally, the precision feature includes the number of significant decimal places; Based on the target metadata set, the floating-point data to be stored is converted into integer data, including: Based on the aforementioned order-of-magnitude characteristics, determine the magnitude factor; Determine the precision factor based on the number of significant decimal places; The floating-point data to be stored is converted into integer data based on the symbol space marker, the storage feature, the magnitude factor, and the precision magnitude factor.
[0012] Optionally, based on the target metadata set and the read instruction, the integer data to be read is inversely converted to floating-point data, including: Based on the read parameters, determine the storage location index in the storage parameters; Based on the storage location index, determine the integer data to be read in the data sequence of the storage device; Based on the target metadata set, the integer data to be read is inversely converted into the floating-point data.
[0013] Optionally, based on the target metadata set, the integer data to be read is inversely converted into the floating-point data, including: Based on the aforementioned order-of-magnitude characteristics, determine the inverse order-of-magnitude factor; Determine the inverse precision factor based on the number of significant decimal places; Based on the symbol space marker, the storage feature, the inverse magnitude factor, and the inverse precision magnitude factor, the integer data to be read is inversely converted into the floating-point data.
[0014] A floating-point data processing system is also provided, which applies the aforementioned floating-point data processing method, including: The metadata collection retrieval module is used for: Obtain an initial set of metadata, including symbol space markers, precision features, and order-of-magnitude features; Establish the association between the initial metadata set and the storage device, determine the storage characteristics based on the initial metadata set, and obtain a target metadata set including the symbol space marker, the precision characteristics, the order of magnitude characteristics, and the storage characteristics; The storage module, in response to a storage instruction, converts the floating-point data to be stored into integer data according to the target metadata set, and then stores the integer data and corresponding storage parameters in the data sequence of the storage device. The storage parameters include storage features, and the storage instruction is used to store the floating-point data to be stored. The read module, in response to a read instruction including read parameters having corresponding integer data to be read, converts the integer data to be read into floating-point data according to the target metadata set and the read instruction, and then reads it. The integer data to be read is stored in the data sequence of the storage device.
[0015] A computer-readable storage medium is also provided, which is a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed, implements the aforementioned floating-point data processing method.
[0016] A computer device is also provided, including a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the floating-point data processing method described above.
[0017] Compared with the prior art, the beneficial effects of the floating-point data processing method of the present invention are as follows: by using the target metadata set, the floating-point data to be stored is directly converted into integer data for storage, without relying on multiple differential encoding and decoding operations, avoiding the cumulative error generated during differential encoding and decoding, greatly reducing the error between the read and restored floating-point data and the original stored floating-point data, and improving the accuracy of data processing; at the same time, the elimination of differential encoding and decoding calculation steps reduces the calculation latency during data reading and speeds up data reading. Attached Figure Description
[0018] Figure 1 This is a flowchart of the floating-point data processing method of the present invention; Figure 2 This is a diagram illustrating how to obtain the target metadata set from the initial metadata set. Figure 1 ; Figure 3 This is a diagram illustrating how to obtain the target metadata set from the initial metadata set. Figure 2 . Detailed Implementation
[0019] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.
[0020] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.
[0021] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.
[0022] See Figure 1 As shown, a floating-point data processing method of the present invention includes: S1: Obtain an initial metadata set including symbol space markers, precision features, and order-of-magnitude features; wherein, the symbol space markers are used to represent the symbol distribution of the data, the precision features are used to represent the precision constraint rules of the floating-point data, and the order-of-magnitude features are used to represent the order-of-magnitude scaling rules of the floating-point data.
[0023] In one embodiment of the present invention, the symbol space markers include: a first space marker, which takes a value of -1, used to represent a floating-point data type where all floating-point data is negative, that is, only including the negative number data type; a second space marker, which takes a value of 0, used to represent a floating-point data type where floating-point data includes negative numbers, that is, including positive numbers, 0, and negative numbers; and a third space marker, which takes a value of 1, used to represent a floating-point data type where floating-point data does not include negative numbers, that is, including positive numbers and 0.
[0024] In one embodiment of the present invention, the precision feature includes the number of significant digits of the floating-point data and the number of significant digits of the decimal point; the order-of-magnitude feature includes the order-of-magnitude base and the order-of-magnitude exponent. The order-of-magnitude factor for converting floating-point data to integer data is determined by the order-of-magnitude base and the order-of-magnitude exponent. For example, when 10 or 2 is used as the order-of-magnitude base and n is used as the order-of-magnitude exponent, the order-of-magnitude factor includes 10 in decimal order of magnitude. n Or binary-scale representation of 2 n .
[0025] S2: Establish the association between the initial metadata set and the storage device, determine the storage characteristics based on the initial metadata set, and obtain the target metadata set including symbol space tags, precision characteristics, order of magnitude characteristics and storage characteristics; In one embodiment of the present invention, the storage characteristics include storage type and storage size; determining the storage characteristics based on the initial metadata set includes: determining the storage type based on symbol space markings; and determining the storage size based on precision characteristics. In one embodiment of the present invention, determining the storage type based on the symbol space marker includes: when the symbol space marker is a first space marker, the storage type is an int type for indicating signed integers or a uint type for indicating unsigned integers; when the symbol space marker is a second space marker, the storage type is an int type for indicating signed integers; and when the symbol space marker is a third space marker, the storage type is a uint type for indicating unsigned integers.
[0026] In one embodiment of the present invention, the precision features include the number of significant decimal places and the number of significant floating-point numbers; see also Figure 2-3 The diagram illustrates how the target metadata set is derived from the initial metadata set. Figure 2-3 As shown, when the symbol space is marked as the third space (1), it only includes two data types: positive numbers and 0. No symbol representation is required, and the storage type used is unit type. like Figure 2 As shown, when the floating-point number has 4 significant digits and 3 significant decimal digits, the maximum value of the floating-point data is 9.999, which is 9999 when converted to integer data. If the storage size is 8 bits (1 byte), combined with the unit storage type, the range of the stored data is 0-255, where 9999 is greater than 255 and does not include the integer data 9999. If the storage size is 16 bits (2 bytes), combined with the unit storage type, the range of the stored data is 0-65535, where 9999 is less than 65535 and includes the integer data 9999. In this case, 16 bits are determined as the storage size, that is, the storage type is unit16. like Figure 3 As shown, when the floating-point number has 18 significant digits and the decimal point has 17 significant digits, the maximum value of the floating-point data is 9.9999999999999999, which is 9999999999999999999. If the storage size is 32 bits (4 bytes), combined with the unit storage type, the range of the stored data is 0-4294967295. 999999999999999999 is greater than 4294967295, but does not include 999999. The integer data 999999999999; if the storage size is 64 bits (5 bytes), combined with the unit storage type, the range of the stored data is 0-18446744073709551615. 99999999999999999 is less than 18446744073709551615, including the integer data 999999999999999999. In this case, 64 bits are determined as the storage size, that is, the storage type is unit64.
[0027] In one embodiment of the invention, the storage feature further includes a write index, which indicates the position of the floating-point data in the data sequence when the floating-point data is stored in the storage device. For example, when the write index is 0, it is located at the first position in the data sequence. Figure 2-3 The write index shown is used to indicate that the data sequence stores floating-point data starting from the first bit. The value of the write index is increased sequentially. When reading multiple data in the data sequence in the future, the data can be directly restored according to the write index. There is no need to go through multiple differential encoding and decoding to restore the integer data to floating-point data before it can be read, which effectively improves the reading speed.
[0028] S3: In response to the storage command, the floating-point data to be stored is converted into integer data according to the target metadata set, and the integer data and corresponding storage parameters are stored in the data sequence of the storage device. The storage parameters include storage characteristics, and the storage command is used to store the floating-point data to be stored. In one embodiment of the present invention, the precision feature includes the number of significant decimal places; converting the floating-point data to be stored into integer data according to the target metadata set includes: determining the magnitude factor according to the magnitude feature; determining the precision magnitude factor according to the number of significant decimal places; and converting the floating-point data to be stored into integer data according to the symbol space label, storage feature, magnitude factor and precision magnitude factor.
[0029] In one embodiment of the present invention, determining the magnitude factor based on the magnitude characteristics includes: obtaining the magnitude factor based on the magnitude base and magnitude exponent in the magnitude characteristics; determining the precision magnitude factor based on the significant number of decimal places includes: using the significant number of decimal places as the precision exponent and the magnitude base as the precision base to obtain the precision magnitude factor.
[0030] In one embodiment of the present invention, converting floating-point data to be stored into integer data according to symbol space markers, storage characteristics, magnitude factors, and precision magnitude factors includes: When the symbol space is marked as the first space mark (-1), the floating-point data to be stored is multiplied sequentially by the symbol space mark, the magnitude factor, and the precision magnitude factor to obtain integer data. Then, the storage characteristics and the position (write index) of the floating-point data to be stored in the time-series floating-point data sequence are used as storage parameters for the integer data. The position of the integer data stored in the data sequence of the storage device is determined according to the write index. This invention increases the range of storage type selection by multiplying the floating-point data with the symbol space mark when all floating-point data are negative.
[0031] When the symbol space is marked as the second space mark (0) or the third space mark (1), the floating-point data to be stored is multiplied by the magnitude factor and the precision magnitude factor in sequence to obtain the integer data. Then, the storage characteristics and the position of the floating-point data to be stored in the time-series floating-point data sequence (write subscript) are used as the storage parameters of the integer data. The position of the integer data stored in the data sequence of the storage device is determined according to the write subscript.
[0032] S4: In response to a read command including read parameters, the read parameters having corresponding integer data to be read, the integer data to be read is inversely converted to floating-point data according to the target metadata set and the read command, and then read. The integer data to be read is stored in the data sequence of the storage device.
[0033] In one embodiment of the present invention, the process of inversely converting the integer data to be read into floating-point data according to the target metadata set and the read instruction includes: determining the storage location index in the storage parameters according to the read parameters; determining the integer data to be read in the data sequence of the storage device according to the storage location index; and inversely converting the integer data to be read into floating-point data according to the target metadata set.
[0034] In one embodiment of the present invention, the storage location index is the write index in the data sequence of the storage device after the floating-point data is converted into integer data, so as to realize the rapid location of integer data in the data sequence and the inverse conversion of the integer data into floating-point data by means of the storage location index (write index), thereby effectively improving the data reading speed.
[0035] This invention directly converts floating-point data to be stored into integer data for storage through a target metadata set, eliminating the need for multiple differential encoding and decoding operations. This avoids the accumulated errors generated during differential encoding and decoding, significantly reducing the error between the read and restored floating-point data and the original stored floating-point data, thus improving the accuracy of data processing. Simultaneously, the read operation also eliminates the need for differential encoding and decoding calculations. By using storage parameters stored along with the integer data, the invention enables rapid location of the integer data to be read within the data sequence. It avoids the need for multiple differential encoding and decoding processes on multiple integer data preceding the integer data to be read; only the inverse conversion of the integer data to be read is required, effectively reducing computational latency during data reading and accelerating the data reading speed.
[0036] In one embodiment of the present invention, the process of inversely converting integer data to be read into floating-point data based on a target metadata set includes: determining an inverse magnitude factor based on magnitude characteristics; determining an inverse precision magnitude factor based on the number of significant decimal places; and inversely converting the integer data to be read into floating-point data based on symbol space markings, storage characteristics, inverse magnitude factor, and inverse precision magnitude factor.
[0037] In one embodiment of the present invention, determining the inverse magnitude factor based on the order-of-magnitude characteristics includes: obtaining a magnitude factor based on the order-of-magnitude base and order-of-magnitude exponent in the order-of-magnitude characteristics, and using the reciprocal of the magnitude factor as the inverse magnitude factor; determining the inverse precision magnitude factor based on the significant decimal places includes: using the significant decimal places as the precision exponent and the order-of-magnitude base as the precision base to obtain a precision magnitude factor, and using the reciprocal of the precision magnitude factor as the inverse precision magnitude factor.
[0038] In one embodiment of the present invention, the integer data to be read is inversely converted into floating-point data according to the symbol space marker, storage characteristics, inverse magnitude factor, and inverse precision magnitude factor, including: When the storage type in the storage feature is an unsigned integer of type uint, and the sign space is marked as the first space mark (-1), the integer data to be read is multiplied by the sign space mark, the inverse magnitude factor, and the inverse precision magnitude factor in sequence to obtain the floating-point data; When the storage type in the storage feature is the int type of a signed integer, and the sign space is marked as the first space mark (-1), the integer data to be read is multiplied by the inverse magnitude factor and the inverse precision magnitude factor in sequence to obtain the floating-point data; When the symbol space is marked as the second space mark (0) or the third space mark (1), the integer data to be read is multiplied by the inverse magnitude factor and the inverse precision magnitude factor in sequence to obtain the floating-point data.
[0039] The present invention provides a floating-point data processing system, which applies the above-described floating-point data processing method, comprising: The metadata collection retrieval module is used for: Obtain an initial set of metadata, including symbol space markers, precision features, and order-of-magnitude features; Establish the association between the initial metadata set and the storage device, determine the storage characteristics based on the initial metadata set, and obtain the target metadata set including symbol space tags, precision characteristics, order of magnitude characteristics and storage characteristics; The storage module, in response to a storage instruction, converts the floating-point data to be stored into integer data according to the target metadata set, and then stores the integer data and the corresponding storage parameters in the data sequence of the storage device. The storage parameters include storage characteristics, and the storage instruction is used to store the floating-point data to be stored. The read module responds to a read command that includes read parameters, which contain corresponding integer data to be read. Based on the target metadata set and the read command, the integer data to be read is inversely converted to floating-point data and then read. The integer data to be read is stored in the data sequence of the storage device.
[0040] This invention discloses a computer-readable storage medium storing a computer program thereon, which, when executed, implements the floating-point data processing method.
[0041] Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM).
[0042] The computer device of the present invention includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set, and the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the above-described floating-point data processing method.
[0043] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0044] The memory can be used to store the computer program or module. The processor implements various functions of the floating-point data processing method by running or executing the computer program or module stored in the memory and calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, SmartMediaCard (SMC), Secure Digital (SD) card, FlashCard, at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0045] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A floating-point data processing method, characterized in that, include: Obtain an initial set of metadata, including symbol space markers, precision features, and order-of-magnitude features; Establish the association between the initial metadata set and the storage device, determine the storage characteristics based on the initial metadata set, and obtain a target metadata set including the symbol space marker, the precision characteristics, the order of magnitude characteristics, and the storage characteristics; In response to a storage instruction, the floating-point data to be stored is converted into integer data according to the target metadata set, and the integer data and corresponding storage parameters are stored in the data sequence of the storage device. The storage parameters include storage features, and the storage instruction is used to store the floating-point data to be stored. In response to a read instruction including read parameters, the read parameters having corresponding integer data to be read, the integer data to be read is inversely converted to floating-point data according to the target metadata set and the read instruction, and then read, the integer data to be read is stored in the data sequence of the storage device.
2. The floating-point data processing method according to claim 1, characterized in that, The symbol space markers include: The first space marker is used to represent a floating-point data type where all floating-point data are negative. The second space marker is used to indicate the floating-point data type, including negative numbers; The third space marker is used to indicate floating-point data types that do not include negative numbers.
3. The floating-point data processing method according to claim 2, characterized in that, The storage characteristics include storage type and storage size; Based on the initial metadata set, storage characteristics are determined, including: The storage type is determined based on the symbol space marker; The storage size is determined based on the accuracy characteristics.
4. The floating-point data processing method according to claim 3, characterized in that, Determining the storage type based on the symbol space marker includes: When the symbol space is marked as the first space mark, the storage type is an int type used to indicate signed integers or a uint type used to indicate unsigned integers; When the symbol space is marked as the second space mark, the storage type is an int type used to indicate a signed integer; When the symbol space is labeled as the third space label, the storage type is a uint type used to indicate unsigned integers.
5. A floating-point data processing method according to claim 2, characterized in that, The precision feature includes the number of significant decimal places; Based on the target metadata set, the floating-point data to be stored is converted into integer data, including: Based on the aforementioned order-of-magnitude characteristics, determine the magnitude factor; Determine the precision factor based on the number of significant decimal places; The floating-point data to be stored is converted into integer data based on the symbol space marker, the storage feature, the magnitude factor, and the precision magnitude factor.
6. The floating-point data processing method according to claim 1, characterized in that, Based on the target metadata set and the read instruction, the integer data to be read is inversely converted to floating-point data, including: Based on the read parameters, determine the storage location index in the storage parameters; Based on the storage location index, determine the integer data to be read in the data sequence of the storage device; Based on the target metadata set, the integer data to be read is inversely converted into the floating-point data.
7. A floating-point data processing method according to claim 6, characterized in that, Based on the target metadata set, the integer data to be read is inversely converted into the floating-point data, including: Based on the aforementioned order-of-magnitude characteristics, determine the inverse order-of-magnitude factor; Determine the inverse precision factor based on the number of significant decimal places; Based on the symbol space marker, the storage feature, the inverse magnitude factor, and the inverse precision magnitude factor, the integer data to be read is inversely converted into the floating-point data.
8. A floating-point data processing system, employing the floating-point data processing method according to any one of claims 1-7, characterized in that, include: The metadata collection retrieval module is used for: Obtain an initial set of metadata, including symbol space markers, precision features, and order-of-magnitude features; Establish the association between the initial metadata set and the storage device, determine the storage characteristics based on the initial metadata set, and obtain a target metadata set including the symbol space marker, the precision characteristics, the order of magnitude characteristics, and the storage characteristics; The storage module, in response to a storage instruction, converts the floating-point data to be stored into integer data according to the target metadata set, and then stores the integer data and corresponding storage parameters in the data sequence of the storage device. The storage parameters include storage features, and the storage instruction is used to store the floating-point data to be stored. The read module, in response to a read instruction including read parameters having corresponding integer data to be read, converts the integer data to be read into floating-point data according to the target metadata set and the read instruction, and then reads it. The integer data to be read is stored in the data sequence of the storage device.
9. A computer-readable storage medium, characterized in that, It is a computer-readable storage medium on which a computer program is stored, which, when executed, implements a floating-point data processing method as described in any one of claims 1-7.
10. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded and executed by the processor to implement a floating-point data processing method as described in any one of claims 1 to 7.