Data processing method and device applied to database system

Abstract

One or more embodiments of the specification provide a data processing method and device applied to a database system. After triggering a data type conversion task, the data processing method obtains a to-be-converted floating point number of the data type conversion task, converts the to-be-converted floating point number into a first floating point number represented by an extended precision, and then calculates a second floating point number scaled to a preset numerical interval through multiple loop integer calculations to perform accurate rounding. According to whether the decimal part of the intermediate value of the last loop integer calculation and the distance between the upper interval and the lower interval of the last loop integer calculation are both greater than the error value of the last loop integer calculation, in the case of satisfying the error value of the last loop integer calculation, the second floating point number is converted into a fixed point number according to the integer part of the intermediate value of each loop integer calculation and the rounding result of the last loop integer calculation, to serve as the execution result of the data type conversion task.

Description

Technical Field

[0001] This specification relates to the field of database system technology, and in particular to a data processing method and apparatus for use in database systems. Background Technology

[0002] Currently, in various databases, when performing high-precision calculation tasks, it is usually necessary to first perform a data type conversion task to convert floating-point data to fixed-point data. That is, it is necessary to convert floating-point numbers to fixed-point numbers through the execution of the data type conversion task. In order to ensure the conversion accuracy, it is necessary to ensure that the obtained fixed-point number can be converted back to the original floating-point number through certain methods, that is, after the floating-point number is converted to a fixed-point number, it can be converted back to the original floating-point number without causing a loss of precision.

[0003] In related technologies, when converting floating-point numbers to fixed-point numbers through data type conversion tasks, it cannot be guaranteed that the obtained fixed-point numbers can be exchanged with the original floating-point numbers. Therefore, how to convert floating-point numbers to fixed-point numbers through data type conversion tasks while ensuring accuracy is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] In view of the above, one or more embodiments of this specification provide a data processing method and apparatus for use in a database system.

[0005] To achieve the above objectives, one or more embodiments of this specification provide the following technical solutions:

[0006] According to a first aspect of one or more embodiments of this specification, a data processing method applied to a database system is provided, the database system including storage services, computing services, and metadata management services, the method applied to the computing services, including:

[0007] Obtain the target computing task, and query the metadata management service based on the target computing task to determine the data storage location information of the data to be processed required by the target computing task;

[0008] The data to be processed is obtained from the storage service according to the data storage location information;

[0009] If the data to be processed includes floating-point numbers to be converted, a data type conversion task is triggered to convert the floating-point numbers to fixed-point numbers. This data type conversion task is then performed in the following manner so that the corresponding converted fixed-point numbers can be used to continue completing the target computation task:

[0010] Convert the floating-point number to be converted into a first floating-point number represented by extended precision;

[0011] The first floating-point number is scaled to a preset value range to obtain a second floating-point number; the preset value range is used to round the first mantissa of the first floating-point data.

[0012] Based on the mantissa and exponent of the first floating-point number, the initial error value generated when scaling the first floating-point number is determined, and the upper and lower intervals of the second floating-point number are determined based on the mantissa and exponent of the second floating-point number.

[0013] The second floating-point number is subjected to multiple cyclic rounding calculations. Each cyclic rounding calculation includes: determining whether the decimal part of the intermediate value of the current cyclic rounding calculation meets the rounding condition based on the upper and lower intervals corresponding to the current cyclic rounding calculation; if the decimal part of the intermediate value of the current cyclic rounding calculation meets the rounding condition, the decimal part of the intermediate value of the current cyclic rounding calculation is rounded, and the cyclic rounding calculation stops; if the decimal part of the intermediate value of the current cyclic rounding calculation does not meet the rounding condition, the next cyclic rounding calculation is performed. The upper and lower intervals corresponding to the current cyclic rounding calculation are ten times the upper and lower intervals corresponding to the previous cyclic rounding calculation, and the intermediate value of the current cyclic rounding calculation is ten times the decimal part of the intermediate value of the previous cyclic rounding calculation. The upper and lower intervals corresponding to the first cyclic rounding calculation are the upper and lower intervals of the second floating-point number, and the intermediate value of the first cyclic rounding calculation is the second floating-point number.

[0014] In response to the determination that the distance between the decimal part of the intermediate value of the last rounding calculation and both the upper and lower intervals of the last rounding calculation are greater than the error value of the last rounding calculation, the second floating-point number is converted into a fixed-point number based on the integer part of the intermediate value of each rounding calculation and the rounding result of the last rounding calculation, as the execution result of the data type conversion task; wherein, the error value of each rounding calculation is determined by multiplying the initial error value by an integer multiple of ten for the current number of rounding calculations.

[0015] According to a second aspect of one or more embodiments of this specification, a data processing apparatus for a database system is provided, the database system including storage services, computing services, and metadata management services, the apparatus being applied to the computing services, comprising:

[0016] The query module obtains the target computing task and queries the metadata management service based on the target computing task to determine the data storage location information of the data to be processed required by the target computing task.

[0017] The acquisition module acquires the data to be processed from the storage service based on the data storage location information;

[0018] The execution module, when the data to be processed includes floating-point numbers to be converted, triggers a data type conversion task to convert the floating-point numbers to fixed-point numbers, and executes the data type conversion task in the following manner so that the corresponding converted fixed-point numbers can be used to continue completing the target calculation task:

[0019] Convert the floating-point number to be converted into a first floating-point number represented by extended precision;

[0020] The first floating-point number is scaled to a preset value range to obtain a second floating-point number; the preset value range is used to round the first mantissa of the first floating-point data.

[0021] Based on the mantissa and exponent of the first floating-point number, the initial error value generated when scaling the first floating-point number is determined, and the upper and lower intervals of the second floating-point number are determined based on the mantissa and exponent of the second floating-point number.

[0022] The second floating-point number is subjected to multiple cyclic rounding calculations. Each cyclic rounding calculation includes: determining whether the decimal part of the intermediate value of the current cyclic rounding calculation meets the rounding condition based on the upper and lower intervals corresponding to the current cyclic rounding calculation; if the decimal part of the intermediate value of the current cyclic rounding calculation meets the rounding condition, the decimal part of the intermediate value of the current cyclic rounding calculation is rounded, and the cyclic rounding calculation stops; if the decimal part of the intermediate value of the current cyclic rounding calculation does not meet the rounding condition, the next cyclic rounding calculation is performed. The upper and lower intervals corresponding to the current cyclic rounding calculation are ten times the upper and lower intervals corresponding to the previous cyclic rounding calculation, and the intermediate value of the current cyclic rounding calculation is ten times the decimal part of the intermediate value of the previous cyclic rounding calculation. The upper and lower intervals corresponding to the first cyclic rounding calculation are the upper and lower intervals of the second floating-point number, and the intermediate value of the first cyclic rounding calculation is the second floating-point number.

[0023] In response to the determination that the distance between the decimal part of the intermediate value of the last rounding calculation and both the upper and lower intervals of the last rounding calculation are greater than the error value of the last rounding calculation, the second floating-point number is converted into a fixed-point number based on the integer part of the intermediate value of each rounding calculation and the rounding result of the last rounding calculation, as the execution result of the data type conversion task; wherein, the error value of each rounding calculation is determined by multiplying the initial error value by an integer multiple of ten for the current number of rounding calculations.

[0024] According to a third aspect of one or more embodiments of this specification, an electronic device is provided, comprising:

[0025] processor;

[0026] Memory used to store processor-executable instructions;

[0027] The processor implements the method as described in the first aspect by running the executable instructions.

[0028] According to a fourth aspect of one or more embodiments of this specification, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0029] As can be seen from the above embodiments, the data processing method and apparatus for database systems provided in one or more embodiments of this specification, after triggering a data type conversion task to convert the floating-point number to a fixed-point number, obtains the floating-point number to be converted for the data type conversion task, and converts the floating-point number to be converted into a first floating-point number represented by extended precision. This extended precision provides a floating-point format with higher precision than the basic floating-point format, which can accurately represent all the mantissas of the floating-point number in the basic floating-point format, providing round-trip assurance for subsequent conversion calculations directly based on the extended precision floating-point number. At the same time, the second floating-point number scaled to a preset numerical range is precisely rounded through multiple loop rounding calculations, and the accuracy of the data type conversion task is determined based on whether the distance between the decimal part of the middle value of the last loop rounding calculation and the upper and lower intervals of the last loop rounding calculation are both greater than the error value of the last loop rounding calculation. Furthermore, since this data type conversion task involves direct rounding calculations based on floating-point numbers, the operation speed of the data type conversion task is improved, ensuring accurate data processing in the database system. Attached Figure Description

[0030] Figure 1 This is an exemplary embodiment of a data processing system architecture applied to a database system.

[0031] Figure 2A This is a schematic flowchart of a data processing method applied to a database system, provided as an exemplary embodiment.

[0032] Figure 2B This is a flowchart illustrating an exemplary embodiment of a method for executing a data type conversion task.

[0033] Figure 3 This is a flowchart illustrating another data processing method applied to a database system, provided by an exemplary embodiment.

[0034] Figure 4 This is a schematic diagram of the structure of a device provided in an exemplary embodiment.

[0035] Figure 5 This is a block diagram of a data processing apparatus applied to a database system, provided as an exemplary embodiment. Detailed Implementation

[0036] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0037] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0038] As described in the background section, when performing high-precision calculations in a database, it is usually necessary to first perform a data type conversion task to convert floating-point data to fixed-point data. However, in current related technologies, when converting floating-point numbers to fixed-point numbers through data type conversion tasks, it cannot be guaranteed that the resulting fixed-point numbers can be exchanged with the original floating-point numbers, thus affecting the calculation results of the high-precision calculations performed earlier in the database. In some embodiments of this specification, if the floating-point numbers to be converted are first converted into fractions represented by large integers with infinite precision before performing data type conversion, it not only increases the computation time but also involves dynamic memory allocation and occupies a large amount of storage space.

[0039] Therefore, in one or more embodiments of this description, a data processing method for database systems is proposed. After triggering a data type conversion task to convert the floating-point number to a fixed-point number, the floating-point number to be converted is obtained, and then converted into a first floating-point number represented by extended precision. This extended precision provides a higher precision floating-point format than the basic floating-point format, accurately representing all the mantissas of the floating-point number in the basic floating-point format. This provides round-trip assurance for subsequent conversion calculations directly based on the extended precision floating-point number. Moreover, performing round-robin operations directly on the extended precision floating-point format results in faster operation, requires less fixed space, and eliminates the need for dynamic memory allocation. Through multiple round-robin calculations, the second floating-point number, scaled to a preset numerical range, is precisely rounded, ensuring that the converted fixed-point number maintains the minimum number of significant digits. Simultaneously, based on the mantissa and exponent of the first floating-point number, the initial error value generated during scaling of the first floating-point number is determined. Then, by calculating the number of rounding operations in the loop, the error value of the final rounding operation is determined. This error value ensures that the error generated during data type conversion operations will not affect the normal round-trip between the fixed-point number and the original floating-point number. Specifically, by checking whether the distance between the decimal part of the middle value of the final rounding operation and both the upper and lower intervals of the final rounding operation are greater than the error value of the final rounding operation, it is determined whether the data type conversion task can guarantee that the obtained fixed-point number and the original floating-point number can make round-trips. This ensures the accuracy of the data type conversion task and further provides a foundation for accurate data processing in the database system.

[0040] Figure 1 This is a schematic diagram of the architecture of a data processing service system applied to a database system, provided as an exemplary embodiment. Figure 1 As shown, the system may include a server 11, a network 12, and several electronic devices, such as a PC (Personal Computer) 13, a mobile phone 14, etc.

[0041] Server 11 can be a physical server containing an independent host, or it can be a virtual server hosted in a host cluster. During operation, server 11 can run server-side programs for a specific application to implement the relevant functions of that application. For example, when server 11 runs an executable program for data processing in a database system, it can function as a corresponding data processing service platform.

[0042] PC13 and mobile phone14 are just some of the types of electronic devices that users can use. In reality, users can obviously also use electronic devices such as tablets, laptops, PDAs (Personal Digital Assistants), wearable devices (such as smart glasses, smartwatches, etc.), etc., and one or more embodiments in this specification do not limit this. During operation, the electronic device can run a client-side program of a certain application to realize the relevant functions of that application. For example, when the electronic device runs a program related to the data processing services applied to the database system, it can realize a client service for that related program. In some embodiments, the related program may be a statistical analysis of a certain amount of data based on the execution results of the data processing applied to the database system. The client application of the aforementioned related program business service can be launched and run on the electronic device. The client-side program can be a native application installed on the electronic device, or it can be a mini-program, quick app, or other similar form. Of course, when using web technologies such as HTML5 or similar, the relevant functions can be realized through a page displayed by a browser. Here, the browser can be a standalone browser application or a browser module embedded in some applications.

[0043] As for the network 12 that enables interaction between electronic devices such as PC13 and mobile phone 14 and server 11, communication can be achieved using either wired or wireless networks, depending on the communication methods supported by the respective electronic devices. This specification does not impose any restrictions on this. For example, PC13 can support both wired and wireless communication, so it can use either wired or wireless networks as needed. Mobile phone 14 typically only supports wireless communication, so it can use a wireless network for communication.

[0044] refer to Figure 2A This specification provides a data processing method applied to a database system, which includes storage services, computing services, and metadata management services. The method is applied to the computing services. The method includes the following steps:

[0045] S202, obtain the target computing task, and query the metadata management service based on the target computing task to determine the data storage location information of the data to be processed required by the target computing task.

[0046] In a database system, the computing service is responsible for executing queries, calculations, data processing, and coordination tasks; the storage service is responsible for the physical storage and retrieval of data; and the metadata management service is responsible for managing system metadata. The metadata management service specifically includes: data storage location information, system topology information (the association between computing nodes and storage nodes), and node status information (node ​​load, health status, etc.). When the database system is a single-machine system, the computing service, storage service, and metadata service can run on the same node of the database system. When the database system is a distributed system, the computing service, storage service, and metadata service can be distributed across different nodes of the database system. One node in the database system can correspond to one host machine.

[0047] In some embodiments, the target computing task can be any computing task executed by the computing service load in the database system. To complete the target computing task, it is necessary to further determine the data storage location information of the data to be processed required by the target computing task. Since the metadata management service includes the data storage location information of each data, the target computing task can be used to query the metadata management service to obtain the data storage location information of the data to be processed required by the target computing task. In some embodiments, the data to be processed required by the target computing task can be determined first, and then the metadata management service can be queried based on the data to be processed. Alternatively, the target computing task can be directly queried based on the metadata management service. In this case, the metadata management service also includes the data storage location information of the data to be processed required by each computing task.

[0048] S204, Obtain the data to be processed from the storage service according to the data storage location information.

[0049] Since the storage service is responsible for the physical storage and retrieval of data, after determining the data storage location information of the data to be processed required by the target computing task through the metadata management service, the data to be processed can be obtained from the storage service through the data storage location information.

[0050] S206, if the data to be processed includes a floating-point number to be converted, a data type conversion task to convert the floating-point number to a fixed-point number is triggered and executed so that the corresponding converted fixed-point number is used to continue to complete the target calculation task.

[0051] When the data to be processed includes floating-point numbers to be converted, in order to ensure the accuracy of the target computation task, the floating-point numbers need to be converted to fixed-point numbers first. Therefore, a data type conversion task to convert the floating-point numbers to fixed-point numbers will be triggered. After the data type conversion task is triggered, the computation service will execute the data type conversion task so that the fixed-point numbers obtained through the data type conversion task can be used to continue completing the target computation task.

[0052] refer to Figure 2B This is a flowchart illustrating an execution method for a data type conversion task provided in an embodiment of this specification. The execution method for the data type conversion task includes the following steps:

[0053] S2062, convert the floating-point number to be converted into a first floating-point number represented by extended precision.

[0054] In some embodiments, the floating-point number to be converted is the object of the current data type conversion task. The basic floating-point formats generally include single-precision floating-point (Float) and double-precision floating-point (Double) types. According to the binary floating-point arithmetic standard (IEEE 754), the double-precision floating-point (Double) type contains 64 binary bits, including 1 sign bit, 11 exponent bits, and 52 mantissa bits. To accurately represent a Double type value using decimal numbers, there must be enough decimal bits to represent all 52 mantissa bits; that is, 17 significant decimal digits are needed to accurately represent the 52 binary digits. Since single-precision floating-point numbers have lower precision than double-precision floating-point numbers, 17 significant decimal digits are sufficient for accurate representation.

[0055] Extended precision provides a higher precision floating-point format than the basic floating-point format, typically referring to 80-bit extended precision. 80-bit extended precision is supported on most processors and compilers, offering performance similar to Double, and is represented by the Long Double type. This type contains 63 mantissa bits, approximately 18 decimal bits of precision, sufficient to accurately represent the entire mantissa of the standard 64-bit Double type. Therefore, converting a floating-point number to its extended precision representation allows for direct type conversion operations on the floating-point number itself. This not only improves the speed of data type conversion tasks but also saves significant time and space compared to using large integers with infinite precision, as it eliminates the need for homomorphic memory allocation.

[0056] It should be noted that in some embodiments of this specification, the extended precision can refer to more than 80-bit extended precision, such as 128-bit extended precision, and there is no limitation on this.

[0057] In some embodiments of this specification, obtaining the floating-point number to be converted for the data type conversion task includes:

[0058] In response to the presence of a floating-point number to be calculated in the target computing task, the floating-point number to be converted in the data type conversion task is obtained from the target computing task;

[0059] Wherein, the floating-point number to be converted is any floating-point number to be calculated that appears in the target calculation task; the target calculation task completes the calculation by converting the floating-point number to be calculated into a fixed-point number during the execution process.

[0060] Data type conversion tasks primarily provide the basis for high-precision calculations within a database. Therefore, to accurately obtain the floating-point number to be converted in the data type conversion task, it's necessary to monitor whether the target calculation task contains the floating-point number to be calculated. If so, the floating-point number to be converted in the data type conversion task is retrieved from that target calculation task. It's important to note that the target calculation task can be any calculation task within a database or a system containing a database. This database could be Oceanbase or MySQL, among others. The floating-point number to be converted in the data type conversion task can be any floating-point number appearing in the target calculation task. When there are multiple floating-point numbers to be calculated, each of these is considered a floating-point number to be converted. During execution, the target calculation task first needs to convert the floating-point number to be calculated into a fixed-point number using the data type conversion task, and then use the resulting fixed-point number to complete the calculation.

[0061] S2064, scale the first floating-point number to a preset value range to obtain a second floating-point number; the preset value range is used to round the first digit of the first floating-point data.

[0062] To facilitate cyclic rounding calculations on the first floating-point number, it needs to be scaled to a preset value range to obtain the second floating-point number. When scaling the first floating-point number to the preset value range, it can be scaled repeatedly using multiples of ten until the scaling result falls within the preset value range. It should be noted that this preset value range is mainly used for rounding the first digit of the first floating-point data. In some embodiments, this preset value range can be set to [1, 10) or [0.1, 1).

[0063] In some embodiments of this specification, scaling the first floating-point number to a preset numerical range includes:

[0064] The first floating-point number is scaled to a first scaling ratio within a preset numerical range using a logarithmic function.

[0065] The first floating-point number is scaled to a preset numerical range based on the first scaling ratio.

[0066] To improve the efficiency of scaling the first floating-point number to a preset numerical range, a first scaling ratio for scaling the first floating-point number to the preset numerical range can be determined first using a logarithmic function, and then... n To represent the first scaling factor mentioned above, we can use the logarithmic function: To calculate the value of n, where v represents the first floating-point number mentioned above.

[0067] In some embodiments of this specification, the logarithmic function includes:

[0068]

[0069] or,

[0070] Wherein, the first scaling ratio is 10 n e v This represents the exponent of the first floating-point number; Represents log 10 The estimated value of 2, 10 -x Represents log 10 2 and The preset error value.

[0071] Considering that, in passing Calculating the value of n requires not only a large amount of computation, but also, for floating-point numbers, the computation... The accuracy of the calculation results cannot be guaranteed. Therefore, in this embodiment of the application, by... A series of derivations were performed to obtain... and Compared to the previous logarithmic function, these two function formulas require only a few arithmetic operations to obtain the value of n, thus improving the computer's calculation speed for the first scaling factor. The specific derivation process is as follows:

[0072]

[0073] Using the change-of-base formula, we can change the base of the logarithmic function:

[0074]

[0075] The problem now becomes finding the value of log2 v, where v is a Long Double value, and the mantissa and exponent of this value are m and m respectively. v and e vAccording to IEEE 754:

[0076]

[0077] Where, m v <2 64 Therefore, we can obtain:

[0078]

[0079] Next to Scaling is applied to represent

[0080] According to the principle of rounding down, we know

[0081] That is, log2 v-1 <e v -16383≤log2 v formula (5);

[0082] Multiply both sides of formula (5) by a positive number log. 10 2. The inequality sign remains unchanged:

[0083] log2 v*log 10 2-log 10 2<(e v -16383)*log 10 2≤log2 v*log 10 2 Formula (6);

[0084] Taking the logarithmic function of formula (6) with a different base, we get:

[0085] log 10 v-log 10 2<(e v -16383)*log 10 2≤log 10 v Formula (7);

[0086] Rounding down both sides, we get:

[0087]

[0088] Due to log 10 2 < 1, therefore Therefore, formula (8) can be transformed into:

[0089]

[0090] Substituting formula (1) into formula (9) yields:

[0091]

[0092] Adding 1 to both sides of inequality (10) yields:

[0093]

[0094] Combining formulas (10) and (11), we can obtain:

[0095]

[0096] because Since n is an integer, we can conclude that:

[0097]

[0098] That is, the above formula (A) can be obtained through formula (13).

[0099] Considering that for floating-point numbers, log 10 2. It still cannot be accurately represented in a computer, meaning there is a certain degree of error, which may lead to... It is not true, but it will certainly not lead to This is not true. To resolve the log issue... 10 2. The problem that it still cannot be accurately represented in a computer and has a certain error exists is addressed in this embodiment by making a small offset of 10 to the above formula (12). -x ,get:

[0100]

[0101] Since all three terms in formula (14) are integers, we can obtain:

[0102]

[0103] That is, the above formula (B) can be obtained through formula (15).

[0104] It should be noted that, and 10 -x The values ​​can all be set as needed, among which, The settings can be configured according to specific accuracy requirements; for example, in some embodiments... It can be set to 0.3010.10 -x Represents log 10 2 and The preset error value, this 10 -x The value can be determined by log. 10 2. The first maximum error value in the computer, and what does not affect The second largest error value, which is rounded down, is used to determine this, i.e., 10.-x The value of x is between the first maximum error value and the second maximum error value mentioned above. For example, in some embodiments, the value of x can be any integer between 6 and 10.

[0105] In some embodiments of this specification, scaling the first floating-point number to a preset numerical range based on the first scaling ratio includes:

[0106] The first floating-point number is scaled according to the first scaling ratio, and it is determined whether the scaling result is within the preset value range;

[0107] In response to determining that the scaling result is within the preset numerical range, the second floating-point number is obtained based on the scaling result;

[0108] In response to determining that the scaling result is not within the preset numerical range, the exponent of the first scaling ratio is increased by 1, and the first floating-point number is scaled according to the first scaling ratio after the exponent is increased by 1 to obtain the second floating-point number.

[0109] As can be seen from the above formula (13) or formula (15), the exponent of the first scaling ratio has two values ​​in actual calculation, and the difference between these two values ​​is 1. In the above formula (A) and formula (B), the first term of formula (13) and formula (15) is selected for calculation. Therefore, in this embodiment, the first floating-point number is first scaled using the first scaling ratio, and it is determined whether the scaling result is within the preset value range. If it is, the scaling result is used as the second floating-point number. If it is not, the first floating-point number is first increased by 1 according to the exponent, that is, the second term of formula (13) and formula (15) is obtained. Then, the first floating-point number is scaled according to the first scaling ratio after increasing the exponent by 1, and the second floating-point number can be obtained at this time.

[0110] It should be noted that in floating-point arithmetic, the result of directly scaling a floating-point number by 1000 times is different from the result of first scaling the floating-point number by 100 times and then by 10 times. This is because each multiplication of floating-point numbers introduces an error. Therefore, in the above embodiment, when the scaling result obtained by directly scaling the first floating-point number using the first scaling ratio is not within the preset value range, the scaling is not continued directly based on the result. Instead, the exponent of the first scaling ratio is increased by 1 before scaling. This can further reduce the error generated during the scaling process.

[0111] In some embodiments of this specification, scaling the first floating-point number to a preset numerical range based on the first scaling ratio includes:

[0112] The first floating-point number is scaled according to the first scaling ratio, and it is determined whether the scaling result is within the preset value range;

[0113] In response to determining that the scaling result is within the preset numerical range, the second floating-point number is obtained based on the scaling result;

[0114] In response to determining that the scaling result is not within the preset numerical range, the scaling result is scaled by a factor of ten to obtain the second floating-point number.

[0115] Unlike the scaling embodiments described above, in this embodiment, if directly scaling the first floating-point number using the first scaling ratio results in a scaling result that is not within a preset value range, the scaling result is further scaled by a factor of ten to obtain the second floating-point number. This reduces one exponent addition operation. It should be noted that the scaling mentioned in this application can be either magnification or reduction. When the floating-point number before scaling is less than the preset range, the corresponding scaling refers to magnification; when the floating-point number before scaling is greater than the preset range, the corresponding scaling refers to reduction.

[0116] S2066, based on the mantissa and exponent of the first floating-point number, determine the initial error value generated when scaling the first floating-point number, and based on the mantissa and exponent of the second floating-point number, determine the upper and lower intervals of the second floating-point number.

[0117] Considering that due to the characteristics of floating-point numbers, extended precision will still have some error, and this error will propagate down the calculation process, leading to errors in the final conversion result, this specification calculates and maintains a maximum error value in several embodiments to determine whether the current conversion calculation is accurate.

[0118] Assuming the first floating-point number is v and the scaling factor is s, the scaling process is v*s, where v and s are both Long Double values. Because the Long Double type can perfectly represent Double values, v can be considered accurate. However, the value of s may exceed the range that 17 significant decimal digits can represent, therefore s contains an error. According to the rules of Long Double... Assume u s The error is in 's', because the mantissa of a Long Double type has 63 bits, so... The maximum error caused by s in the corresponding v*s can be determined by... Let's determine this. Assume the error caused by s to v*s is u1, then we have:

[0119]

[0120] The maximum error caused by s to v*s can be obtained by formula (16), which is the initial error value generated when the first floating-point number is scaled.

[0121] In floating-point numbers, the upper and lower intervals represent the distances between the floating-point number and the two values ​​with the smallest difference, respectively. For example, the upper and lower intervals of the floating-point number 10.23499965667724609375 (0x4123c28f) are 10.2350006103515625 (0x4123c290) and 10.2349987030029296875 (0x4123c28e), respectively. When determining the upper and lower intervals of the second floating-point number based on its mantissa and exponent, we can first determine whether the second floating-point number is a power of 2. If it is, the corresponding upper interval is: m + =2 e1-m1 The lower interval is: m - =2 e1-m1-1 If it is not a power of 2, then the corresponding upper interval is: m + =2 e1-m1 The lower interval is: m - =2 e1-m1 Where e1 represents the exponent of the second floating-point number, and m1 represents the length of the mantissa of the second floating-point number. For example, the length of the mantissa of an 80-bit extended precision floating-point number is 63.

[0122] In some embodiments of this specification, the initial error value generated when scaling the first floating-point number is determined based on the mantissa and exponent of the first floating-point number, including:

[0123] The first maximum error value generated by scaling the first floating-point number according to the first scaling ratio or by scaling the first floating-point number according to the first scaling ratio plus 1 is calculated using the following formula:

[0124] u 1max =5*2 -m Formula (C);

[0125] Among them, u 1max This represents the first maximum error value, and m represents the length of the mantissa of the first floating-point number.

[0126] Based on the first maximum error value, the initial error value generated when scaling the first floating-point number is determined.

[0127] Considering that determining the initial error value generated when scaling the first floating-point number using the above formula (16) requires not only complex calculations but also floating-point multiplication, a certain error may occur. To avoid this, in this embodiment, the maximum error value generated by scaling the first floating-point number according to the first scaling ratio or scaling the first floating-point number according to the first scaling ratio plus 1 is directly calculated using the above formula (C). After determining the first maximum error value, it can be directly used as the initial error value. Taking 80-bit extended precision as an example, the derivation process of formula (C) is as follows:

[0128] Since the preset numerical range can be set to [1, 10) or [0.1, 1), the second floating-point number obtained after scaling will definitely be less than 10. Therefore:

[0129]

[0130] because Since it is greater than 1, simplifying formula (17) yields:

[0131]

[0132] Multiply both sides of formula (18) by 0.5*2. -63 We can obtain:

[0133]

[0134] Referring to formula (16) above, since Therefore, we can conclude that:

[0135] u1≤5*2 -63 Formula (20);

[0136] Using the above formula (20), it can be determined that scaling the first floating-point number according to the first scaling ratio or scaling the first floating-point number according to the first scaling ratio plus 1 will result in a maximum error of 5*2. -63 Thus, we obtain the above formula (C). It should be noted that since the length of the mantissa corresponding to the 80-bit extended precision is 63, that is, there are 63 mantissa bits, m is 63 in the above formula (20).

[0137] In some embodiments of this specification, the initial error value generated when scaling the first floating-point number is determined based on the mantissa and exponent of the first floating-point number, including:

[0138] The second maximum error value is calculated by scaling the first floating-point number according to the first scaling ratio, and scaling the scaling result by a factor of ten, using the following formula:

[0139] u 2max =10*2 -m Formula (D)

[0140] Among them, u 2max This represents the second maximum error value, and m represents the length of the mantissa of the first floating-point number.

[0141] Based on the second maximum error value, the initial error value generated when scaling the first floating-point number is determined.

[0142] Referring to the derivation process of formula (C) above, since in this embodiment, at most two scaling operations are required to scale the first floating-point number to within the preset value range, the resulting second maximum error value is twice the first maximum error value. That is, formula (D) can be directly obtained by multiplying formula (C) by 2.

[0143] S2068, Perform multiple cyclic rounding calculations on the second floating-point number; wherein, each cyclic rounding calculation includes: determining whether the decimal part of the intermediate value of the current cyclic rounding calculation meets the rounding condition based on the upper and lower intervals corresponding to the current cyclic rounding calculation; if the decimal part of the intermediate value of the current cyclic rounding calculation meets the rounding condition, rounding is performed on the decimal part of the intermediate value of the current cyclic rounding calculation, and the cyclic rounding calculation stops; if the decimal part of the intermediate value of the current cyclic rounding calculation does not meet the rounding condition, the next cyclic rounding calculation continues; the upper and lower intervals corresponding to the current cyclic rounding calculation are ten times the upper and lower intervals corresponding to the previous cyclic rounding calculation, and the intermediate value of the current cyclic rounding calculation is ten times the decimal part of the intermediate value of the previous cyclic rounding calculation; the upper and lower intervals corresponding to the first cyclic rounding calculation are the upper and lower intervals of the second floating-point number, and the intermediate value of the first cyclic rounding calculation is the second floating-point number.

[0144] In some embodiments of this specification, the integer value of the converted fixed-point number is determined by multiple iterative rounding calculations. During each rounding calculation, the decimal part of the intermediate value of the current rounding calculation is checked against the upper and lower intervals to determine if it meets the rounding condition. If it does, the decimal part of the intermediate value is rounded directly, and the rounding calculation stops. This rounding process can be selected as needed, such as rounding to the nearest whole number or truncation. If the rounding condition is not met, the next rounding calculation continues. The upper and lower intervals for each rounding calculation are ten times the upper and lower intervals for the previous rounding calculation, and the intermediate value for each rounding calculation is ten times the decimal part of the intermediate value for the previous rounding calculation. The upper and lower intervals for the first rounding calculation are the upper and lower intervals of the second floating-point number, and the intermediate value for the first rounding calculation is the second floating-point number. For example, suppose a second floating-point number is 1.12345..., the middle value of the first rounding calculation is 1.12345..., the decimal part is 0.12345..., the middle value of the second rounding calculation is 1.2345..., the decimal part is 0.2345..., and so on, until the rounding calculation stops.

[0145] In some embodiments of this specification, the decimal part of the intermediate value in the current rounding calculation can be determined using the following formula:

[0146] or,

[0147] Where v' represents the decimal part of the middle value in this rounding calculation, m - This indicates the next interval corresponding to the current rounding calculation in the loop, m. + This indicates the upper interval corresponding to the current rounding calculation. or, If the result is positive, it indicates that the rounding requirement is met. It should be noted that when determining whether the decimal part of the intermediate value of the current rounding calculation meets the rounding condition by using the upper and lower intervals corresponding to the current rounding calculation, it is also possible to directly determine whether the error between the rounded fixed-point number and the original floating-point number is within an acceptable range, that is, whether it is between the upper and lower intervals, to determine whether the rounding condition is met. There is no limitation on this.

[0148] In some embodiments of this specification, the rounding calculation in each iteration further includes:

[0149] Determine whether the distance between the decimal part of the middle value of the current rounding calculation and both the upper and lower intervals of the current rounding calculation are greater than the error value of the current rounding calculation;

[0150] In response to the determination that the distance between the decimal part of the intermediate value of the current rounding calculation and the upper or lower interval of the current rounding calculation is less than or equal to the error value of the current rounding calculation, the data type conversion task is terminated.

[0151] Considering that the error value of each rounding calculation increases as the number of rounding calculations increases, it is possible to determine whether the error value of the current rounding calculation meets the conversion accuracy requirements in each rounding calculation. If it does not meet the requirements, the current data type conversion task can be stopped directly, thereby avoiding unnecessary rounding calculations.

[0152] In some embodiments of this specification, when the distance between the decimal part of the intermediate value of the current rounding calculation and the upper or lower interval of the current rounding calculation is less than or equal to the error value of the current rounding calculation, it indicates that the error value of the current rounding calculation no longer meets the conversion precision requirements, i.e., the data type conversion cannot be completed on the basis of floating-point numbers. In this case, in addition to directly stopping the current data type conversion task, other methods can be used to complete the above data type conversion task, i.e., to achieve accurate conversion from floating-point numbers to fixed-point numbers through other methods. For example, in some embodiments, the floating-point number to be converted can first be converted into a fraction represented by two infinitely precise integers, i.e., using an infinitely precise integer as the numerator and an infinitely precise integer as the denominator, and then the floating-point number to be converted can be represented by this fraction. This achieves the conversion from floating-point numbers to infinitely precise integers, and then the fraction can be converted into a fixed-point number.

[0153] In some embodiments of this specification, the rounding calculation in each iteration further includes:

[0154] Determine whether the number of iterations corresponding to the current rounding calculation is greater than the preset number of iterations;

[0155] In response to determining that the number of iterations corresponding to the current rounding calculation is greater than the preset number of iterations, it is determined whether the distance between the decimal part of the middle value of the current rounding calculation and the upper and lower intervals of the current rounding calculation is greater than the error value of the current rounding calculation.

[0156] In response to determining that the distance between the decimal part of the intermediate value of the current rounding calculation and the upper or lower interval of the current rounding calculation is less than or equal to the error value of the current rounding calculation, the rounding calculation is stopped.

[0157] Considering that determining whether the distance between the decimal part of the intermediate value of the current rounding calculation and both the upper and lower intervals of the current rounding calculation is greater than the error value of the current rounding calculation requires a certain amount of computation time during the actual operation of the data type conversion task, and that the error value of the rounding calculation increases with the number of rounding calculations, it is unlikely that the error value will fail to meet the conversion accuracy requirements in the early stages of the rounding calculation. If the error value judgment starts from the first rounding calculation, it will affect the overall running speed of the data type conversion task. Therefore, in this embodiment, it is first determined whether the number of rounding calculations corresponding to the current rounding calculation is greater than the preset number of rounding calculations. This process is relatively fast and has little impact on the task's running speed. Then, only when it is determined that the number of rounding calculations corresponding to the current rounding calculation is greater than the preset number of rounding calculations will it be determined whether the distance between the decimal part of the intermediate value of the current rounding calculation and both the upper and lower intervals of the current rounding calculation is greater than the error value of the current rounding calculation. This can further improve the overall running speed of the data type conversion task. It should be noted that the preset number of loops can be set as needed, and there is no limitation on it.

[0158] S20610, in response to determining that the distance between the decimal part of the intermediate value of the last rounding calculation and both the upper and lower intervals of the last rounding calculation are greater than the error value of the last rounding calculation, based on the integer part of the intermediate value of each rounding calculation and the rounding result of the last rounding calculation, the second floating-point number is converted into a fixed-point number as the execution result of the target data type conversion task; wherein, the error value of each rounding calculation is determined by multiplying the initial error value by an integer multiple of ten for the current number of rounding calculations.

[0159] Considering that determining whether the distance between the decimal part of the intermediate value in each rounding calculation and the upper and lower intervals of the last rounding calculation is greater than the error value of each rounding calculation requires a certain amount of computation time, and that the determination result is yes in most cases, i.e., the conversion accuracy requirement is met, in order to further improve the computation speed of the data type conversion task, in this embodiment, after the rounding calculation is completed, it is determined that the distance between the decimal part of the intermediate value in the last rounding calculation and the upper and lower intervals of the last rounding calculation is greater than the error value of the last rounding calculation. This improves the execution efficiency of the data type conversion task while ensuring conversion accuracy. When it is determined that the distance between the decimal part of the intermediate value in the last rounding calculation and the upper and lower intervals of the last rounding calculation is greater than the error value of the last rounding calculation, the conversion accuracy requirement is met. At this time, the second floating-point number can be converted into a fixed-point number based on the integer part of the intermediate value in each rounding calculation and the rounding result of the last rounding calculation. That is, the integer value of the converted fixed-point number is determined by the integer part of the intermediate value calculated in each loop and the rounding result of the last loop. After obtaining the integer value, the converted fixed-point number can be obtained based on the integer value and the corresponding scaling factor.

[0160] It should be noted that the error value for each rounding calculation is determined by multiplying the initial error value by an integer multiple of ten based on the current rounding count. That is, the error value for each rounding calculation is: initial error value * 10. t -1 Where t represents the number of iterations in the current rounding calculation. For example, the error value of the first rounding calculation is: initial error value * 10. 0 = Initial error value, the error value calculated by rounding in the second loop is: initial error value * 10 1 = Initial error value * 10.

[0161] In some embodiments of this specification, the following formula can be used to determine whether the distance between the decimal part of the intermediate value in each rounding calculation and the upper and lower intervals of each rounding calculation is greater than the error value of each rounding calculation:

[0162] and,

[0163] Where v' represents the decimal part of the middle value in this rounding calculation, m - This indicates the next interval corresponding to the current rounding calculation in the loop, m. +This represents the upper interval corresponding to the current rounding calculation, and u represents the error value corresponding to the current rounding calculation. It should be noted that in the above formula, the distance between two numbers is determined by the absolute value of the difference between them. In other embodiments, the distance can also be determined in other ways. If the distance between the decimal part of the middle value of the current rounding calculation and both the upper and lower intervals of the current rounding calculation is greater than the error value of the current rounding calculation, it indicates that the error value of the current rounding calculation will not affect the floating-point conversion accuracy. Conversely, if the distance between the decimal part of the middle value of the current rounding calculation and the upper interval of the current rounding calculation is less than or equal to the error value of the current rounding calculation, or the distance between the decimal part of the middle value of the current rounding calculation and the lower interval of the current rounding calculation is less than or equal to the error value of the current rounding calculation, it indicates that the error value of the current rounding calculation will affect the floating-point conversion accuracy.

[0164] In some embodiments of this specification, the second floating-point number is converted into a fixed-point number based on the integer part of the intermediate value of each rounding calculation and the rounding result of the last rounding calculation, including:

[0165] Based on the integer part of the intermediate value calculated in each loop and the rounding result of the last loop calculation, a target value for converting the second floating-point number into a fixed-point number is determined; wherein, the target value is stored as an int64 type.

[0166] Obtain a first scaling factor to scale the first floating-point number to a preset value range, and determine a second scaling factor corresponding to the target value based on the first scaling factor and the total number of rounding calculations in multiple loops; wherein the exponent of the second scaling factor is stored as an int16 type.

[0167] Based on the target value and the second scaling ratio, the second floating-point number is converted into a fixed-point number.

[0168] After the rounding calculation is completed, when it is determined that the current error value meets the conversion accuracy requirement, the target value for converting the second floating-point number to a fixed-point number can be determined based on the integer part of the intermediate value of each rounding calculation and the rounding result of the last rounding calculation. This target value is the integer value of the converted fixed-point number. Since the int64 type can accurately represent a 17-bit decimal number, in this embodiment, the integer value of the converted fixed-point number is stored using the int64 type. Simultaneously, based on the first scaling ratio for scaling the first floating-point number to a preset value range and the total number of rounding calculations, a second scaling ratio corresponding to the target value is determined. The first scaling ratio can be represented as 10. nThe corresponding second scaling factor can be: 10 n-t-1 Where t represents the total number of rounding calculations, i.e., the number of iterations corresponding to the last rounding calculation, and nt-1 represents the exponent of the second scaling factor, which can be stored using an int16 type. After obtaining the target value and the second scaling factor, multiplying them converts the second floating-point number to a fixed-point number.

[0169] In some embodiments of this specification, converting the second floating-point number into a fixed-point number based on the target value and the second scaling ratio includes:

[0170] Retrieve the target index from the database that matches a fixed number of points;

[0171] Based on the target index, the target value and the second scaling ratio are adjusted to obtain the fixed-point number corresponding to the second floating-point number.

[0172] Considering that different databases have their own required data formats when storing fixed-point numbers, for example, a database might use the int1024 storage format, which corresponds to a fixed exponent to ensure that the fixed-point number in the database has a unique representation, to meet the storage requirements of different databases, when converting the second floating-point number to a fixed-point number based on the intermediate value and the second scaling ratio, the target exponent matching the database for fixed-point numbers can be obtained first. Then, the target value and the second scaling ratio can be adjusted according to the target exponent to obtain the fixed-point number corresponding to the second floating-point number. For example, the target value and the second scaling ratio for converting a certain second floating-point number to a fixed-point number might be 123 and 10, respectively. -9 The target index corresponding to the current database is 10. -7 Therefore, 123 needs to be adjusted to 1.23, 10 -9 Adjusted to 10 -7 .

[0173] This specification provides one or more embodiments of a data processing method applied to a database system. After triggering a data type conversion task to convert a floating-point number to a fixed-point number, the method obtains the floating-point number to be converted and converts it into a first floating-point number represented by extended precision. This extended precision provides a higher precision floating-point format than the basic floating-point format, accurately representing all mantissas of the basic floating-point number. This provides round-trip assurance for subsequent conversion calculations directly based on the extended precision floating-point number. Furthermore, performing round-robin operations directly on the extended precision floating-point format results in faster operation, requires less fixed space, and eliminates dynamic memory allocation. Multiple round-robin calculations are used to precisely round a second floating-point number scaled to a preset numerical range, ensuring the converted fixed-point number maintains the minimum number of significant digits. Simultaneously, based on the mantissa and exponent of the first floating-point number, the initial error value generated during scaling of the first floating-point number is determined. Then, by calculating the number of rounding operations in the loop, the error value of the final rounding operation is determined. This error value ensures that the error generated during data type conversion operations will not affect the normal round-trip between the fixed-point number and the original floating-point number. Specifically, by checking whether the distance between the decimal part of the middle value of the final rounding operation and both the upper and lower intervals of the final rounding operation are greater than the error value of the final rounding operation, it is determined whether the data type conversion task can guarantee that the obtained fixed-point number and the original floating-point number can make round-trips. This ensures the accuracy of the data type conversion task and further provides a foundation for accurate data processing in the database system.

[0174] refer to Figure 3 This specification provides another data processing method for a database system, which includes the following steps:

[0175] S302, Begin.

[0176] This initiates the data type conversion task.

[0177] S304, Extended precision initialization, upper and lower interval initialization.

[0178] Extended precision initialization involves converting the floating-point number to be converted into a first floating-point number represented by extended precision. Upper and lower interval initialization involves calculating the upper and lower intervals of the first floating-point number. It should be noted that in this embodiment, the upper and lower intervals of the first floating-point number are calculated directly. These upper and lower intervals are scaled along with the first floating-point number, meaning the upper and lower intervals of the second floating-point number can be obtained from the scaled result.

[0179] S306, scale the value to be converted to the range [1.0, 10.0).

[0180] It should be noted that, since the extended precision initialization has been performed, the value to be converted here is the first floating-point number represented by the extended precision. In this embodiment, the preset value range is [1.0, 10.0].

[0181] S308, determine whether the scaled value is within the range of the fixed-point type representation. If yes, proceed to step S310; otherwise, proceed to step S312.

[0182] Considering that different databases have corresponding fixed-point type representation ranges, if the value exceeds the fixed-point type representation range, it means that the fixed-point number converted from the floating-point number cannot be represented in the current database, that is, the data type conversion task cannot be performed, and therefore, error handling will be performed.

[0183] S310, calculate the initial error value.

[0184] Once it is determined that the scaled value (i.e., the second floating-point number) is within the range of the fixed-point type representation, the initial error value generated when scaling the first floating-point number is further determined by the mantissa and exponent of the first floating-point number.

[0185] S312, Error Handling.

[0186] It should be noted that the specific error handling can be configured as needed. For example, the error can be handled directly in the system, or the data type conversion task can be switched to another method. There are no restrictions on this.

[0187] S314, Convert the value to be converted and save the result.

[0188] In this embodiment, converting the value to be converted involves performing multiple cyclic rounding calculations on the second floating-point number, and converting the second floating-point number into a fixed-point number based on the results of the multiple cyclic rounding calculations. The specific process can be referred to the corresponding content in the above embodiment, and will not be repeated here.

[0189] S316. Determine whether the result is within the error range. If yes, proceed to step S318; otherwise, proceed to step S312.

[0190] To determine if the result is within the error range, we need to determine whether the distance between the decimal part of the middle value of the last rounding calculation and the upper and lower intervals of the last rounding calculation is greater than the error value of the last rounding calculation. If yes, it means that the conversion accuracy requirement is met and the conversion result can be used. If no, it means that the conversion result cannot meet the accuracy requirement, that is, it cannot be guaranteed that the obtained fixed-point number can be exchanged with the original floating-point number. Therefore, error handling needs to be performed.

[0191] S318, End.

[0192] Once the conversion result is determined to be within the error range, the execution of this data type conversion task can be terminated directly, which also means that the data type conversion task has been completed.

[0193] The above Figure 3 The data processing method of the database system in this embodiment is similar to that of the database system in the foregoing embodiment, and the specific implementation methods and effects of the same implementation steps are the same, so they will not be repeated here.

[0194] Figure 4 This is a schematic structural diagram of a device provided in an exemplary embodiment. For example... Figure 4 As shown, device 400 mainly consists of a communication interface 402, a user interface 404, a processor 406, and a data storage 408. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 410. The communication interface 402 enables device 400 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 402 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 402 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 402 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 402 may also include multiple physical communication interfaces, such as Wi-Fi, Bluetooth, and wide-area wireless interfaces.

[0195] User interface 404 includes receiving user input and providing output to the user. Therefore, user interface 404 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 404 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 404 may include software, circuitry, or other forms of logic capable of transmitting and receiving data from external user input / output devices. Additionally or alternatively, device 400 may support remote access from other devices via communication interface 402 or another physical interface (not shown). User interface 404 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 404 may also be configured as a display device for rendering or displaying text fragments.

[0196] Processor 406 may contain one or more general-purpose processors and / or special-purpose processors.

[0197] Data storage 408 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 406. Data storage 408 may include removable and non-removable components.

[0198] Processor 406 is capable of executing program instructions 418 (e.g., compiled or uncompiled program logic and / or machine code) stored in data storage 408 to perform the various functions described herein. Data storage 408 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 400, enable device 400 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Processor 406 executing program instructions 418 may result in processor 406 using data 412.

[0199] For example, program instructions 418 may include an operating system 422 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 400 and one or more applications 420 (e.g., a browser, social application, or game application). Similarly, data 412 may include operating system data 416 and application data 414. Operating system data 416 is primarily accessible to the operating system 422, while application data 414 is primarily accessible to one or more applications 420. Application data 414 may reside in a file system visible or hidden from the user of device 400.

[0200] Application 420 can communicate with operating system 422 through one or more application programming interfaces (APIs). These APIs help application 420 read and / or write application data 414, transmit or receive information via communication interface 402, receive or display information on user interface 404, etc.

[0201] In some terminology, application 420 may be simply referred to as "app". Furthermore, application 420 can be downloaded to device 400 through one or more online app stores or app markets. However, applications can also be installed on device 400 in other ways, such as through a web browser or a physical interface on device 400 (e.g., a USB port).

[0202] Please refer to Figure 5 Data processing devices used in database systems can be applied to, for example... Figure 4 The device shown implements the technical solution of this specification. The database system includes storage services, computing services, and metadata management services; the device is applied to the computing services; the data processing device applied to the database system may include:

[0203] The query module 502 obtains the target computing task and queries the metadata management service based on the target computing task to determine the data storage location information of the data to be processed required by the target computing task.

[0204] The acquisition module 504 acquires the data to be processed from the storage service according to the data storage location information;

[0205] Execution module 506, when the data to be processed includes floating-point numbers to be converted, triggers a data type conversion task to convert the floating-point numbers to fixed-point numbers, and executes the data type conversion task in the following manner so that the corresponding converted fixed-point numbers can be used to continue completing the target calculation task:

[0206] Convert the floating-point number to be converted into a first floating-point number represented by extended precision;

[0207] The first floating-point number is scaled to a preset value range to obtain a second floating-point number; the preset value range is used to round the first mantissa of the first floating-point data.

[0208] Based on the mantissa and exponent of the first floating-point number, the initial error value generated when scaling the first floating-point number is determined, and the upper and lower intervals of the second floating-point number are determined based on the mantissa and exponent of the second floating-point number.

[0209] The second floating-point number is subjected to multiple cyclic rounding calculations. Each cyclic rounding calculation includes: determining whether the decimal part of the intermediate value of the current cyclic rounding calculation meets the rounding condition based on the upper and lower intervals corresponding to the current cyclic rounding calculation; if the decimal part of the intermediate value of the current cyclic rounding calculation meets the rounding condition, the decimal part of the intermediate value of the current cyclic rounding calculation is rounded, and the cyclic rounding calculation stops; if the decimal part of the intermediate value of the current cyclic rounding calculation does not meet the rounding condition, the next cyclic rounding calculation is performed. The upper and lower intervals corresponding to the current cyclic rounding calculation are ten times the upper and lower intervals corresponding to the previous cyclic rounding calculation, and the intermediate value of the current cyclic rounding calculation is ten times the decimal part of the intermediate value of the previous cyclic rounding calculation. The upper and lower intervals corresponding to the first cyclic rounding calculation are the upper and lower intervals of the second floating-point number, and the intermediate value of the first cyclic rounding calculation is the second floating-point number.

[0210] In response to the determination that the distance between the decimal part of the intermediate value of the last rounding calculation and both the upper and lower intervals of the last rounding calculation are greater than the error value of the last rounding calculation, the second floating-point number is converted into a fixed-point number based on the integer part of the intermediate value of each rounding calculation and the rounding result of the last rounding calculation, as the execution result of the data type conversion task; wherein, the error value of each rounding calculation is determined by multiplying the initial error value by an integer multiple of ten for the current number of rounding calculations.

[0211] In some embodiments of this specification, the execution module includes a scaling module, the scaling module comprising:

[0212] The calculation unit scales the first floating-point number to a first scaling factor within a preset numerical range using a logarithmic function.

[0213] The scaling unit scales the first floating-point number to a preset numerical range based on the first scaling ratio.

[0214] In some embodiments of this specification, the logarithmic function includes:

[0215] or,

[0216] Wherein, the first scaling ratio is 10 n e v This represents the exponent of the first floating-point number; Represents log 10 The estimated value of 2, 10 -x Represents log 10 2 and The preset error value.

[0217] In some embodiments of this specification, the scaling unit is specifically used for:

[0218] The first floating-point number is scaled according to the first scaling ratio, and it is determined whether the scaling result is within the preset value range;

[0219] In response to determining that the scaling result is within the preset numerical range, the second floating-point number is obtained based on the scaling result;

[0220] In response to determining that the scaling result is not within the preset numerical range, the exponent of the first scaling ratio is increased by 1, and the first floating-point number is scaled according to the first scaling ratio after the exponent is increased by 1 to obtain the second floating-point number.

[0221] In some embodiments of this specification, the determining module is specifically used for:

[0222] The first maximum error value generated by scaling the first floating-point number according to the first scaling ratio or by scaling the first floating-point number according to the first scaling ratio plus 1 is calculated using the following formula:

[0223] u 1max =5*2 -m ;

[0224] Among them, u 1max The maximum error value is represented by m, which represents the length of the mantissa of the first floating-point number.

[0225] Based on the first maximum error value, the initial error value generated when scaling the first floating-point number is determined.

[0226] In some embodiments of this specification, the scaling unit is specifically used for:

[0227] The first floating-point number is scaled according to the first scaling ratio, and it is determined whether the scaling result is within the preset value range;

[0228] In response to determining that the scaling result is within the preset numerical range, the second floating-point number is obtained based on the scaling result;

[0229] In response to determining that the scaling result is not within the preset numerical range, the scaling result is scaled by a factor of ten to obtain the second floating-point number.

[0230] In some embodiments of this specification, the execution module includes a determining module for:

[0231] The second maximum error value is calculated by scaling the first floating-point number according to the first scaling ratio, and scaling the scaling result by a factor of ten, using the following formula:

[0232] u 2max =10*2 -m ;

[0233] Among them, u 2max This represents the second maximum error value, and m represents the length of the mantissa of the first floating-point number.

[0234] Based on the second maximum error value, the initial error value generated when scaling the first floating-point number is determined.

[0235] In some embodiments of this specification, the rounding calculation in each iteration further includes:

[0236] Determine whether the distance between the decimal part of the middle value of the current rounding calculation and both the upper and lower intervals of the current rounding calculation are greater than the error value of the current rounding calculation;

[0237] In response to the determination that the distance between the decimal part of the intermediate value of the current rounding calculation and the upper or lower interval of the current rounding calculation is less than or equal to the error value of the current rounding calculation, the data type conversion task is terminated.

[0238] In some embodiments of this specification, the rounding calculation in each iteration further includes:

[0239] Determine whether the number of iterations corresponding to the current rounding calculation is greater than the preset number of iterations;

[0240] In response to determining that the number of iterations corresponding to the current rounding calculation is greater than the preset number of iterations, it is determined whether the distance between the decimal part of the middle value of the current rounding calculation and the upper and lower intervals of the current rounding calculation is greater than the error value of the current rounding calculation.

[0241] In response to determining that the distance between the decimal part of the intermediate value of the current rounding calculation and the upper or lower interval of the current rounding calculation is less than or equal to the error value of the current rounding calculation, the rounding calculation is stopped.

[0242] In some embodiments of this specification, the execution module includes a conversion module, the conversion module comprising:

[0243] The target data determination unit determines the target value to be converted into a fixed-point number based on the integer part of the intermediate value of each loop rounding calculation and the rounding result of the last loop rounding calculation; wherein the target value is stored as an int64 type.

[0244] The scaling ratio determination unit obtains a first scaling ratio for scaling the first floating-point number to a preset value range, and determines a second scaling ratio corresponding to the target value based on the first scaling ratio and the total number of rounding calculations in multiple loops; wherein the exponent of the second scaling ratio is stored as an int16 type.

[0245] The conversion unit converts the second floating-point number into a fixed-point number based on the target value and the second scaling ratio.

[0246] In some embodiments of this specification, the conversion unit is specifically used for:

[0247] Retrieve the target index from the database that matches a fixed number of points;

[0248] Based on the target index, the target value and the second scaling ratio are adjusted to obtain the fixed-point number corresponding to the second floating-point number.

[0249] In some embodiments of this specification, the execution model includes an acquisition module for:

[0250] In response to the presence of a floating-point number to be calculated in the target computing task, the floating-point number to be converted in the data type conversion task is obtained from the target computing task;

[0251] Wherein, the floating-point number to be converted is any floating-point number to be calculated that appears in the target calculation task; the target calculation task completes the calculation by converting the floating-point number to be calculated into a fixed-point number during the execution process.

[0252] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more of these specifications, the functions of each module or unit can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0253] Based on the same concept as the above methods, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor executes the executable instructions to implement the steps of the execution method for a data type conversion task as described in any of the above embodiments.

[0254] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the execution method for the data type conversion task as described in any of the above embodiments.

[0255] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the execution method for the data type conversion task as described in any of the above embodiments.

[0256] What those skilled in the art will understand is:

[0257] In this specification, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded.

[0258] In this specification, “a,” “an,” and “the” do not specifically refer to the singular, but may also include the plural.

[0259] In this specification, ordinal numbers such as "first," "second," etc., do not necessarily indicate order; they are often used to distinguish between objects. For example, "first server" and "second server" usually refer to two servers. To differentiate between these two servers, they are described as "first server" and "second server." Of course, sometimes these two servers may be the same server.

[0260] In this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can also mean indirect receiving and sending. For example, A receiving data sent by B can be understood as A directly receiving the data sent by B, or it can be understood as A indirectly receiving the data sent by B through other entities such as C. Similarly, B sending data to A can be understood as B sending the data directly to A, or it can be understood as B indirectly sending the data to A through other entities such as C. Here, C can be one entity, or it can be two or more entities.

[0261] In this specification, unless explicitly stated otherwise, the relationships between structures can be direct or indirect. For example, when describing "A is connected to B," unless it is explicitly stated that A and B are directly connected, it should be understood that A can be directly connected to B or indirectly connected to B. Similarly, when describing "A is on top of B," unless it is explicitly stated that A is directly above B (AB is adjacent and A is above B), it should be understood that A can be directly above B or indirectly above B (AB is separated by other elements, and A is above B). And so on.

[0262] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0263] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is only one of many possible execution orders and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims.

Claims

1. A data processing method applied to a database system, the database system including storage services, computing services, and metadata management services, the method applied to the computing services, comprising: Obtain the target computing task, and query the metadata management service based on the target computing task to determine the data storage location information of the data to be processed required by the target computing task; The data to be processed is obtained from the storage service according to the data storage location information; If the data to be processed includes floating-point numbers to be converted, a data type conversion task is triggered to convert the floating-point numbers to fixed-point numbers. This data type conversion task is then performed in the following manner so that the corresponding converted fixed-point numbers can be used to continue completing the target computation task: Convert the floating-point number to be converted into a first floating-point number represented by extended precision; The first floating-point number is scaled to a preset value range to obtain a second floating-point number; the preset value range is used to round the first mantissa of the first floating-point data. Based on the mantissa and exponent of the first floating-point number, the initial error value generated when scaling the first floating-point number is determined, and the upper and lower intervals of the second floating-point number are determined based on the mantissa and exponent of the second floating-point number. The second floating-point number is subjected to multiple cyclic rounding calculations. Each cyclic rounding calculation includes: determining whether the decimal part of the intermediate value of the current cyclic rounding calculation meets the rounding condition based on the upper and lower intervals corresponding to the current cyclic rounding calculation; if the decimal part of the intermediate value of the current cyclic rounding calculation meets the rounding condition, the decimal part of the intermediate value of the current cyclic rounding calculation is rounded, and the cyclic rounding calculation stops; if the decimal part of the intermediate value of the current cyclic rounding calculation does not meet the rounding condition, the next cyclic rounding calculation is performed. The upper and lower intervals corresponding to the current cyclic rounding calculation are ten times the upper and lower intervals corresponding to the previous cyclic rounding calculation, and the intermediate value of the current cyclic rounding calculation is ten times the decimal part of the intermediate value of the previous cyclic rounding calculation. The upper and lower intervals corresponding to the first cyclic rounding calculation are the upper and lower intervals of the second floating-point number, and the intermediate value of the first cyclic rounding calculation is the second floating-point number. In response to the determination that the distance between the decimal part of the intermediate value of the last rounding calculation and both the upper and lower intervals of the last rounding calculation are greater than the error value of the last rounding calculation, the second floating-point number is converted into a fixed-point number based on the integer part of the intermediate value of each rounding calculation and the rounding result of the last rounding calculation, as the execution result of the data type conversion task; wherein, the error value of each rounding calculation is determined by multiplying the initial error value by an integer multiple of ten for the current number of rounding calculations.

2. The method according to claim 1, wherein, Scaling the first floating-point number to a preset numerical range includes: The first floating-point number is scaled to a first scaling ratio within a preset numerical range using a logarithmic function. The first floating-point number is scaled to a preset numerical range based on the first scaling ratio.

3. The method according to claim 2, wherein, The logarithmic function includes: or, Wherein, the first scaling ratio is 10 n e v This represents the exponent of the first floating-point number; Represents log 10 The estimated value of 2, 10 -x Represents log 10 2 and The preset error value.

4. The method according to claim 3, wherein, Scaling the first floating-point number to a preset numerical range based on the first scaling ratio includes: The first floating-point number is scaled according to the first scaling ratio, and it is determined whether the scaling result is within the preset value range; In response to determining that the scaling result is within the preset numerical range, the second floating-point number is obtained based on the scaling result; In response to determining that the scaling result is not within the preset numerical range, the exponent of the first scaling ratio is increased by 1, and the first floating-point number is scaled according to the first scaling ratio after the exponent is increased by 1 to obtain the second floating-point number.

5. The method according to claim 4, wherein, Based on the mantissa and exponent of the first floating-point number, determine the initial error value generated when scaling the first floating-point number, including: The first maximum error value generated by scaling the first floating-point number according to the first scaling ratio or by scaling the first floating-point number according to the first scaling ratio plus 1 is calculated using the following formula: u 1max ＝5*2 -m ； Among them, u 1max This represents the first maximum error value, and m represents the length of the mantissa of the first floating-point number. Based on the first maximum error value, the initial error value generated when scaling the first floating-point number is determined.

6. The method according to claim 3, wherein, Scaling the first floating-point number to a preset numerical range based on the first scaling ratio includes: The first floating-point number is scaled according to the first scaling ratio, and it is determined whether the scaling result is within the preset value range; In response to determining that the scaling result is within the preset numerical range, the second floating-point number is obtained based on the scaling result; In response to determining that the scaling result is not within the preset numerical range, the scaling result is scaled by a factor of ten to obtain the second floating-point number.

7. The method according to claim 6, wherein, Based on the mantissa and exponent of the first floating-point number, determine the initial error value generated when scaling the first floating-point number, including: The second maximum error value is calculated by scaling the first floating-point number according to the first scaling ratio, and scaling the scaling result by a factor of ten, using the following formula: u 2max ＝10*2 -m ； Among them, u 2max This represents the second maximum error value, and m represents the length of the mantissa of the first floating-point number. Based on the second maximum error value, the initial error value generated when scaling the first floating-point number is determined.

8. The method according to claim 1, wherein, Each rounding calculation in the loop also includes: Determine whether the distance between the decimal part of the middle value of the current rounding calculation and both the upper and lower intervals of the current rounding calculation are greater than the error value of the current rounding calculation; In response to the determination that the distance between the decimal part of the intermediate value of the current rounding calculation and the upper or lower interval of the current rounding calculation is less than or equal to the error value of the current rounding calculation, the data type conversion task is terminated.

9. The method according to claim 1, wherein, Each rounding calculation in the loop also includes: Determine whether the number of iterations corresponding to the current rounding calculation is greater than the preset number of iterations; In response to determining that the number of iterations corresponding to the current rounding calculation is greater than the preset number of iterations, it is determined whether the distance between the decimal part of the middle value of the current rounding calculation and the upper and lower intervals of the current rounding calculation is greater than the error value of the current rounding calculation. In response to determining that the distance between the decimal part of the intermediate value of the current rounding calculation and the upper or lower interval of the current rounding calculation is less than or equal to the error value of the current rounding calculation, the rounding calculation is stopped.

10. The method according to claim 1, wherein, Based on the integer part of the intermediate value calculated in each loop and the rounding result of the last loop, the second floating-point number is converted into a fixed-point number, including: Based on the integer part of the intermediate value calculated in each loop and the rounding result of the last loop calculation, a target value for converting the second floating-point number into a fixed-point number is determined; wherein, the target value is stored as an int64 type. Obtain a first scaling factor to scale the first floating-point number to a preset value range, and determine a second scaling factor corresponding to the target value based on the first scaling factor and the total number of rounding calculations in multiple loops; wherein the exponent of the second scaling factor is stored as an int16 type. Based on the target value and the second scaling ratio, the second floating-point number is converted into a fixed-point number.

11. The method according to claim 10, wherein, Based on the target value and the second scaling ratio, the second floating-point number is converted into a fixed-point number, including: Retrieve the target index from the database that matches a fixed number of points; Based on the target index, the target value and the second scaling ratio are adjusted to obtain the fixed-point number corresponding to the second floating-point number.

12. A data processing apparatus for use in a database system, the database system including storage services, computing services, and metadata management services, the apparatus being used for the computing services, comprising: The query module obtains the target computing task and queries the metadata management service based on the target computing task to determine the data storage location information of the data to be processed required by the target computing task. The acquisition module acquires the data to be processed from the storage service based on the data storage location information; If the data to be processed includes floating-point numbers to be converted, the execution module triggers a data type conversion task to convert the floating-point numbers to fixed-point numbers, and performs the data type conversion task in the following manner so that the corresponding converted fixed-point numbers can be used to continue completing the target calculation task: Convert the floating-point number to be converted into a first floating-point number represented by extended precision; The first floating-point number is scaled to a preset value range to obtain a second floating-point number; the preset value range is used to round the first mantissa of the first floating-point data. Based on the mantissa and exponent of the first floating-point number, the initial error value generated when scaling the first floating-point number is determined, and the upper and lower intervals of the second floating-point number are determined based on the mantissa and exponent of the second floating-point number. The second floating-point number is subjected to multiple cyclic rounding calculations. Each cyclic rounding calculation includes: determining whether the decimal part of the intermediate value of the current cyclic rounding calculation meets the rounding condition based on the upper and lower intervals corresponding to the current cyclic rounding calculation; if the decimal part of the intermediate value of the current cyclic rounding calculation meets the rounding condition, the decimal part of the intermediate value of the current cyclic rounding calculation is rounded, and the cyclic rounding calculation stops; if the decimal part of the intermediate value of the current cyclic rounding calculation does not meet the rounding condition, the next cyclic rounding calculation is performed. The upper and lower intervals corresponding to the current cyclic rounding calculation are ten times the upper and lower intervals corresponding to the previous cyclic rounding calculation, and the intermediate value of the current cyclic rounding calculation is ten times the decimal part of the intermediate value of the previous cyclic rounding calculation. The upper and lower intervals corresponding to the first cyclic rounding calculation are the upper and lower intervals of the second floating-point number, and the intermediate value of the first cyclic rounding calculation is the second floating-point number. In response to the determination that the distance between the decimal part of the intermediate value of the last rounding calculation and both the upper and lower intervals of the last rounding calculation are greater than the error value of the last rounding calculation, the second floating-point number is converted into a fixed-point number based on the integer part of the intermediate value of each rounding calculation and the rounding result of the last rounding calculation, as the execution result of the data type conversion task; wherein, the error value of each rounding calculation is determined by multiplying the initial error value by an integer multiple of ten for the current number of rounding calculations.

13. An electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor implements the method as described in any one of claims 1-11 by executing the executable instructions.

14. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1-11.

Images