Business data processing method and device, computer equipment and readable storage medium

Abstract

The invention relates to a business data processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps of obtaining first business data, target data belonging to a target dimension in the first business data being range data, and the range data being data comprising at least two values; determining a data coding system based on a value range of target data in the first business data; according to the data coding system, coding target data in the first service data to obtain a system coding result, the system coding result comprising the data coding system and system coding range information corresponding to the target data; and performing data compression on the first service data based on the system coding result to obtain first compressed service data corresponding to the first service data. By adopting the method, the business data processing efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a business data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of computer technology, the amount of business data in various business systems has increased dramatically. Data compression technology has become an important means to improve the storage and transmission efficiency of business data. Data compression technology can effectively reduce the storage space requirements of business data by reducing redundant information in business data, which is conducive to improving the transmission efficiency of business data.

[0003] However, when compressing and decompressing business data, various complex and tedious operations are often required, resulting in low processing efficiency for the business data. Summary of the invention

[0004] Based on this, it is necessary to provide a business data processing method, device, computer equipment, computer-readable storage medium and computer program product that can improve the efficiency of business data processing in response to the above technical problems.

[0005] In a first aspect, the present application provides a business data processing method, comprising:

[0006] Acquire first business data, where target data belonging to a target dimension in the first business data is range data, and the range data is data including at least two values;

[0007] Determine the data encoding system based on the value range of the target data in the first business data;

[0008] Encode the target data in the first business data according to the data coding system to obtain a system coding result, where the system coding result includes the data coding system and system coding range information corresponding to the target data;

[0009] The first service data is compressed based on the binary encoding result to obtain first compressed service data corresponding to the first service data.

[0010] In a second aspect, the present application also provides a business data processing device, including:

[0011] A first business data acquisition module is used to acquire first business data, wherein target data belonging to a target dimension in the first business data is range data, and the range data is data including at least two values;

[0012] A coding system determination module, used to determine the data coding system based on the value range of the target data in the first business data;

[0013] A coding processing module, used for encoding the target data in the first business data according to the data coding system to obtain a system coding result, wherein the system coding result includes the data coding system and the system coding range information corresponding to the target data;

[0014] The compression processing module is used to compress the first business data based on the binary encoding result to obtain first compressed business data corresponding to the first business data.

[0015] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above business data processing method when executing the computer program.

[0016] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above business data processing method when executed by a processor.

[0017] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the above business data processing method are implemented.

[0018] The above-mentioned business data processing method, apparatus, computer equipment, computer-readable storage medium and computer program product, for the first business data whose target data belonging to the target dimension is range data, determine the data coding system based on the value range of the target data in the first business data, encode the target data in the first business data according to the data coding system, obtain the system coding result including the data coding system and the system coding range information corresponding to the target data, and perform data compression on the first business data based on the system coding result to obtain the first compressed business data corresponding to the first business data. For the first business data whose target data belonging to the target dimension is range data, encode according to the data coding system determined by the value range of the target data, obtain the system coding result including the data coding system and the system coding range information corresponding to the target data, so as to realize the compression processing of the first business data, simplify the compression processing of the business data, and thus improve the compression processing efficiency of the business data.

[0019] In a sixth aspect, the present application provides a business data processing method, including:

[0020] Obtaining a service data processing request, and obtaining first compressed service data according to the service data processing request;

[0021] Obtaining a base encoding result from the first compressed service data, where the base encoding result includes data encoding base and base encoding range information corresponding to data belonging to a target dimension in the first compressed service data;

[0022] Decoding the binary encoding range information according to the data encoding binary system to obtain target data belonging to the target dimension in the first compressed service data, where the target data is range data including at least two values;

[0023] Based on the target data belonging to the target dimension in the first compressed business data, the first compressed business data is decompressed to obtain the first business data.

[0024] In a seventh aspect, the present application further provides a business data processing device, including:

[0025] A processing request acquisition module, used to acquire a business data processing request, and obtain first compressed business data according to the business data processing request;

[0026] A coding result acquisition module, used to obtain a base coding result from the first compressed business data, where the base coding result includes data coding base and base coding range information corresponding to data belonging to a target dimension in the first compressed business data;

[0027] A decoding processing module, used for decoding the binary encoding range information according to the data encoding binary system to obtain target data belonging to the target dimension in the first compressed service data, where the target data is range data including at least two values;

[0028] The decompression processing module is used to decompress the first compressed business data based on the target data belonging to the target dimension in the first compressed business data to obtain the first business data.

[0029] In an eighth aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the above business data processing method are implemented.

[0030] In a ninth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above business data processing method when executed by a processor.

[0031] In a tenth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the above business data processing method are implemented.

[0032] The above-mentioned business data processing method, device, computer equipment, computer-readable storage medium and computer program product obtain the first compressed business data according to the business data processing request, obtain the base coding result from the first compressed business data, the base coding result includes the data coding base and the base coding range information corresponding to the data belonging to the target dimension in the first compressed business data, decode the base coding range information according to the data coding base, obtain the target data belonging to the target dimension in the first compressed business data, and decompress the first compressed business data based on the target data belonging to the target dimension in the first compressed business data to obtain the decompressed first business data. For the first compressed business data including the base coding result, decode the base coding range information according to the data coding base in the base coding result, and decompress the data based on the target data belonging to the target dimension obtained by decoding to obtain the first business data, which simplifies the decompression processing for the first business data, thereby improving the decompression processing efficiency of the business data. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 An application environment diagram of a business data processing method in an embodiment;

[0035] Figure 2 A flowchart of a method for processing business data in one embodiment;

[0036] Figure 3 A schematic diagram of a flow chart of binary encoding in one embodiment;

[0037] Figure 4 is a flowchart of a method for processing business data in yet another embodiment;

[0038] Figure 5 It is a flowchart of performing Cartesian calculation based on two types of business data access in one embodiment;

[0039] Figure 6 A schematic diagram of data compression based on binary coding in one embodiment;

[0040] Figure 7 A schematic diagram of a process of reading data and performing Cartesian calculation in one embodiment;

[0041] Figure 8A schematic diagram of a flow chart of reading data and performing Cartesian calculation in yet another embodiment;

[0042] Fig. 9 is a structural block diagram of a business data processing device in an embodiment;

[0043] Fig.10 is a structural block diagram of a business data processing device in yet another embodiment;

[0044] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] The business data processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process, and the data storage system can be integrated on the server 104, or placed on the cloud or other network servers.

[0047] On the one hand, the terminal 102 can send the business data that needs to be compressed to the server 104. The business data that needs to be compressed can include the first business data, and the first business data includes the target data belonging to the target dimension, and the target data is a range data including at least two values. Wherein, in different business system scenarios, the target dimension can correspond to different data dimensions, such as the target dimension can be at least one of various types of dimensions such as salary grade, salary grade, employee rank, customer satisfaction, product quality, sales performance, employee performance, customer value, etc. For the first business data, the server 104 can determine the data encoding system based on the value range of the target data in the first business data, encode the target data in the first business data according to the data encoding system, and obtain the system encoding result including the data encoding system and the system encoding range information corresponding to the target data. The server 104 can perform data compression on the first business data based on the system encoding result, and obtain the first compressed business data corresponding to the first business data, thereby realizing data compression processing for the first business data. In some embodiments, the server 104 can feed back the first compressed business data corresponding to the first business data to the terminal 102 so that the user can view the data compression result through the terminal 102. In some embodiments, the compression processing method for business data may also be implemented by the server 104 or the terminal 102 alone.

[0048] On the other hand, the terminal 102 may send a service data processing request to the server 104 to request the server 104 to trigger data decompression processing. The server 104 may obtain the first compressed service data according to the service data processing request, and obtain the base coding result from the first compressed service data. The base coding result includes the data coding base and the base coding range information corresponding to the data belonging to the target dimension in the first compressed service data. The server 104 may decode the base coding range information according to the data coding base, obtain the target data belonging to the target dimension in the first compressed service data, and decompress the first compressed service data based on the target data belonging to the target dimension in the first compressed service data, thereby obtaining the decompressed first service data, thereby realizing data decompression processing for the first service data. In some embodiments, the server 104 feeds back the decompressed first service data to the terminal 102. In some embodiments, the decompression processing method for the first compressed service data may also be implemented by the server 104 or the terminal 102 alone.

[0049] Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0050] In an exemplary embodiment, Figure 2 As shown, a business data processing method is provided, which is applied to Figure 1 The server in the example is used to illustrate, including the following steps 202 to 208. Among them:

[0051] Step 202, obtaining first business data, wherein target data belonging to a target dimension in the first business data is range data, and the range data is data including at least two values.

[0052] Among them, the business data can be data in the business system, and different business systems can include different business data, such as the salary data of enterprise employees in the financial system, the performance data of enterprise employees in the enterprise assessment system, the test score data of students in the education system, and the account flow data of users in the bank system. The first business number includes data of the target dimension, and the target data belonging to the target dimension in the first business number is range data including at least two values. In different business systems, the target dimension can correspond to different data dimensions, such as the target dimension can include but is not limited to at least one of various dimensions including salary grade, salary grade, employee rank, customer satisfaction, product quality, sales performance, employee performance, customer value, etc. Range data refers to data that can include at least two values, that is, the data value can include a certain range. For example, the target dimension includes salary grade and salary grade of salary, then the data under the salary grade and salary grade dimension in the first business data includes at least two values, that is, the data under the salary grade and salary grade dimension in the first business data is range data. For example, the data under the salary grade and salary grade dimension in the first business data A can be range data of "grade 1, grade 1 to grade 2, grade 4", which can specifically include 9 values ​​such as "grade 1, grade 1", "grade 1, grade 2", "grade 1, grade 3", "grade 1, grade 4", "grade 1, grade 5", "grade 2, grade 1", "grade 2, grade 2", "grade 2, grade 3" and "grade 2, grade 4".

[0053] Optionally, the server may obtain first business data that needs to be compressed, the first business data including target data of a target dimension, and the target data is range data, that is, the target data includes at least two values. In some embodiments, the server may obtain the first business data that needs to be compressed from a database of a business system.

[0054] Step 204: determine the data encoding system based on the value range of the target data in the first business data.

[0055] Among them, the target data includes multiple values, and the value range of the target data can be determined according to each value of the target data. For example, when the target dimension is employee performance, the target data belonging to employee performance in the first business data B may include six data such as "Grade A, Grade 1", "Grade A, Grade 2", "Grade A, Grade 3", "Grade B, Grade 1", "Grade B, Grade 2" and "Grade B, Grade 3", that is, the value range of the target data can be determined to be "Grade A, Grade 1" to "Grade B, Grade 3". The data coding system is the coding method for the target data, and specifically can be the system required for encoding the target data. The data coding system can be determined based on the value range of the target data, such as the corresponding data coding system can be determined based on the number of values ​​in the value range.

[0056] Optionally, for the first business data, the server may determine the value range of the target data based on each value of the target data, and the server may determine the corresponding data encoding system based on the value range. In some embodiments, the server may perform dimension division based on the relationship between each value of the target data in the first business data, and determine the corresponding data encoding system based on the dimension division result and the value range. For example, the value range of the target data in the first business data C is "C1 to C30", that is, the target data can include 30 one-dimensional values, and the server can determine that the data encoding system is thirty-digit base 3; for another example, the value range of the target data in the first business data D is "D1E1-D1E6, D2E1-D2E6, D3E1-D3E6, D4E1-D4E6, D5E1-D5E6", that is, the target data also includes 30 values, and the 30 values ​​can be divided into two-dimensional data of 5 (D1, D2, D3, D4, D5)*6 (E1, E2, E3, E4, E5, E6), and the server can determine that the data encoding system is hexadecimal.

[0057] Step 206, encode the target data in the first business data according to the data coding system to obtain a system coding result, which includes the data coding system and the system coding range information corresponding to the target data.

[0058] Among them, the base encoding result is the encoding result obtained by encoding the target data, and the base encoding range information is the range information obtained based on the base encoding results corresponding to each value of the target data, so that the value range of the target data can be represented by the base encoding range information. By combining the base encoding range information with the corresponding data encoding base, decoding can be performed to restore the corresponding target data.

[0059] Exemplarily, the server may encode the target data in the first business data according to the data coding system to obtain a corresponding system coding result, which may include the data coding system and system coding range information corresponding to the target data. For example, the value range of the target data in the first business data D is "D1E1-D1E6, D2E1-D2E6, D3E1-D3E6, D4E1-D4E6, D5E1-D5E6". After determining that the data coding system is hexadecimal, the target data is encoded according to hexadecimal, so that the various values ​​of the target data can be encoded as "00-05, 10-15, 20-25, 30-35, 40-45" respectively. After further conversion into decimal, "0-29" can be obtained, and the obtained system coding result can be [6,0-29], where 6 indicates that the data coding system is hexadecimal, and 0-29 indicates the system coding range information corresponding to the target data, and each serial number in 0-29 can correspond to a value of the target data.

[0060] In some embodiments, if the value range of the target data in each first business data is different, the corresponding data coding system can be determined according to the value range of the target data in each first business data, and the first business data can be encoded by the corresponding data coding system, so that each first business data can be compressed. In some embodiments, the server can also comprehensively determine the data coding system based on the respective value ranges of all the first business data, and the comprehensively determined data coding system can be used to compress the target data in all the first business data.

[0061] Step 208: compress the first service data based on the binary encoding result to obtain first compressed service data corresponding to the first service data.

[0062] Among them, the first compressed business data is the compression result obtained after data compression is performed on the first business data. Optionally, the server can perform data compression on the first business data based on the binary encoding result, such as the server can replace the target data in the first business data with the corresponding binary encoding result, thereby achieving data compression for the first business data and obtaining the first compressed business data. When the string length of each value of the target data is long and the number of values ​​of the target data is large, the data length of the target data will be very long, and after compression according to the data encoding binary, a binary encoding result including the data encoding binary and the binary encoding range information corresponding to the target data can be obtained, and the data length of the binary encoding result can be effectively compressed, thereby achieving data compression for the first business data. The server can obtain the data compression result based on the first business data, and perform storage or transmission and other processing on the data compression result. In some embodiments, after receiving the first compressed business data, the server decompresses the first compressed business data to obtain a base coding result. The server can decode based on the data coding base and base coding range information included in the base coding result to restore the target data, and obtain the first business data based on the target data, thereby achieving decompression of the first compressed business data and restoring the original first business data.

[0063] In the above-mentioned business data processing method, for the first business data whose target data belonging to the target dimension is range data, the data coding system is determined based on the value range of the target data in the first business data, and the target data in the first business data is encoded according to the data coding system to obtain a system coding result including the data coding system and the system coding range information corresponding to the target data, and the first business data is compressed based on the system coding result to obtain the first compressed business data corresponding to the first business data. For the first business data whose target data belonging to the target dimension is range data, the data coding system is determined based on the value range of the target data to obtain a system coding result including the data coding system and the system coding range information corresponding to the target data, so as to realize the compression processing of the first business data, simplify the compression processing of the business data, and thus improve the compression processing efficiency of the business data.

[0064] In an exemplary embodiment, Figure 3 As shown, the binary coding process, that is, encoding the target data in the first service data according to the data coding binary system to obtain the binary coding result, includes steps 302 to 308. Among them:

[0065] Step 302: Perform sequence mapping on the value range to obtain a numerical sequence range corresponding to the value range.

[0066] The sequence mapping is a mapping process of mapping the value range of the target data in the first business data into a numerical sequence range, and the numerical sequence range is a sequence mapping result of the value range of the target data, and the numerical sequence range represents the value range of the target data in numerical form. By mapping the target data into the numerical sequence range, the numerical sequence range can be encoded according to the data encoding system to encode and compress the target data.

[0067] Exemplarily, the server may perform sequence mapping for the value range of the target data, such as the server may perform sequence mapping for the value range of the target data {A, B, C, D, E, F}, to obtain a numerical sequence range corresponding to the value range of the target data, such as {1-6}, where each sequence number in 1-6 corresponds to each value in the value range, such as 1 represents value A, 2 represents value B, 3 represents value C, 4 represents value D, 5 represents value E, and 6 represents value F. In some embodiments, the server may arrange the values ​​of the target data into a sequence, and map each value in the sequence into a numerical value, and then obtain a numerical sequence range based on the numerical value corresponding to each value. For example, the server may map each value in the value range of the target data {A, B, C, D, E, F} into {10, 20, 30, 40, 50, 60}, respectively, to obtain a numerical sequence range {10-60}. In some embodiments, the server may also directly map the value range of the target data. For example, the server may directly map the value range {AF} of the target data to {1-6}.

[0068] Step 304, performing system conversion on the range of the numerical sequence according to the data coding system to obtain a system conversion result of the target data in the data coding system.

[0069] Among them, the numerical sequence range is characterized by a numerical form for the value range of the target data, and the numerical sequence range can be converted into a base, so as to further encode and compress the numerical sequence range according to the base. Optionally, the server can convert the numerical sequence range into a base according to the data encoding base, encode the numerical sequence range by converting the base through the data encoding base, and obtain the base conversion result of the target data under the data encoding base, so that each value of the target data can be characterized by the corresponding base conversion result. In some embodiments, the server can record the mapping relationship between each value of the target data and each base conversion result, so that the target data can be decoded based on the mapping relationship.

[0070] Step 306: Based on the result of the base-code conversion of the target data in the data coding base, obtain base-code range information corresponding to the target data.

[0071] Exemplarily, the server may obtain the base encoding range information corresponding to the target data based on the base conversion result corresponding to each value of the target data. For example, the various values ​​of the target data may include {XXXXXX11, XXXXXX12, XXXXXX13, XXXXXX14, XXXXXX21, XXXXXX22, XXXXXX23, XXXXXX24, XXXXXX31, XXXXXX32, XXXXXX33, XXXXXX34}. After mapping the value range sequence, the numerical sequence range can be {11-34}, and the specific values ​​are represented as {11, 12, 13, 14, 21, 22, 23, 24, 31, 32, 33, 34}. If the data encoding system is quaternary, after performing system conversion according to the quaternary system, the system conversion results of each value can be obtained, respectively, {00, 01, 02, 03, 10, 11, 12, 13, 20, 21, 22, 23}, and the obtained system encoding range information can include {00-23}. In some embodiments, the base encoding range information may be further converted into a base, such as a decimal base, so that the base encoding range information is updated to {00-11}.

[0072] Step 308, obtaining a base coding result according to the data coding base and the base coding range information corresponding to the target data.

[0073] Optionally, the server can combine the data encoding system and the system encoding range information corresponding to the target data to obtain the system encoding result for the target data. For example, if the data encoding system is quaternary and the system encoding range information is {00-11}, the system encoding result can be "{4:00-11}", so that the various values ​​of the target data "{XXXXXX11, XXXXXX12, XXXXXX13, XXXXXX14, XXXXXX21, XXXXXX22, XXXXXX23, XXXXXX24, XXXXXX31, XXXXXX32, XXXXXX33, XXXXXX34}" are compressed into a system encoding result of "{4:00-11}", which can greatly improve the data compression rate and ensure the compression effect. In some embodiments, the server can perform system decoding on the system encoding result based on the mapping relationship between the system encoding result "{4:00-11}" and the various values ​​of the target data and their respective system conversion results, so as to restore the various values ​​of the target data.

[0074] In this exemplary embodiment, after the server maps the value range into a numerical sequence range through a sequence, it performs base conversion according to the data coding base, determines the base coding range information based on the base conversion result, and obtains the base coding result according to the data coding base and the base coding range information, thereby encoding the target data according to the data coding base determined by the value range, without the need for complex compression algorithm processing, simplifying the compression processing of business data, and thus improving the compression processing efficiency of business data.

[0075] In an exemplary embodiment, the business data processing method also includes: when the data coding system is updated, determining the updated data coding system; updating the system coding result in the first compressed business data according to the updated data coding system to obtain the updated first compressed business data; wherein the updated first compressed business data includes the updated system coding result, and the updated system coding result includes the updated data coding system and the system coding range information corresponding to the target data encoded according to the updated data coding system.

[0076] Exemplarily, the data encoding system for the target data can be updated according to actual needs, such as after the value range of the target data changes, the data encoding system can be updated. In some embodiments, the server can detect the value range of the target data, and when it is determined that the value range of the target data has changed, the server can make a judgment based on the changed value range to determine whether the data encoding system needs to be updated, and when the data encoding system needs to be updated, the server determines the updated data encoding system based on the updated value range. In some embodiments, the data encoding system update can also be actively triggered by the user, and the server can determine the updated data encoding system. The server can perform corresponding updates for each first compressed service data according to the updated data encoding system, and specifically can update the system encoding result in the first compressed service data, so as to obtain the updated first compressed service data. After the first compressed service data is updated, the updated system encoding result is obtained by encoding based on the updated data encoding system, and the updated first compressed service data includes the updated system encoding result, and the updated system encoding result includes the updated data encoding system and the system encoding range information corresponding to the target data encoded according to the updated data encoding system.

[0077] In this exemplary embodiment, when the data coding system is updated, the server encodes the system result in the first compressed business data according to the updated data coding system to obtain the updated first compressed business data, so that corresponding updates can be made in time according to the changes in the data coding system to ensure the accuracy of the first compressed business data.

[0078] In an exemplary embodiment, the business data processing method also includes: obtaining second business data associated with the first business data, the second business data including data belonging to a target dimension; serializing and reducing the dimension of the second business data according to the data dimension to obtain the reduced second business data; mapping the data belonging to the target dimension in the reduced second business data into serial number information to obtain compressed second business data; and obtaining second compressed business data based on the mapping relationship between the compressed second business data and the data belonging to the target dimension in the compressed second business data and each serial number information.

[0079] Among them, the second business data is different business data in the business system that has an association relationship with the first business data. Specifically, both the first business data and the second business data include data of the target dimension, so that the first business data and the second business data can be associated based on the target dimension. For the first business data and the second business data that both include data of the target dimension, the first business data and the second business data can be combined and processed based on the target dimension, such as merging the first business data and the second business data according to the target dimension to obtain new business data. The second business data may include multiple data dimensions, and each data dimension may include corresponding data. For example, the second business data may be a student's test score in the education system, and specifically may include the test scores of various subjects. Each subject can be used as a data dimension, so that each second business data may include the test scores of each subject of a student. The sequence number information is the result of mapping the data belonging to the target dimension in the second business data, so that the corresponding data is characterized by the sequence number information. For example, data 1 under the target dimension is {XXXXXXXXX1}, which can be mapped to {1}, and data 2 under the target dimension is {WWWWWWWWWW2}, which can be mapped to 2, so that data compression for the first business data can be achieved through the sequence number information. A mapping relationship may be set between the sequence number information and the corresponding represented data, so that the sequence number information can be queried and restored to the corresponding data based on the mapping relationship, thereby realizing decompression and restoration of the compressed data.

[0080] Exemplarily, the server can obtain the second business data associated with the first business data, specifically, the second business data associated with the first business data including the target dimension data can be queried from the database of the business system. The server can perform data compression processing on the second business data according to the data dimension, specifically, the second business data can be serialized and reduced in dimension according to the data dimension, such as serializing and reducing the dimension of the second business data based on JSON (JavaScript Object Notation, JavaScript, object representation) to obtain the second business data after dimension reduction. The server can map the data belonging to the target dimension, specifically mapping it to sequence number information, thereby mapping and compressing the data belonging to the target dimension to obtain the compressed second business data. In some embodiments, when the server maps the data belonging to the target dimension to sequence number information, it can also record the mapping relationship between each data and the respective sequence number information. The server can construct the second compressed business data based on the compressed second business data and the mapping relationship between the individual data and the respective sequence number information, thereby realizing compression processing for the second business data.

[0081] In this exemplary embodiment, the server can perform serialization and dimensionality reduction according to the data dimension for the second business data including the target dimension data, and map the data belonging to the target dimension into serial number information, thereby realizing compression processing for the second business data to obtain second compressed business data. The second business data can be compressed by serialization and dimensionality reduction and serial number information mapping methods to ensure the compression processing efficiency of the second business data.

[0082] In an exemplary embodiment, Figure 4 As shown, a business data processing method is provided, which is applied to Figure 1 The server in the example is used to illustrate, including the following steps 402 to 408. Among them:

[0083] Step 402: Obtain a service data processing request, and obtain first compressed service data according to the service data processing request.

[0084] The business data processing request is used to request processing of business data, and can be obtained by sending it to the server through the terminal by the user. The first compressed business data is obtained by compressing the first business data, and the first compressed business data is decompressed to obtain the first business data, so as to perform corresponding business processing based on the first business data. The target data belonging to the target dimension in the first business data is range data, and the range data is data including at least two values.

[0085] Exemplarily, a user may send a service data processing request to a server through a terminal, and the service data processing request may carry a data identifier or service data to be processed, and the server may obtain the first compressed service data to be processed based on the service data processing request. In some embodiments, the service data processing request may directly carry a service data processing request, and the server may directly parse the service data processing request to obtain the first compressed service data. In some embodiments, the service data processing request may carry a data identifier, and the server may query and obtain the corresponding first compressed service data based on the data identifier.

[0086] Step 404, obtaining a base coding result from the first compressed business data, where the base coding result includes data coding base and base coding range information corresponding to data belonging to the target dimension in the first compressed business data.

[0087] Exemplarily, the server can obtain a base encoding result based on the first compressed business data, and the base encoding result includes data encoding base and base encoding range information. The data encoding base describes the encoding method for the target data belonging to the target dimension in the first business data, and the base encoding range information is the encoding result corresponding to the value range of the target data. Based on the data encoding base and base encoding range information, the target data in the first business data can be decoded to achieve decompression and restoration of the first business data.

[0088] Step 406, decode the base encoding range information according to the data encoding base to obtain target data belonging to the target dimension in the first compressed business data, where the target data is range data including at least two values.

[0089] Optionally, the server may decode the base encoding range information according to the data encoding base to obtain target data belonging to the target dimension, thereby decoding and restoring the target data carried in the first compressed service data.

[0090] Step 408: Based on the target data belonging to the target dimension in the first compressed business data, decompress the first compressed business data to obtain the first business data.

[0091] Exemplarily, the server can decompress the first compressed business data based on the decoded target data. For example, the server can replace the decoded target data with the binary encoding result in the first compressed business data, thereby decompressing the first compressed business data and obtaining the first business data.

[0092] In the above-mentioned business data processing method, first compressed business data is obtained according to a business data processing request, and a base coding result is obtained from the first compressed business data, wherein the base coding result includes the data coding base and the base coding range information corresponding to the data belonging to the target dimension in the first compressed business data, and the base coding range information is decoded according to the data coding base to obtain the target data belonging to the target dimension in the first compressed business data, and based on the target data belonging to the target dimension in the first compressed business data, the first compressed business data is decompressed to obtain the decompressed first business data. For the first compressed business data including the base coding result, the base coding range information included is decoded according to the data coding base in the base coding result, and the data is decompressed based on the target data belonging to the target dimension obtained by decoding to obtain the first business data, which simplifies the decompression processing for the first business data, thereby improving the decompression processing efficiency of the business data.

[0093] In an exemplary embodiment, the base coding range information is decoded according to the data coding base to obtain the target data belonging to the target dimension in the first compressed business data, including: determining the base conversion result in the data coding base based on the base coding range information; performing base conversion on the base conversion result according to the data coding base to obtain a numerical sequence range; and performing mapping and restoration on the numerical sequence range to obtain the target data belonging to the target dimension in the first compressed business data.

[0094] Among them, each value of the target data under the target dimension in the first business data is characterized by the corresponding base conversion result, and the base conversion result data encoding base is encoded by converting the base to obtain the numerical sequence range. Exemplarily, the server can determine the base conversion result under the data encoding base based on the base encoding range information, such as the server can expand each base conversion result in the base encoding range information to obtain the base conversion result corresponding to each value of the target data. The server can perform base conversion on the base conversion result according to the data encoding base to obtain the numerical sequence range. The numerical sequence range is the sequence mapping result of the value range of the target data, and the numerical sequence range represents the value range of the target data in numerical form. The server can perform mapping and restoration on the numerical sequence range to obtain the target data belonging to the target dimension in the first compressed business data.

[0095] In this exemplary embodiment, the server decodes based on the data encoding base and base encoding range information in the base encoding result, thereby decompressing the first compressed business data. There is no need to use a complex compression algorithm for decompression, which simplifies the decompression of the business data and improves the decompression efficiency of the business data.

[0096] In an exemplary embodiment, the business data processing method also includes: obtaining second compressed business data according to a business data processing request; obtaining the compressed second business data and the mapping relationship between the data belonging to the target dimension in the compressed second business data and each serial number information from the second compressed business data; according to the mapping relationship, mapping and restoring the serial number information corresponding to the target dimension in the second compressed business data to obtain the reduced-dimensional second business data; performing dimensionality-upgrading and restoration according to the data dimension on the reduced-dimensional second business data to obtain the second business data, wherein the second business data includes the data belonging to the target dimension.

[0097] Among them, the second compressed business data is obtained by compressing the second business data, and the second business data includes data belonging to the target dimension. Exemplarily, when the business data processing request also includes the second compressed business data or includes the data identifier of the second compressed business data, the server can obtain the second compressed business data based on the business data processing request, and parse the second compressed business data to obtain the mapping relationship between the compressed second business data and the data belonging to the target dimension and each sequence number information. The server can map and restore the sequence number information corresponding to the target dimension in the second compressed business data according to the mapping relationship to restore the second business data after dimensionality reduction, and the data belonging to the target dimension in the second business data after dimensionality reduction has been restored. The server can perform dimensionality increase and restoration according to the data dimension based on the second business data after dimensionality reduction, thereby realizing data decompression for the second compressed business data and obtaining the second business data, so as to perform corresponding business processing based on the second business data.

[0098] In this exemplary embodiment, the server may decompress the second compressed business data based on the mapping relationship between the data dimension and the sequence number information, and may determine the compression processing efficiency of the second business data.

[0099] In an exemplary embodiment, the business data method also includes: determining a business processing data scope based on a business data processing request; traversing the first business data to obtain first target business data according to the business processing data scope, wherein the data belonging to the target dimension in the first target business data includes data with a single value; obtaining second target business data from the second business data according to the business processing data scope, wherein the data belonging to the target dimension in the second target business data matches the data belonging to the target dimension in the first target business data; and performing business processing on the first target business data and the second target business data according to the business data processing request.

[0100] Among them, the business processing data range is used to filter business data, so as to determine the business data that needs to be processed. Specifically, the first business data and the second business data obtained by decompression can be filtered to determine the data that needs to be processed accordingly according to the business data processing request. The first target business data is the first business data that needs to be processed according to the business processing data range. The data value under the target dimension in the first target business data is a single value, that is, the data belonging to the target dimension in the first target business data has 1 value. The second target business data is the second business data that needs to be processed according to the business processing data range. The data belonging to the target dimension in the second target business data matches the data belonging to the target dimension in the first target business data. For example, the data belonging to the target dimension in the second target business data is the same as the data belonging to the target dimension in the first target business data. Therefore, the first target business data and the second target business data can be jointly processed through the target dimension.

[0101] Exemplarily, the server may determine the business processing data range based on the business data processing request, such as the server may parse the business data processing request to determine the business processing data range based on the data range field of the business data processing request. The server may filter the first business data obtained by decompression based on the business processing data range to obtain the first target business data, the data belonging to the target dimension in the first target business data is data including a single value, and the data belonging to the target dimension in the first target business data is split into a single value. For example, the target data values ​​belonging to the target dimension in the first business data A include A, B and C, then the first business data A may be split according to each value of the target data, so as to split the first business data A into the first business data A1 (the data value belonging to the target dimension is A), the first business data A2 (the data value belonging to the target dimension is B) and the first business data A3 (the data value belonging to the target dimension is C), and when the first business data A1 and the first business data A2 belong to the business processing data range, the server may determine from the first business data A that the first business data A1 and the first business data A2 belong to the first target business data.

[0102] The server can filter the second business data obtained by decompression according to the business processing data range to filter out the second target business data from the second business data, and the second target business data is associated with the first target business data, and the data belonging to the target dimension in the specific second target business data matches the data belonging to the target dimension in the first target business data. For example, the data belonging to the target dimension in the second target business data B1 is the same as the data belonging to the target dimension in the first target business data A1, both of which are a1; for another example, the difference between the data belonging to the target dimension in the second target business data B2 and the data belonging to the target dimension in the first target business data A2 does not exceed the difference threshold. The server can perform corresponding business processing on the first target business data and the second target business data obtained according to the business data processing request, such as the server can splice the first target business data and the second target business data according to the target dimension to obtain the spliced ​​target business data. The server can also feed back the spliced ​​target business data to the user terminal for intuitive display in the terminal.

[0103] In this exemplary embodiment, the server screens out matching first target business data and second target business data from the decompressed first business data and second business data according to the business processing data range determined by the business data processing request, and performs corresponding business processing on the first target business data and the second target business data according to the business data processing request, thereby ensuring the processing efficiency of the business data.

[0104] The present application also provides an application scenario, and the application scenario applies the above-mentioned business data processing method. Specifically, the application of the business data processing method in the application scenario is as follows:

[0105] This application scenario involves the processing of employee salary data in an enterprise financial system. In the enterprise financial system, for range data including multiple values, in terms of data storage, the range data is serialized into text storage using JSON format, and a general text compression tool is used to further shorten the text length, thereby effectively reducing the storage space occupied. For example, the original information is "{17943334254:[17911232211,17911232212,17911232213],17943334255:[17911232211,17911232212,17911232213],17943334256:[17911232211,17911232212,17911232213]}". The text is compressed using the lz series algorithm (a type of data compression algorithm based on dictionary encoding, which extracts all phrases and replaces the phrases themselves with the subscripts of the phrases in the full text). The possible compression result is "map:{17943334254,1794333425517943334256,17911232211,17911232212,17911232213},val:{0:[3,4,5],1:[3,4,5],2:[3,4,5]}". When the data is used, the text is read into the memory, decompressed using the compression algorithm, and then the decompressed text is deserialized into readable mapping table information. However, the data compression result must be decompressed and deserialized before it can be used, which will frequently decode and convert data, thereby wasting computing resources, resulting in low compression and decompression processing efficiency of business data, thereby reducing the processing efficiency of business data.

[0106] Based on this, in the business data processing method provided by the present application, the business data including the range data is compressed and decompressed using the base, which mainly depends on how the original data is encoded, the encoding result is directly used for calculation (without decoding), and the data decoding method. Furthermore, the business data processing method provided by the present application understands the scope of range information, focuses on solving the range information that has one-dimensional, two-dimensional or multi-dimensional point composition, has the high and low bit properties of numerical values ​​in mathematics, and the range has continuity. Moreover, when encoding the range data using the base scheme, it is necessary to pay close attention to the changes in the base to prevent data decoding failure from obtaining erroneous identification code information.

[0107] Specifically, Figure 5As shown, in this application scenario, the business data of the business data processing method provided by the present application may include first business data (business data B) and second business data (business data A), and the target dimension includes the salary level and salary grade dimension of salary, that is, the data under the salary level and salary grade dimension in business data B includes multiple values. For business data A, the server can compress it through data dimension reduction and primary key mapping to obtain compressed data A, and store it for ; for business data B, the server can compress it through binary encoding to obtain compressed data B and store it for . When it is necessary to perform Cartesian calculation processing on business data A and business data B, for business data A, the server can obtain data from the database by row and then decompress it through data dimension upgrading and sequence number mapping to obtain business data A; for business data B, the server can calculate the total number to be obtained from the database and then obtain data by row, and obtain business data B after binary decoding for . The server can perform Cartesian calculation processing based on the decompressed business data A and business data B, and preview the calculation results.

[0108] Among them, when reducing the dimension of business data A, it is achieved through data dimension reduction and primary key mapping. The structure of business data A is a three-dimensional data structure, and the three dimensions are: project, salary level, and salary grade, as shown in Table 1. Projects include overtime pay, performance pay, and position pay; salary levels are salary grade divisions, such as 200, 100, etc.; salary grades are further divisions of salary grades, such as grade 1 and grade 2.

[0109] Table 1

[0110]

[0111]

[0112] When the number of projects = 5, the number of salary levels = 200, and the number of salary grades = 100, the total number of rows of business data A = 5*200*100 = 10w. It takes a long time to save 10w rows of data. In order to speed up business response, you can perform "data dimensionality reduction" on business data A, reduce the original three dimensions to one dimension, retain only the project dimension, and serialize the two dimensions of salary level and salary grade into json strings. After dimensionality reduction, 10w rows of data are converted into 5 rows, which speeds up the business response of storage and query.

[0113] At the same time, the 19-digit primary key of salary level and salary grade, which is relatively long, is mapped into a short serial number using the "primary key mapping" method to save storage space. The specific primary key mapping relationship of the salary level dimension is shown in Table 2 below.

[0114] Table 2

[0115] Salary Primary Key Serial number 1st class 1766825583079188488 1 2nd level 1766825583079188489 2 3rd grade 1766825583079188490 3

[0116] The primary key mapping relationship of the salary grade dimension can be shown in Table 3 below.

[0117] Table 3

[0118] Salary Primary Key Serial number 1st gear 1766825073790019584 1 2nd gear 1766825073790019585 2 3rd gear 1766825073790019586 3 4 gears 1766825073790019587 4 5 gears 1766825073790019588 5

[0119] The data structure obtained after compressing the business data A can be shown in Table 4 below.

[0120] Table 4

[0121]

[0122] For business data B, its data structure is a multidimensional data structure, and the number of attributes is uncertain. Attributes such as positions, departments, and ranks, business data B may include 2 attributes, 3 attributes, 10 attributes, etc., and salary grades and salary grades are range data. The details are shown in Tables 5 and 6 below.

[0123] Table 5

[0124] post department Salary level Software Development Changsha 1 1st grade 1st grade~20th grade 8th grade Software Development Shenzhen 1 6th grade 1st grade~20th grade 8th grade Product Design Changsha 1 1st grade, 1st grade ~ 10th grade, 8th grade Product Design Shenzhen 1 3rd grade, 1st grade ~ 10th grade, 8th grade

[0125] Table 6

[0126] post department Rank Salary level Software Development Changsha 1 T2 1st grade 1st grade~20th grade 8th grade Software Development Shenzhen 1 T3 6th grade 1st grade~20th grade 8th grade Product Design Changsha 1 P1 1st grade, 1st grade ~ 10th grade, 8th grade Product Design Shenzhen 1 P2 3rd grade, 1st grade ~ 10th grade, 8th grade

[0127] Among them, for range data, if the number of salary grades is 3 and the number of salary grades is 8, then grade 1 to grade 2 can be represented by: {grade 1, grade 2, grade 3, grade 4, grade 5, grade 6, grade 7, grade 8, grade 1, grade 2, grade 2, grade 3, grade 4, grade 5, grade 6, grade 7, grade 8}, that is, the data includes multiple values ​​and belongs to range data.

[0128] Furthermore, if Figure 6 As shown, when the business data B is encoded, calculated, and decoded, the business data B with 3 salary levels and 3 salary grades can be encoded according to the system to obtain the encoding result {rangeCode:”0-8”, ranknum:3}. When updating the encoding system, the server can parse the subscript based on the old system, and calculate based on the new system to obtain a new encoding result {rangeCode:”0-14”, ranknum:6}, where the change in salary grade represents the change in the system, that is, it represents the change in the system. When performing difference and union calculations on different business data B, the set operation can be directly converted into a merge calculation of digital intervals, thereby improving the processing efficiency of the difference and union calculation. For range data queries, the mapping relationship of the query response can be obtained after the carry value is decoded into the business data id, thereby obtaining the queried data.

[0129] Furthermore, the server can significantly reduce the space occupied by converting the salary grade range based on the algorithm of base, and at the same time, the calculation performance for difference, union and other operations is better. Specifically, the salary grade number is used as the mathematical base to dynamically encode the salary grade range information to reduce the storage space occupied. At the same time, this encoding result supports the fast calculation of the grade range (especially in the intersection and union calculation), and saves more computing memory consumption. For example, in the range of 3 grades and 3 grades, the base is 3 (that is, every three is incremented), and the data encoding process may include:

[0130] A certain range is (1st grade 1st grade to 3rd grade 3rd grade), and the corresponding identification number information is:

[0131] {

[0132] 17943334254:[17911232211,17911232212,17911232213],

[0133] 17943334255:[17911232211,17911232212,17911232213],

[0134] 17943334256:[17911232211,17911232212,17911232213]

[0135] }

[0136] Convert the identification number information to a subscript value, and the result is "{0:[0,1,2],1:[0,1,2],2:[0,1,2]}". Consider the numbers in the brackets as the lower digits of the two-digit number, and the numbers before the colon as the higher digits of the two-digit number, thus obtaining nine ternary number sequences [00,01,02,10,11,12,20,21,22]. That is, in the 3*3 dimension, the maximum data range is 9 numbers. In the above processing, all the numbers in the range are actually enumerated. When the dimension value becomes very large, its range can be recorded. Specifically, we can directly calculate the range of [00,01,02,10,11,12,20,21,22] above as 0 to 8 (decimal). However, in actual situations, we do not need to enumerate and then evaluate in the encoding stage, because during use, we can set the start and end of the value range. For example, we can set the start range to 1 grade and 1 gear, and set the end range to 3 grades and 3 gears. The subscript range is 01 to 22, and then converted to decimal 0 to 8. The storage format becomes "{rangeCode: "0-8", ranknum: 3}". This storage information contains the base and range, and the storage information can be interpreted as a range between 0 and 8 in base 3.

[0137] Through this encoding method, the huge ID mapping table can be converted into a very short text string. And this text string is easy to understand. Specifically, it is easy to calculate the intersection and union of range data. For example, a multi-range data "1st grade 1~3rd grade 3rd grade, 2nd grade 1~4th grade 1st grade, 5th grade 1~6th grade 3rd grade", because there may be overlaps between ranges, the union needs to be calculated. According to the storage format introduced above, the actual range data is "0-14, 6-18, 24-32", and the union result can be calculated very quickly as "0-18; 24-32". Similarly, the intersection is also easy to calculate.

[0138] Furthermore, when the data encoding base changes from 3 to 6, if the user adjusts the gear up or down, the previously recorded results must be inaccurate (the numbers are all in base 3), and need to be converted to base 6. The conversion process is as follows: the base value is calculated using modulo division to calculate the high and low digits (the high digit of 0 = 0 (0 / 3), the low digit of 0 = 0 (0%3)). Then the range of "0-8" is calculated to the high and low digits of the starting range, which are 0 / 0 and 2 / 2 respectively. Then the new value is calculated using base 6, and the starting value is 0 (0*6+0) and the ending value is 14 (2*6+2). The new storage value obtained is "{rangeCode:"0-14",ranknum:6}". When the business needs to query range information, the server reads the stored range "0-3" (4-bit) and needs to flatten the range data to get [0,1,2,3] and perform modulo division to get the high and low bits of each range value as [0=(0,0),1=(0,1),2=(0,2),3=(0,3)], and then get the corresponding identification number according to the high and low bit subscripts: "{17943334254:[17911232211,17911232212,17911232213,17911232213]}".

[0139] When performing Cartesian calculation processing on the decompressed business data A and business data B, it can be implemented based on the memory Cartesian data acquisition model. In terms of Cartesian calculation, most of the existing technologies directly store the result data of Cartesian calculation, and then alleviate the database pressure by horizontal partitioning to solve the interactive performance problem. Moreover, the stored results cannot support the reading and recognition of information by the human eye, and the readability is poor. The business data processing method provided in this application does not directly store the Cartesian calculation results, but only stores the original data, and then accurately reads part of the original data into the memory to perform Cartesian calculation to obtain the desired data. Because the amount of data of the Cartesian calculation results is too large, it will reach billions or even hundreds of billions.

[0140] like Figure 5As shown, the preview effect required after Cartesian calculation of business data A and business data is achieved, that is, business data B is fully expanded and combined with business data A for display, and each column of business data B in each row needs to be Cartesian product calculated. After querying business data B from the database by row and performing "base decoding", the data structure in Table 7 below can be obtained.

[0141] Table 7

[0142]

[0143] In the display view, data is displayed according to paging, and the current page number currentPage and the size of data displayed on each page pageSize are passed in. The present invention dynamically calculates the Cartesian product in memory through currentPage and pageSize, and returns the Cartesian calculation result to the view model for display.

[0144] 1. Calculate the total number of Descartes rows

[0145] totalCount=row1Count+row2Count+row3Count+...

[0146] row1Count=count(attribute A)*count(attribute B)*count(attribute B)*count(attribute Z)*count(salary level and salary grade range);

[0147] For example, the number of rows with row number 1 in Table 7 is:

[0148] row1Count=3*2*2*2*(2*100)=4800;

[0149] count(salary grade and salary range) = 1st grade 1~1st grade 100+2nd grade 1~2nd grade 100=1*100+1*100=200;

[0150] For example, the number of rows with row number 2 in Table 2-7 is:

[0151] row2Count=1*1*1*2*(159*100+1*98)=31996;

[0152] count(salary grade and salary range) = 1st grade 1 to 159th grade 100th grade + 160th grade 1 to 160th grade 98th grade = 159*100+1*98=15998.

[0153] 2. Calculate the starting coordinates of Cartesian data acquisition

[0154] The server calculates the fetching coordinates by taking the salary grade and range as the units digit, attribute Z as the tens digit, attribute C as the hundreds digit, attribute B as the thousands digit, and so on. Here, the base between the units digit and the tens digit is not necessarily a carry of 1 for every 10, but is determined by the number of salary grade and range in the current row. For example, in Table 2-7, the base between the units digit and the tens digit for row number 1 is count(salary grade and range) = 200; the base between the units digit and the tens digit for row number 2 is count(salary grade and range) = 15998. Of course, the bases between the tens digit and the hundreds digit, the hundreds digit and the thousands digit, etc. are not fixed either, but are determined by the number of attribute values in the current row and the current attribute column. For example, in Table 2-7, the base between the tens digit and the hundreds digit for row number 1 is count(attribute Z) = 2; the base between the hundreds digit and the thousands digit for row number 12 is count(attribute C) = 1. Specifically as follows:

[0155] (1) Based on currentPage and pageSize, the start parameter startParam and the end parameter endParam can be obtained.

[0156] (2) Calculate the start row index startRowIndex of the Cartesian fetching starting coordinate according to the value of startParam.

[0157] ① Calculate the total number of rows in row 1, row1Count. When startParam < row1Count, it means the Cartesian fetching starting coordinate is in row 1, and return; otherwise, go to the next row;

[0158] ② Calculate the total number of rows in row 2, row2Count. When startParam < row1Count + row2Count, it means the Cartesian fetching starting coordinate is in row 2, and return; otherwise, go to the next row;

[0159] ③ Calculate the total number of rows in row 3, row3Count. When startParam < row1Count + row2Count + row3Count, it means the Cartesian fetching starting coordinate is in row 3, and return; otherwise, go to the next row;

[0160] (3) Calculate the precise position of the Cartesian fetching starting coordinate to determine from which column's which attribute value to start fetching.

[0161] ① If startRowIndex is determined to be in row n in (2), then the starting position needs to be calculated in row n:

[0162] rowCalCount = startParam - row1Count - row2Count -... - rown - 1Count;

[0163] ② Initialize the Cartesian Coordinate of the nth row as:

[0164] {

[0165] Attribute A: 0,

[0166] Attribute B: 0,

[0167] Attribute C: 0,

[0168] Attribute Z: 0,

[0169] Salary level: 0

[0170] }

[0171] ③ Add the Cartesian coordinates of different bases, and add rowCalCount:

[0172] cartesianCoordinate=cartesianCoordinate+rowCalCount. The result is the starting coordinate of Cartesian counting.

[0173] 3. Start taking data from the Cartesian starting coordinates, return data when the endParam-startParam quantity is reached or there is no more data, and perform Cartesian calculations. Figure 7 As shown, when fetching data, the server can set the pointer 1 of the salary level and salary file to point to GR1, select A1 according to attribute A, select B1 according to attribute B, select C1 according to attribute C, and select N1 according to attribute N, so that the data fetched is a combination of the data selected by each attribute, specifically {N1, ..., C1, B1, A1, GR1}; then read the next data, such as Figure 8 As shown, the specific pointer 1 can be switched to point to GR2, thereby obtaining {N1, ..., C1, B1, A1, GR2}; after traversing each attribute in this way, the required business data can be obtained.

[0174] In the business data processing method provided by the present application, the Cartesian calculation results are not directly stored, but only the original data is stored, and then the desired data is obtained by accurately reading part of the original data into the memory for Cartesian calculation. Moreover, in Cartesian memory calculation, the Cartesian number coordinates, the units, tens, hundreds, etc. of the Cartesian number coordinates are abstracted. The business data processing method provided by the present application can effectively reduce the storage space occupied by the data and speed up the response. By utilizing the characteristic that the memory calculation rate is much higher than the disk reading rate, the memory Cartesian calculation can achieve a 3-second response for loading billions of data.

[0175] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0176] Based on the same inventive concept, the embodiment of the present application also provides a business data processing device for implementing the business data processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more business data processing device embodiments provided below can refer to the limitations on the business data processing method above, and will not be repeated here.

[0177] In an exemplary embodiment, Fig. 9 As shown, a service data processing device 900 is provided, comprising: a first service data acquisition module 902, a coding system determination module 904, a coding processing module 906 and a compression processing module 908, wherein:

[0178] A first business data acquisition module 902 is used to acquire first business data, where target data belonging to a target dimension in the first business data is range data, and the range data is data including at least two values;

[0179] A coding system determination module 904, configured to determine a data coding system based on a value range of target data in the first business data;

[0180] The encoding processing module 906 is used to encode the target data in the first business data according to the data encoding system to obtain a system encoding result, where the system encoding result includes the data encoding system and system encoding range information corresponding to the target data;

[0181] The compression processing module 908 is used to compress the first business data based on the binary encoding result to obtain first compressed business data corresponding to the first business data.

[0182] In an exemplary embodiment, the encoding processing module 906 is also used to perform sequence mapping on the value range to obtain a numerical sequence range corresponding to the value range; perform base conversion on the numerical sequence range according to the data coding base to obtain a base conversion result of the target data in the data coding base; obtain base coding range information corresponding to the target data based on the base conversion result of the target data in the data coding base; obtain a base coding result according to the data coding base and the base coding range information corresponding to the target data.

[0183] In an exemplary embodiment, the encoding processing module 906 is also used to determine the updated data coding system when the data coding system is updated; update the system encoding result in the first compressed business data according to the updated data coding system to obtain the updated first compressed business data; wherein the updated first compressed business data includes the updated system encoding result, and the updated system encoding result includes the updated data coding system and the system encoding range information corresponding to the target data encoded according to the updated data coding system.

[0184] In an exemplary embodiment, it also includes a second business data processing module, which is used to obtain second business data associated with the first business data, wherein the second business data includes data belonging to the target dimension; serializing and reducing the dimension of the second business data according to the data dimension to obtain the reduced-dimensional second business data; mapping the data belonging to the target dimension in the reduced-dimensional second business data into serial number information to obtain compressed second business data; and obtaining the second compressed business data based on the mapping relationship between the compressed second business data and the data belonging to the target dimension in the compressed second business data and each serial number information.

[0185] In an exemplary embodiment, Fig.10 As shown, a service data processing device 1000 is provided, comprising: a processing request acquisition module 1002, an encoding result acquisition module 1004, a decoding processing module 1006 and a decompression processing module 1008, wherein:

[0186] The processing request acquisition module 1002 is used to acquire a service data processing request and obtain the first compressed service data according to the service data processing request;

[0187] The encoding result acquisition module 1004 is used to obtain a base encoding result from the first compressed service data, where the base encoding result includes the data encoding base and base encoding range information corresponding to the data belonging to the target dimension in the first compressed service data;

[0188] A decoding processing module 1006 is used to decode the binary encoding range information according to the data encoding binary system to obtain target data belonging to the target dimension in the first compressed service data, where the target data is range data including at least two values;

[0189] The decompression processing module 1008 is used to decompress the first compressed business data based on the target data belonging to the target dimension in the first compressed business data to obtain the first business data.

[0190] In an exemplary embodiment, the decoding processing module 1006 is also used to determine the base conversion result under the data coding base based on the base coding range information; perform base conversion on the base conversion result according to the data coding base to obtain a numerical sequence range; perform mapping and restoration on the numerical sequence range to obtain target data belonging to the target dimension in the first compressed business data.

[0191] In an exemplary embodiment, it also includes a second business data processing module, which is used to obtain second compressed business data according to a business data processing request; obtain the compressed second business data and the mapping relationship between the data belonging to the target dimension in the compressed second business data and each serial number information from the second compressed business data; according to the mapping relationship, map and restore the serial number information corresponding to the target dimension in the second compressed business data to obtain the reduced-dimensional second business data; perform dimensionality-upgrading and restoration according to the data dimension on the reduced-dimensional second business data to obtain the second business data, which includes the data belonging to the target dimension.

[0192] In an exemplary embodiment, a joint processing module is also included, which is used to determine the business processing data range based on the business data processing request; according to the business processing data range, first target business data is traversed from the first business data, and the data belonging to the target dimension in the first target business data includes data with a single value; according to the business processing data range, second target business data is obtained from the second business data, and the data belonging to the target dimension in the second target business data matches the data belonging to the target dimension in the first target business data; according to the business data processing request, business processing is performed on the first target business data and the second target business data.

[0193] Each module in the above-mentioned business data processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0194] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.11 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data involved in the business data processing method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a business data processing method is implemented.

[0195] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0196] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0197] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0198] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0199] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0200] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0201] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0202] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A business data processing method, characterized in that: The method comprises: Acquire first business data, wherein target data belonging to a target dimension in the first business data is range data, and the range data is data including at least two values; Determine a data encoding system based on a value range of the target data in the first service data; According to the data coding system, the target data in the first service data is encoded to obtain a system encoding result, wherein the system encoding result includes the data coding system and system encoding range information corresponding to the target data; The first service data is compressed based on the binary encoding result to obtain first compressed service data corresponding to the first service data.

2. The method according to claim 1, characterized in that The step of encoding the target data in the first service data according to the data encoding system to obtain a system encoding result includes: Perform sequence mapping on the value range to obtain a numerical sequence range corresponding to the value range; Performing system conversion on the numerical sequence range according to the data coding system to obtain a system conversion result of the target data in the data coding system; Based on the base conversion result of the target data in the data coding base, obtaining base coding range information corresponding to the target data; A base coding result is obtained according to the data coding base and the base coding range information corresponding to the target data.

3. The method according to claim 1, characterized in that The method further comprises: When the data coding system is updated, determining the updated data coding system; The binary encoding result in the first compressed service data is updated according to the updated data encoding binary system to obtain updated first compressed service data; Among them, the updated first compressed service data includes an updated system coding result, and the updated system coding result includes the updated data coding system and the system coding range information corresponding to the target data encoded according to the updated data coding system.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Acquire second business data associated with the first business data, where the second business data includes data belonging to the target dimension; Performing serialization and dimensionality reduction on the second business data according to the data dimension to obtain the second business data after dimensionality reduction; Mapping the data belonging to the target dimension in the second business data after dimensionality reduction into sequence number information to obtain compressed second business data; Based on the compressed second business data and the mapping relationship between the data belonging to the target dimension in the compressed second business data and each sequence number information, the second compressed business data is obtained.

5. A business data processing method, characterized in that: The method comprises: Obtaining a service data processing request, and obtaining first compressed service data according to the service data processing request; Obtaining a base encoding result from the first compressed service data, where the base encoding result includes data encoding base and base encoding range information corresponding to data belonging to a target dimension in the first compressed service data; Decoding the system encoding range information according to the data encoding system to obtain target data belonging to the target dimension in the first compressed service data, wherein the target data is range data including at least two values; Based on the target data belonging to the target dimension in the first compressed business data, the first compressed business data is decompressed to obtain first business data.

6. The method according to claim 5, characterized in that The decoding of the system encoding range information according to the data encoding system to obtain the target data belonging to the target dimension in the first compressed service data includes: Determine a base conversion result in the data encoding base based on the base encoding range information; Performing base conversion on the base conversion result according to the data encoding base to obtain a numerical sequence range; Mapping and restoration are performed on the numerical sequence range to obtain target data belonging to the target dimension in the first compressed business data.

7. The method according to claim 5 or 6, characterized in that: The method further comprises: Obtaining second compressed service data according to the service data processing request; Obtaining, from the second compressed business data, the compressed second business data and a mapping relationship between the data belonging to the target dimension in the compressed second business data and each sequence number information; According to the mapping relationship, mapping and restoring are performed on the sequence number information corresponding to the target dimension in the second compressed business data to obtain the second business data after dimensionality reduction; The second business data after dimensionality reduction is restored by dimensionality increase according to the data dimension to obtain second business data, wherein the second business data includes data belonging to the target dimension.

8. The method according to claim 7, characterized in that The method further comprises: Determining a business processing data scope based on the business data processing request; According to the business processing data range, first target business data is obtained by traversing from the first business data, wherein the data belonging to the target dimension in the first target business data is data including a single value; According to the business processing data range, second target business data is obtained from the second business data, and data belonging to the target dimension in the second target business data matches data belonging to the target dimension in the first target business data; According to the business data processing request, business processing is performed on the first target business data and the second target business data.

9. A business data processing device, characterized in that: The device comprises: A first business data acquisition module is used to acquire first business data, wherein target data belonging to a target dimension in the first business data is range data, and the range data is data including at least two values; A coding system determination module, used to determine a data coding system based on a value range of the target data in the first business data; A coding processing module, used for encoding the target data in the first business data according to the data coding system to obtain a system coding result, wherein the system coding result includes the data coding system and system coding range information corresponding to the target data; A compression processing module is used to compress the first business data based on the binary encoding result to obtain first compressed business data corresponding to the first business data.

10. A business data processing device, characterized in that: The device comprises: A processing request acquisition module, used to acquire a business data processing request, and obtain first compressed business data according to the business data processing request; A coding result acquisition module, used to obtain a base coding result from the first compressed service data, wherein the base coding result includes data coding base and base coding range information corresponding to data belonging to a target dimension in the first compressed service data; A decoding processing module, used for decoding the system encoding range information according to the data encoding system to obtain target data belonging to the target dimension in the first compressed service data, wherein the target data is range data including at least two values; The decompression processing module is used to decompress the first compressed business data based on the target data belonging to the target dimension in the first compressed business data to obtain the first business data.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 or 5 to 8 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 or 5 to 8 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 or 5 to 8 are implemented.

Images