A business data processing method, device and equipment

Abstract

This specification relates to the field of distributed data processing technology, specifically disclosing a business data processing method, apparatus, and device. The method is applied to the management node of a distributed business system. The method includes: receiving a data extraction request from a business terminal under specified business parameters and extraction indicators; when it is determined based on the extraction indicators that the data extraction request cannot be directly sent to each physical node for execution, obtaining a first operation instruction and a second operation instruction corresponding to the data extraction request; the first operation instruction refers to an operation instruction that can be directly sent to each physical node for execution; the second operation instruction refers to an operation instruction that calculates the indicator data corresponding to the extraction indicators based on the execution results fed back by each physical node; sending the first operation instruction to each physical node, and processing the execution results fed back by each physical node based on the second operation instruction to obtain the indicator data of the data extraction request, and feeding the results back to the business terminal.

Description

Technical Field

[0001] The present specification relates to the field of distributed data processing technology, and in particular, to a business data processing method, device and equipment. Background Art

[0002] In today's increasingly competitive financial market, financial institutions are becoming more and more flexible in processing business data, the amount of data to be processed is increasing, and there are more and more complex factors to consider. Therefore, how to process business data more quickly and accurately is becoming increasingly important to improve the business competitiveness of financial institutions.

[0003] At present, the business data of financial institutions are usually stored in distributed databases. Under the distributed characteristics, business data is stored in various physical nodes in a dispersed manner. It is usually necessary to use push-down functions and statements to push the processing logic of business data to each physical node for execution, so as to complete the business data processing more quickly and accurately. However, there are still some business data processing logics that cannot be directly pushed to each physical node for execution. For the processing logic of this part of business data, after processing the request, the management node needs to concentrate the data involved in each physical node to one physical node for processing. However, the amount of business data of financial institutions is large. When the data involved reaches a high level, the data distribution will be seriously skewed, making the performance of the physical node worse, and may even cause the physical node to be unavailable, indirectly affecting the normal execution of a large number of statements in the entire distributed system, and thus affecting the stability of business data processing. Summary of the invention

[0004] The purpose of the embodiments of this specification is to provide a business data processing method, device and equipment, which can further improve the overall stability of the distributed business system during the business data processing process.

[0005] This specification provides a business data processing method, device and equipment which are implemented in the following ways:

[0006] A business data processing method is applied to a management node of a distributed business system, the method comprising: receiving a data extraction request for a specified business parameter under an extraction index sent by a business terminal; obtaining a first operation instruction and a second operation instruction corresponding to the data extraction request when it is determined based on the extraction index that the data extraction request cannot be directly issued to each physical node for execution; wherein the first operation instruction refers to an operation instruction that can be directly issued to each physical node for execution; the second operation instruction refers to an operation instruction that calculates index data corresponding to the extraction index based on the execution result fed back by each physical node; issuing the first operation instruction to each physical node and receiving the execution result fed back by each physical node; processing the execution result fed back by each physical node based on the second operation instruction to obtain the index data of the data extraction request; and feeding back the index data to the business terminal.

[0007] In some other embodiments provided by the method described in this specification, the extracted index includes extracting the median or querying the data value located at a specified position.

[0008] In some other embodiments of the method described in this specification, a first operation instruction is issued to each physical node, and the execution result fed back by each physical node is processed based on the second operation instruction, including: issuing an instruction to each physical node to extract the node maximum value and node minimum value of the business data corresponding to the specified business parameter; calculating the maximum value and minimum value of the specified business parameter in the distributed business system based on the node maximum value and node minimum value fed back by each physical node; using the average value of the maximum value and the minimum value as the reference value, issuing an instruction to each physical node to extract the first node business record amount of the business data of the specified business parameter which is less than / greater than the reference value; calculating the total first business record amount of the specified business parameter which is less than / greater than the reference value in the distributed business system based on the first node business record amount fed back by each physical node; and determining the index data of the specified business parameter under the extraction index based on the reference value when the first total business record amount meets the record amount condition corresponding to the extraction index.

[0009] In some other embodiments provided by the method described in this specification, the record quantity condition includes that the total amount of the first business records is equal to a reference value, or the difference between the total amount of the first business records less than the benchmark value and the total amount of the first business records greater than the benchmark value is equal to 1 and the difference between the maximum value and the minimum value is less than a specified precision value.

[0010] In some other embodiments provided by the method described in this specification, when the total amount of the first business records does not meet the record amount condition corresponding to the extraction index, the method further includes: using the benchmark value to update the maximum value or minimum value, and taking the average of the updated minimum value and the maximum value as the updated benchmark value; issuing an instruction to each physical node to extract the first node business record amount of the business data of the specified business parameter which is less than / greater than the updated benchmark value; based on the first node business record amount fed back by each physical node, calculating the total amount of the first business records of the specified business parameter which is less than / greater than the updated benchmark value in the distributed business system; when the total amount of the first business records does not meet the record amount condition corresponding to the extraction index, repeating the above-mentioned updating steps until the total amount of the first business records meets the record amount condition corresponding to the extraction index; and determining the index data of the specified business parameter under the extraction index based on the benchmark value updated in the last step.

[0011] In some other embodiments of the method described in this specification, the extraction index is the extraction median, and the reference value is determined in the following manner: an instruction is sent to each physical node to extract the amount of business records corresponding to all business records of the specified business parameter under the corresponding physical node, and the extracted amount of business records is used as the second-node business record amount; based on the second-node business record amount fed back by each physical node, the total amount of second business records of the specified business parameter in the distributed business system is calculated; one-half of the total amount of the second business records is calculated, and the integer obtained by discarding the decimal places in the calculation result is used as the reference value corresponding to the extraction index.

[0012] On the other hand, an embodiment of the present specification also provides a business data processing device, which is applied to a management node of a distributed business system, and the device includes: a request receiving module, which is used to receive a data extraction request for a specified business parameter under an extraction index sent by a business terminal; an instruction acquisition module, which is used to obtain a first operation instruction and a second operation instruction corresponding to the data extraction request when it is determined based on the extraction index that the data extraction request cannot be directly issued to each physical node for execution; wherein the first operation instruction refers to an operation instruction that can be directly issued to each physical node for execution; the second operation instruction refers to an operation instruction that calculates the index data corresponding to the extraction index based on the execution result fed back by each physical node; an instruction issuing module, which is used to issue the first operation instruction to each physical node; a result receiving module, which is used to receive the execution result fed back by each physical node; an instruction execution module, which is used to process the execution result fed back by each physical node based on the second operation instruction to obtain the index data of the data extraction request; and a result feedback module, which is used to feed back the index data to the business terminal.

[0013] In some other embodiments provided by the device described in this specification, the extracted index includes extracting the median or querying the data value located at a specified position.

[0014] In some other embodiments provided by the device described in this specification, the instruction issuing module is further used to issue instructions to each physical node for extracting node maximum values ​​and node minimum values ​​of business data corresponding to the specified business parameter; the instruction execution module is further used to calculate the maximum value and minimum value of the specified business parameter in the distributed business system based on the node maximum value and node minimum value fed back by each physical node; the instruction issuing module is further used to issue instructions to each physical node for extracting the first node business record quantity whose business data of the specified business parameter is less than / greater than the benchmark value, using the average value of the maximum value and the minimum value as the benchmark value; the instruction execution module is further used to calculate the total amount of first business records whose specified business parameter is less than / greater than the benchmark value in the distributed business system based on the first node business record quantity fed back by each physical node; the instruction execution module is further used to determine the indicator data of the specified business parameter under the extraction indicator based on the benchmark value when the total amount of the first business records meets the record quantity condition corresponding to the extraction indicator.

[0015] On the other hand, an embodiment of this specification also provides a business data processing device, which includes at least one processor and a memory for storing processor executable instructions, and when the instructions are executed by the processor, the steps of the method described in any one or more of the above embodiments are implemented.

[0016] One or more embodiments of the present specification provide a business data processing method, device, and equipment. After receiving a data extraction request, the management node can first determine whether the data extraction request can be directly executed by the physical node. If it cannot be directly executed, it can be processed based on a pre-configured processing method. The processing method may at least include an executable operation instruction that is sent to the physical node for execution, and an operation instruction that is further processed based on the execution result of the physical node. Then, related processing can be performed based on the corresponding operation instruction. Based on this processing method, distributed processing of corresponding data processing requests can be achieved, avoiding performance problems and unstable factors caused by data relocation, and ensuring the stable operation of the distributed system during the processing process. At the same time, fast and accurate processing of data extraction requests can also be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art description. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0018] Figure 1 A flowchart of an embodiment of a business data processing method provided in this specification;

[0019] Figure 2 A schematic diagram of the module structure of a business data processing device provided in this specification. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of the specification, not all of the embodiments. Based on one or more embodiments of the specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the embodiments of this specification.

[0021] In a scenario example provided in the embodiments of this specification, a business data processing method can be applied to a distributed business system, wherein the distributed business system includes a management node and at least one physical node. The business data in the distributed business system is stored in each physical node in a dispersed manner. The management node can analyze and process the processing request received by the distributed business system; and can also issue instructions to each physical node so that each physical node executes the processing corresponding to the corresponding instruction, and can receive the processing results fed back by each physical node, and further perform other processing based on the processing results. The layout architecture of the distributed business system can be configured as needed, which is not limited here.

[0022] For example, a business person needs to extract data that is ranked Nth in the order of the deposit amount within a certain time interval. The business terminal can send a data extraction request with a specified business parameter under the extraction index to the management node. In this scenario example, the data extraction request is to extract data that is ranked Nth in the order of the deposit amount within a certain time interval. Accordingly, the specified business parameter is the deposit amount within the certain time interval, and the extraction index is the data that is ranked Nth in the order of the deposit amount within the certain time interval.

[0023] The management node may issue instructions to each physical node to extract the node maximum value and node minimum value of the business data corresponding to the specified business parameter. The node maximum value and node minimum value refer to the maximum value and minimum value of the business data corresponding to the specified business parameter under a certain physical node. After receiving the instruction from the management node, each physical node may extract the node maximum value and node minimum value of the business data corresponding to the specified business parameter under the physical node, and feed back the extracted node maximum value and node minimum value to the management node.

[0024] After receiving the node maximum value and node minimum value fed back by each physical node, the management node can compare the node maximum value and node minimum value fed back by each physical node to determine the maximum value and minimum value of the business data of the specified business parameter in the distributed business system.

[0025] The management node may use the average value of the maximum value and the minimum value as a reference value, and send an instruction to each physical node to extract the first node business record quantity whose business data of the specified business parameter is less than the reference value. The first node business record quantity may refer to the number of business records whose business data of the specified business parameter under a certain physical node is less than the reference value. After receiving the instruction to extract the first node business record quantity whose business data of the specified business parameter is less than the reference value, the physical node may extract the number of business records whose business data of the specified business parameter is less than the reference value as the first node business record quantity, and feed it back to the management node.

[0026] The management node may calculate the total amount of first business records of the designated business parameter that is less than the reference value in the distributed business system based on the first node business record amount fed back by each physical node. In the case where the total amount of the first business records meets the record amount condition corresponding to the extraction index, the management node may determine the index data of the designated business parameter under the extraction index based on the reference value. The record amount condition may be pre-configured as needed. For example, different record amount conditions may be set according to different extraction indexes.

[0027] For example, the record quantity condition may be that the total amount of the first business records is equal to N. Accordingly, if the total amount of the first business records is less than N, this may indicate that the reference value is smaller than the data value at the Nth position in the ascending order, and the reference value may be used as the minimum value of the designated business parameter in the distributed business system, and the maximum value of the designated business parameter in the distributed business system may be kept unchanged to obtain the updated minimum value and maximum value. Then, the average of the updated minimum value and maximum value is used as the updated reference value.

[0028] The management node may issue an instruction to each physical node to extract the business data of the specified business parameter that is less than the first node business record amount after the updated benchmark value. Each physical node may feed back the extracted first node business record amount to the management node. The management node may calculate the sum of the first node business record amounts fed back by each physical node to obtain the total amount of first business records corresponding to the updated benchmark value. Then, it is determined whether the total amount of first business records is equal to N. If it is still less than N, repeat the above steps. If it is greater than N, it can be explained that the benchmark value is larger than the data value in the Nth position sorted from small to large. The benchmark value can be used as the maximum value of the specified business parameter in the distributed business system, and the minimum value after the update in the previous step is kept unchanged to obtain the minimum and maximum values ​​after the update in this step.

[0029] The average value of the minimum value and the maximum value after the update of this step is used as the updated reference value of this step. The management node can issue an instruction to each physical node to extract the business data of the specified business parameter that is less than the first node business record amount of the reference value after the update of this step. Correspondingly, the management node can also obtain the total amount of the first business record corresponding to the reference value after the update of this step. Then determine whether the total amount of the first business record is equal to N. If it is not equal to N, continue to repeat the above iterative steps until the output result is equal to N.

[0030] When the output result satisfies the record quantity condition corresponding to the extraction index, that is, when the total amount of the first business records is equal to N, it means that the data ranked Nth from smallest to largest is close to the reference value. The management node can further process based on the reference value to more quickly determine the data ranked Nth from smallest to largest.

[0031] If the data distribution characteristics of the specified business parameters can be analyzed, a smaller business data value interval can be set based on the benchmark value, and the upper limit of the business data value interval is the benchmark value. The management node can issue instructions to each physical node to extract data within the business data value interval. The management node can receive data fed back by each physical node to form a data set. The amount of data included in the data set will be very small, and the data in the data set can be sorted from large to small, and the data with the highest sorting can be used as the data ranked Nth from small to large. In this way, the data ranked Nth from small to large can be quickly and accurately determined.

[0032] Of course, the above-mentioned method of extracting the data ranked Nth based on the benchmark value is only a preferred example for illustration, and those skilled in the art may also choose other processing methods as needed. For example, the record quantity condition may be that the total amount of the first business records belongs to a value interval, and the upper and lower limits of the value interval are N+n and Nn, respectively, where n is a relatively small positive integer, such as 1, 2, 3, etc. By setting the value interval, the business data extraction range can be locked more quickly and the number of iterations can be reduced. Accordingly, the data ranked nth in the order from front to back in the extracted data set can be used as the data ranked Nth from small to large in the entire business system.

[0033] Based on the above scenario example, a business data processing method is provided in an embodiment of this specification, and the method is applied to the management node of the distributed business system. Figure 1 As shown, the method may include the following steps.

[0034] S20: receiving a data extraction request for a specified business parameter under an extraction index sent by a business terminal.

[0035] The business terminal may refer to a terminal device with business needs. The terminal device may include a computer or an intelligent business counter. Business personnel may initiate a data extraction request in the business terminal as needed. The business terminal may send the data extraction request to the management node of the distributed business system; or, the data extraction request may be sent to the management node of the distributed business system through other intermediate servers.

[0036] The specified business parameters may include business parameters such as deposit amount and interest rate that need to be processed by data statistics. The type of the specified business parameters may be configured according to actual needs and is not limited here. The extraction index may be, for example, extracting the median, querying the data value at a specified position, etc. For example, querying the data value ranked Nth from smallest to largest or querying the data value ranked Nth from largest to smallest.

[0037] For example, in order to better set the bank's deposit interest rate, business personnel need to analyze the median of each user's deposit amount within a certain time interval to analyze user contributions and deposit habits, etc., and establish a relevant deposit pricing model. The business terminal sends a data extraction request to the management node of the distributed business system, and the data extraction request may include extracting the median of the deposit amount within a certain time interval. Accordingly, the specified business parameter is the deposit amount within the certain time interval, and the extraction index is the median.

[0038] S22: When it is determined based on the extraction index that the data extraction request cannot be directly sent to each physical node for execution, obtain the first operation instruction and the second operation instruction corresponding to the data extraction request; wherein the first operation instruction refers to an operation instruction that can be directly sent to each physical node for execution; the second operation instruction refers to an operation instruction for calculating the indicator data corresponding to the extraction index based on the execution results fed back by each physical node.

[0039] After receiving the data extraction request, the management node may analyze the data extraction request to determine whether the data extraction request can be directly sent to each physical node for execution. For example, the management node may pre-configure the extraction indicators that can be directly executed by each physical node or the extraction indicators that cannot be directly executed. After receiving the data extraction request, the management node may compare the extraction indicators involved in the data extraction request based on the pre-configured extraction indicators that can be directly executed by each physical node or the extraction indicators that cannot be directly executed.

[0040] Of course, other methods may be used to determine whether the data extraction request can be directly sent to each physical node, such as analyzing the execution function corresponding to the data extraction request and determining whether it can be directly sent to each physical node based on the execution function.

[0041] After determining that the data extraction request cannot be directly sent to each physical node, the management node can further obtain the processing method corresponding to the data extraction request. The processing methods for different extraction indicators can be pre-configured in the management node so that the management node can accurately and quickly perform the data extraction operation after receiving the data extraction request. The processing method may include a first operation instruction and a second operation instruction. The first operation instruction may refer to an operation instruction that can be directly sent to each physical node for execution. The second operation instruction may refer to an operation instruction for the management node to calculate the indicator data corresponding to the extraction indicator based on the execution results fed back by each physical node.

[0042] As shown in the above scenario example, the first operation instruction may include an instruction to extract the node maximum value and node minimum value of the business data corresponding to the specified business parameter. The first operation instruction may also include an instruction to extract the first node business record amount whose business data of the specified business parameter is less than the reference value. Of course, the first operation instruction may also include other forms as needed.

[0043] The second operation instruction may include comparing the node maximum value and node minimum value fed back by each physical node to determine the maximum value and minimum value of the business data of the specified business parameter in the distributed business system. The second operation instruction may also include calculating the total amount of first business records of the specified business parameter in the distributed business system that is less than the reference value based on the amount of the first node business records fed back by each physical node. Of course, the second operation instruction may also include other forms as needed.

[0044] As shown in the above scenario example, each first operation instruction and second operation instruction may have a certain execution order and reference relationship between parameter output values. Accordingly, the management node can also pre-configure the execution order of each first operation instruction and second operation instruction corresponding to the extraction index and the reference relationship between parameter output values.

[0045] The management node can also pre-configure commonly used first operation instructions and second operation instructions, and configure different instruction identifiers. Then, the instruction identifiers of the operation instructions required for different extraction indicators can be stored in sequence and associated with the extraction indicators. After receiving the data extraction request, the management node can obtain the instruction identification sequence corresponding to the extraction indicator contained in the data extraction request, and then, based on the instruction identification sequence, sequentially call the corresponding first operation instruction and the second operation instruction to perform the corresponding operation to obtain the final execution result.

[0046] By decomposing the processing methods of different extraction indicators into operation instructions that can be directly sent to physical nodes for execution, and operation instructions that are further processed based on the execution results of physical nodes to determine the indicator data corresponding to the extraction indicators, it is possible to effectively implement distributed processing of data processing requests that cannot be directly executed by physical nodes, avoid performance problems and unstable factors caused by data migration, and ensure the stable operation of the distributed system during the processing process. At the same time, it is also possible to achieve fast and accurate processing of data extraction requests.

[0047] S24: Sending a first operation instruction to each physical node, and receiving an execution result fed back by each physical node.

[0048] After determining the operation instruction and execution order corresponding to the data processing request, the management node may issue a first operation instruction to each physical node. After receiving the first operation instruction issued by the management node, each physical node may perform a corresponding operation based on the first operation instruction, such as extracting the node maximum value and node minimum value of the business data corresponding to the specified business parameter under the physical node. After the execution is completed, each physical node may feed back the execution result to the management node.

[0049] S26: Processing the execution results fed back by each physical node based on the second operation instruction to obtain indicator data of the data extraction request.

[0050] The management node may receive the execution results fed back by each physical node, and process the execution results fed back by each physical node based on the second operation instruction, such as calculating the maximum value and minimum value of the specified service parameter in the distributed service system based on the maximum value and minimum value of the node fed back by each physical node.

[0051] In the case that the data extraction request includes more than two first operation instructions or second operation instructions, the above steps S24 and S26 can be repeated based on the execution order until all operation instructions are executed to obtain the indicator data of the data extraction request.

[0052] S28: Feedback the indicator data to the business terminal.

[0053] The management node may feed back the indicator data to the business terminal. After receiving the indicator data, the business terminal may display the indicator data to the business personnel. For example, the value of the extracted Nth data may be displayed on the display of the business terminal. Alternatively, the business terminal may perform other analysis and processing based on the indicator data.

[0054] If the indicator data of multiple business parameters under different extraction indicators are counted, the extraction results can also be associated with the business parameters and the extraction indicators and stored. After all extractions are completed, the extraction results are fed back to the business terminal, and the business terminal can display them to the user.

[0055] Through the above method, after receiving the data extraction request, the management node first determines whether the data extraction request can be directly executed by the physical node. If it cannot be directly executed, it can be processed based on the pre-configured processing method. The processing method can at least include an executable operation instruction sent to the physical node for execution, and an operation instruction for further processing based on the execution result of the physical node. Based on this processing method, the distributed processing of the data processing request is realized, avoiding performance problems and unstable factors caused by data relocation, and ensuring the stable operation of the distributed system during the processing process. At the same time, it can also realize the rapid and accurate processing of data extraction requests.

[0056] The solution of the above embodiment can also be applied to the extraction of index data of various extraction indicators. In a scenario example, this specification also provides a method for extracting the median of business data based on the solution of the above embodiment.

[0057] The management node may first send instructions to each physical node to extract the node maximum value, node minimum value and second node business record volume of the business data corresponding to the specified business parameter. The second node business record volume may refer to the number of business records corresponding to the full amount of business records of the specified business parameter under the physical node.

[0058] The management node can calculate the total amount of second business records of the specified business parameter in the distributed business system based on the amount of second node business records fed back by each physical node, and calculate half of the total amount of business records, and discard the decimal places of the calculation result as the integer obtained as the reference value corresponding to the extraction index.

[0059] The management node may also sort the node maximum value and node minimum value based on the node maximum value and node minimum value fed back by each physical node, determine the maximum value and minimum value of the specified service parameter in the distributed service system, and use the average value of the maximum value and the minimum value as the initial reference value.

[0060] The management node may further issue an instruction to each physical node to extract the amount of first-node business records whose business data of the specified business parameter is less than / greater than the reference value, and calculate the total amount of first-node business records whose specified business parameter is less than / greater than the reference value in the distributed business system based on the amount of first-node business records fed back by each physical node.

[0061] If the business data extracted is less than the reference value for the specified business parameter, then if the total amount of the first business record is less than the reference value, it can be explained that the reference value is smaller than the median, and the minimum value can be updated using the reference value, keeping the maximum value unchanged, and obtaining the updated maximum value and minimum value. Then, the average of the updated maximum value and minimum value can be calculated as the updated reference value.

[0062] If the total amount of the first business record is greater than the reference value, it can be explained that the reference value is larger than the median, and the maximum value can be updated using the reference value, while the minimum value remains unchanged, to obtain the updated maximum value and minimum value. Then, the average of the updated maximum value and minimum value can be calculated as the updated reference value.

[0063] The management node may issue an instruction to each physical node to extract the first node business record quantity whose business data of the specified business parameter is less than the updated reference value. Based on the first node business record quantity fed back by each physical node, the total first business record quantity whose specified business parameter is less than the updated reference value in the distributed business system is calculated.

[0064] The management node can then determine whether the total amount of the first business records of the updated benchmark value is equal to the reference value. If not, repeat the above updating steps until the total amount of the first business records of the updated benchmark value is equal to the reference value. Output the last updated benchmark value.

[0065] In the above iteration process, there may be a situation where the total amount of the second business records is an odd number, which may lead to an infinite loop. You can also configure the difference between the total amount of the first business records and the reference value to be equal to 1 and the difference between the maximum value and the minimum value to be less than the specified precision value as a judgment condition. If this condition is met, the above iteration process is jumped out and the last updated benchmark value is output.

[0066] Of course, if the business data extracted is less than the reference value, if the total amount of the first business record is less than the reference value, it can be said that the reference value is larger than the median, and the reference value can be used to update the maximum value, keeping the minimum value unchanged. If the total amount of the first business record is less than the reference value, it can be said that the reference value is smaller than the median, and the reference value can be used to update the minimum value, keeping the maximum value unchanged. Thus, the update of the reference value is completed. Other implementation schemes refer to the above implementation scheme until the iteration ends and the last updated reference value is output.

[0067] The management node can determine the median based on the last updated benchmark value. For example, the benchmark value can be used as the median. Alternatively, the maximum and minimum values ​​of the last update can be extracted, and then the data between the maximum and minimum values ​​are extracted from each physical node to form a data set. If the data in the data set is an odd number, the number in the middle is used as the median. If the data in the data set is an even number, the average of the two numbers in the middle is used as the median.

[0068] With the lifting of the deposit interest rate ceiling of financial institutions, the accuracy and speed of deposit interest rates have a great impact on the deposit business of financial institutions. To address this issue, a relevant deposit pricing model can be established by analyzing the user's deposit contribution and deposit habits, so as to better determine the deposit interest rate for different deposit amounts. A very important parameter in the deposit pricing model is the median user contribution. The user contribution value can be calculated by the business system through other calculation models, associated with the user identifier, and stored in a distributed database in a decentralized manner. The scheme provided in the above embodiment can be used to quickly extract the median of the user contribution value from the distributed database. Practice shows that the above processing method is applied to the MPPDB distributed database to obtain the median of nearly 1 billion customer contribution values, and the result is obtained after running for only 10 minutes. In contrast, the median is extracted by running in a centralized manner on a physical node, and no result is output after running for 2 hours.

[0069] Based on the above scenario examples, in some embodiments of this specification, when the extraction indicator is to extract the median or query the data value at a specified position, the method further includes the following steps.

[0070] S40: Sending an instruction to each physical node to extract the node maximum value and the node minimum value of the service data corresponding to the specified service parameter.

[0071] S42: Calculate the maximum value and the minimum value of the designated service parameter in the distributed service system based on the node maximum value and the node minimum value fed back by each physical node.

[0072] S44: Using the average of the maximum value and the minimum value as a reference value, an instruction is issued to each physical node to extract the first node service record amount whose service data of the specified service parameter is less than / greater than the reference value.

[0073] S46: Based on the first node service record quantity fed back by each physical node, calculate the total first service record quantity of the designated service parameter being less than / greater than the reference value in the distributed service system.

[0074] S48: When the total amount of the first business records meets the record amount condition corresponding to the extraction index, determine the index data of the designated business parameter under the extraction index based on the reference value.

[0075] Among them, the record quantity condition may include that the total amount of the first business records is equal to the reference value, or the difference between the total amount of the first business records less than the benchmark value and the total amount of the first business records greater than the benchmark value is equal to 1 and the difference between the maximum value and the minimum value is less than the specified accuracy value.

[0076] Among them, when the extraction index is to extract the median, the reference value is determined in the following manner: an instruction is sent to each physical node to extract the amount of business records corresponding to all business records of the specified business parameter under the corresponding physical node, and the extracted amount of business records is used as the second-node business record amount; based on the second-node business record amount fed back by each physical node, the total amount of second business records of the specified business parameter in the distributed business system is calculated; one half of the total amount of the second business records is calculated, and the integer obtained by discarding the decimal places in the calculation result is used as the reference value corresponding to the extraction index.

[0077] S410: When the total amount of the first business records does not satisfy the record amount condition corresponding to the extraction index, the following iterative steps S412 to S418 are executed.

[0078] S412: Using the reference value to update the maximum value or the minimum value, and taking the average value of the updated minimum value and the updated maximum value as the updated reference value.

[0079] S414: Send an instruction to each physical node to extract the first node service record amount whose service data of the specified service parameter is less than / greater than the updated reference value.

[0080] S416: Based on the first node service record quantity fed back by each physical node, calculate the total first service record quantity for which the designated service parameter is less than / greater than the updated reference value in the distributed service system.

[0081] S418: When the total amount of the first business records does not satisfy the record amount condition corresponding to the extraction index, repeat the above updating steps S412 to S416 until the total amount of the first business records satisfies the record amount condition corresponding to the extraction index.

[0082] S420: Determine the indicator data of the specified business parameter under the extracted indicator based on the reference value updated in the last step.

[0083] Through the above method, the performance problems and unstable factors caused by data migration in the process of extracting the median or querying the data value of the specified position of the distributed business system are avoided, and the stable operation of the distributed system during the processing is guaranteed. At the same time, the data extraction request can be processed quickly and accurately.

[0084] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. For details, please refer to the description of the above-mentioned related processing related embodiments, and no further description is given here.

[0085] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0086] Based on the business data processing method described above, one or more embodiments of this specification also provide a business data processing device. The device may include a system, software (application), module, component, server, etc. that uses the method described in the embodiments of this specification and is combined with necessary implementation hardware. Based on the same innovative concept, the device in one or more embodiments provided in the embodiments of this specification is as described in the following embodiments. Since the implementation scheme of the device to solve the problem is similar to the method, the implementation of the specific device in the embodiments of this specification can refer to the implementation of the aforementioned method, and the repetitions will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived. Specifically, Figure 2 A schematic diagram of the module structure of an embodiment of a business data processing device provided in the specification is shown as follows: Figure 2 As shown, the device is applied to a management node of a distributed business system and may include the following modules.

[0087] The request receiving module 102 may be configured to receive a data extraction request for a specified service parameter under an extraction index sent by a service terminal.

[0088] The instruction acquisition module 104 can be used to obtain the first operation instruction and the second operation instruction corresponding to the data extraction request when it is determined based on the extraction index that the data extraction request cannot be directly sent to each physical node for execution; wherein the first operation instruction refers to an operation instruction that can be directly sent to each physical node for execution; the second operation instruction refers to an operation instruction that calculates the indicator data corresponding to the extraction indicator based on the execution results fed back by each physical node.

[0089] The instruction issuing module 106 may be configured to issue a first operation instruction to each physical node.

[0090] The result receiving module 108 may be used to receive the execution result fed back by each physical node.

[0091] The instruction execution module 110 can be used to process the execution results fed back by each physical node based on the second operation instruction to obtain the indicator data of the data extraction request.

[0092] The result feedback module 112 may be used to feed back the indicator data to the service terminal.

[0093] In some other embodiments, the extracting indicator may include extracting the median or querying the data value at a specified position.

[0094] The instruction issuing module 106 may also be used to issue instructions for extracting node maximum values ​​and node minimum values ​​of the service data corresponding to the specified service parameters to each physical node.

[0095] The instruction execution module 110 may also be used to calculate the maximum value and the minimum value of the specified service parameter in the distributed service system based on the node maximum value and the node minimum value fed back by each physical node.

[0096] The instruction issuing module 106 may also be used to issue an instruction to each physical node to extract the first node business record quantity whose business data of the specified business parameter is less than / greater than the reference value, using the average value of the maximum value and the minimum value as the reference value.

[0097] The instruction execution module 110 may also be used to calculate the total amount of first business records for which the specified business parameter is less than / greater than the reference value in the distributed business system based on the amount of the first node business records fed back by each physical node.

[0098] The instruction execution module 110 may also be configured to determine the index data of the designated business parameter under the extraction index based on the reference value when the total amount of the first business records satisfies the record amount condition corresponding to the extraction index.

[0099] It should be noted that the above-mentioned device may also include other implementations according to the description of the method embodiment. The specific implementation methods can refer to the description of the relevant method embodiment, and will not be described one by one here.

[0100] This specification also provides a business data processing device, which may include at least one processor and a memory for storing processor executable instructions, and when the instructions are executed by the processor, the steps of the method described in any one or more of the above embodiments are implemented.

[0101] The memory may include a physical device for storing information, which is usually to digitize the information and then store it in a medium using electrical, magnetic or optical means. The storage medium may include: a device that uses electrical energy to store information, such as various memories, such as RAM, ROM, etc.; a device that uses magnetic energy to store information, such as a hard disk, a floppy disk, a magnetic tape, a magnetic core memory, a magnetic bubble memory, a USB flash drive; a device that uses optical means to store information, such as a CD or a DVD. Of course, there are other types of readable storage media, such as quantum memory, graphene memory, etc.

[0102] It should be noted that the above-mentioned device may also include other implementation methods according to the description of the method or device embodiment. The specific implementation method can refer to the description of the relevant method embodiment, which will not be described one by one here.

[0103] It should be noted that the embodiments of this specification are not limited to the situations that must comply with the standard data model / template or the embodiments of this specification. Certain industry standards or slightly modified implementation plans based on the implementation described in the custom method or the embodiment can also achieve the same, equivalent or similar, or predictable implementation effects after deformation of the above-mentioned embodiments. The embodiments obtained by using these modified or deformed data acquisition, storage, judgment, processing methods, etc. can still fall within the scope of the optional implementation plans of this specification.

[0104] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In the description of this specification, the description of the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, in the absence of contradiction, a person skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0105] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A business data processing method, It is characterized in that Applied to a management node of a distributed business system, the method comprises: Receiving a data extraction request for a specified business parameter under an extraction indicator sent by a business terminal; In the case where it is determined based on the extraction index that the data extraction request cannot be directly sent to each physical node for execution, obtaining a first operation instruction and a second operation instruction corresponding to the data extraction request; wherein the first operation instruction refers to an operation instruction that can be directly sent to each physical node for execution; and the second operation instruction refers to an operation instruction that calculates the indicator data corresponding to the extraction indicator based on the execution results fed back by each physical node; A first operation instruction is issued to each physical node, and an execution result fed back by each physical node is received; including: issuing an instruction to each physical node to extract the node maximum value and node minimum value of the business data corresponding to the specified business parameter; calculating the maximum value and minimum value of the specified business parameter in the distributed business system based on the node maximum value and node minimum value fed back by each physical node; using the average value of the maximum value and the minimum value as a reference value, issuing an instruction to each physical node to extract the first node business record amount whose business data of the specified business parameter is less than / greater than the reference value; Processing the execution results fed back by each physical node based on the second operation instruction to obtain the index data of the data extraction request; including: calculating the total amount of first business records of the specified business parameter less than / greater than the reference value in the distributed business system based on the amount of the first node business records fed back by each physical node; determining the index data of the specified business parameter under the extraction index based on the reference value when the total amount of the first business records meets the record amount condition corresponding to the extraction index; The indicator data is fed back to the service terminal.

2. The method according to claim 1, It is characterized in that The extracting index includes extracting the median or querying the data value at a specified position.

3. The method according to claim 2, It is characterized in that The record quantity condition includes that the total amount of the first business records is equal to the reference value, or the difference between the total amount of the first business records less than the reference value and the total amount of the first business records greater than the reference value is equal to 1 and the difference between the maximum value and the minimum value is less than the specified accuracy value.

4. The method according to claim 3, It is characterized in that In a case where the total amount of the first business records does not satisfy a record amount condition corresponding to the extraction index, the method further includes: Using the reference value to update the maximum value or the minimum value, and taking the average of the updated minimum value and the updated maximum value as the updated reference value; Sending an instruction to each physical node to extract the first node service record amount whose service data of the specified service parameter is less than / greater than the updated reference value; Based on the first node service record quantity fed back by each physical node, calculate the total amount of first service records for which the specified service parameter is less than / greater than the updated reference value in the distributed service system; In the case where the total amount of the first business records does not satisfy the record amount condition corresponding to the extraction index, repeating the above updating step until the total amount of the first business records satisfies the record amount condition corresponding to the extraction index; The indicator data of the designated business parameter under the extracted indicator is determined based on the reference value updated in the last step.

5. The method according to claim 3, It is characterized in that The extraction index is the extraction median, and the reference value is determined in the following manner: Sending an instruction to each physical node to extract the service record volume corresponding to all service records of the specified service parameter under the corresponding physical node, and using the extracted service record volume as the service record volume of the second node; Calculate the total amount of second service records of the specified service parameter in the distributed service system based on the amount of the second node service records fed back by each physical node; Calculate one-half of the total amount of the second business records, and use the integer obtained by discarding decimal places from the calculation result as the reference value corresponding to the extraction index.

6. A business data processing device, It is characterized in that A management node applied to a distributed service system, the device comprising: A request receiving module, used for receiving a data extraction request for a specified business parameter under an extraction index sent by a business terminal; An instruction acquisition module, used for acquiring a first operation instruction and a second operation instruction corresponding to the data extraction request when it is determined based on the extraction index that the data extraction request cannot be directly sent to each physical node for execution; wherein the first operation instruction refers to an operation instruction that can be directly sent to each physical node for execution; and the second operation instruction refers to an operation instruction that calculates the indicator data corresponding to the extraction indicator based on the execution results fed back by each physical node; An instruction issuing module, used for issuing a first operation instruction to each physical node; A result receiving module, used to receive the execution results fed back by each physical node; An instruction execution module, used for processing the execution result fed back by each physical node based on the second operation instruction to obtain the indicator data of the data extraction request; A result feedback module, used to feed back the indicator data to the business terminal; Among them, the instruction issuing module is also used to issue instructions to each physical node for extracting the node maximum value and node minimum value of the business data corresponding to the specified business parameter; the instruction execution module is also used to calculate the maximum value and minimum value of the specified business parameter in the distributed business system based on the node maximum value and node minimum value fed back by each physical node; the instruction issuing module is also used to issue instructions to each physical node for extracting the first node business record amount of the business data of the specified business parameter which is less than / greater than the reference value, using the average value of the maximum value and the minimum value as the reference value; the instruction execution module is also used to calculate the total amount of the first business records of the specified business parameter which is less than / greater than the reference value in the distributed business system based on the first node business record amount fed back by each physical node; the instruction execution module is also used to determine the indicator data of the specified business parameter under the extraction indicator based on the reference value when the total amount of the first business records meets the record amount condition corresponding to the extraction indicator.

7. The device according to claim 6, It is characterized in that The extracting index includes extracting the median or querying the data value at a specified position.

8. A business data processing device, It is characterized in that The device is applied to a management node of a distributed business system, and includes at least one processor and a memory for storing processor executable instructions, and when the instructions are executed by the processor, the steps of the method described in any one of claims 1 to 5 are implemented.

Images