Method, device and system for adjusting distribution of computing operator types in computing system

Abstract

The present application discloses a method, device and system for adjusting the distribution of computing operator types in a computing system, wherein the method includes: obtaining a data record of the computing amount of computing operators in the actual data processing process of the computing system, predicting the predicted demand for computing operators on the computing system in the future time period T, and then adjusting the types and distribution ratios of computing operators implemented in programmable hardware on the computing system before the arrival of the future time period T according to the predicted demand for computing operators on the computing system. By adopting the method for adjusting the distribution of computing operator types in a computing system described in the present application, the computing power of computing operators implemented in programmable hardware can be used to serve the busiest data processing tasks as much as possible, thereby improving the computing efficiency of the entire computing system.

Description

Technical Field

[0001] The present application relates to the field of intelligent control, and more particularly to a method for adjusting the distribution of computing operator types in a computing system, and also to a device and system for adjusting the distribution of computing operator types in a computing system. Background Art

[0002] In recent years, with the rapid development of network technology, the storage and management scale of database data is growing exponentially, and users are sending more and more data processing requests to databases through applications. How to speed up data processing and improve the operating efficiency of database systems has become a common technical goal pursued by technical personnel in this field.

[0003] In order to solve the above technical problems, the existing technology usually adopts the method of integrating CPU (Central Processing Unit) + hardware accelerator (which can be one or more combinations of GPU, FPGA, ASIC chips) into an integrated heterogeneous hardware computing system to perform data processing tasks in the database. Through the combination of different hardware, each can play its own strengths and effectively improve the operating efficiency of the database system. Among them, the CPU provides computing power by executing the instructions provided to them. Depending on the instructions, they can perform different tasks at any time. The GPU provides the basic structure of multi-core parallel computing, which can support the acceleration of parallel computing tasks of large amounts of data; while the FPGA has a different working mode from the CPU and GPU.

[0004] FPGA (Field Programmable Gate Array) is a semi-customized circuit in the field of Application-Specific Integrated Circuit (ASIC). It needs to be programmed before use so that its specific units can perform specific tasks. Once the programming is completed, the tasks that the FPGA can handle are fixed until the next programming changes its role. Because FPGA has higher integration density, larger capacity, and lower power consumption in the process of performing specific tasks, it is often used as a hardware-level accelerator by some database manufacturers to undertake the work of computing operators with high computing requirements in computing systems.

[0005] However, when using FPGA and similar programmable hardware that performs fixed tasks for acceleration, the existing technology can only implement a certain part of the CPU's high-load computing operators into the FPGA based on the designer's own estimation. This implementation method is usually not accurate enough, resulting in computing operators with high computing power requirements not getting enough computing power support, while computing operators with low computing power requirements occupy computing power resources for a long time, resulting in poor acceleration of the entire computing system, unable to meet the requirements for efficient operation of the database system, and reducing hardware utilization. Summary of the invention

[0006] The present application provides a method for adjusting the distribution of computing operator types in a computing system to solve the problem of low computing efficiency caused by manually configuring the distribution of computing operator types in the prior art. The present application also provides a device and system for adjusting the distribution of computing operator types in a computing system.

[0007] The present application provides a method for adjusting the distribution of computing operator types in a computing system, including: obtaining a data record of the amount of computation of a computing operator in actual data processing of the computing system, wherein the computing operator is used to complete a data processing task; obtaining a predicted demand for the computing operator on the computing system in a future time period T based on the data record of the computing operator; and adjusting the computing operator implemented in a programmable hardware manner on the computing system before the future time period T arrives based on the predicted demand for the computing operator on the computing system, wherein the adjustment includes at least one of the following: the type of the computing operator implemented in the programmable hardware manner, and the distribution ratio of the computing operator implemented in the programmable hardware manner.

[0008] Optionally, the adjusting of the computing operators implemented in programmable hardware on the computing system before the future time period T arrives according to the predicted demand for the computing operators on the computing system specifically includes: determining, according to the predicted demand for the computing operators on the computing system, the computing operators on the computing system that reach or exceed a preset demand threshold; determining the type of the computing operators that reach or exceed the preset demand threshold as the type of computing operators that need to be implemented in programmable hardware on the computing system before the future time period T arrives; and adjusting, according to the type of computing operators that need to be implemented in programmable hardware on the computing system, the type of computing operators implemented in programmable hardware preset on the computing system before the future time period T arrives.

[0009] Optionally, the type of computing operator implemented in programmable hardware on the computing system as needed is adjusted before the future time period T arrives, specifically including: comparing the type of computing operator required to be implemented in programmable hardware on the computing system with the actual type of computing operator implemented in programmable hardware preset on the computing system, and determining whether there is a corresponding type of computing operator on the computing system; if not, adding the corresponding type of computing operator implemented in programmable hardware on the computing system before the future time period T arrives.

[0010] Optionally, the adjusting of the computing operators implemented in programmable hardware on the computing system before the future time period T arrives based on the predicted demand for the computing operators on the computing system specifically includes: determining, based on the predicted demand for the computing operators on the computing system, the computing operators among the computing operators on the computing system that reach or exceed a preset demand threshold; determining the predicted demand for the computing operators that reach or exceed the preset demand threshold as the demand for the computing operators implemented in programmable hardware on the computing system before the future time period T arrives; and adjusting, based on the demand for the computing operators implemented in programmable hardware on the computing system, the distribution ratio of the computing operators implemented in programmable hardware preset on the computing system before the future time period T arrives.

[0011] Optionally, the distribution ratio of computing operators implemented in programmable hardware on the computing system that is preset before the future time period T arrives is adjusted according to the demand for computing operators implemented in programmable hardware on the computing system, specifically including: when the demand for computing operators implemented in programmable hardware on the computing system decreases, reducing the distribution ratio of corresponding types of computing operators implemented in programmable hardware before the future time period T arrives; when the demand for computing operators implemented in programmable hardware on the computing system increases, increasing the distribution ratio of corresponding types of computing operators implemented in programmable hardware before the future time period T arrives; when the demand for computing operators implemented in programmable hardware on the computing system is stable within a preset interval, maintaining the distribution ratio of corresponding types of computing operators implemented in programmable hardware on the computing system unchanged.

[0012] Optionally, the method of adjusting the distribution ratio of computing operators implemented in programmable hardware on the computing system before the future time period T arrives based on the demand for computing operators implemented in programmable hardware on the computing system specifically includes: comparing the demand for computing operators implemented in programmable hardware on the computing system with the distribution of computing operators implemented in programmable hardware on the computing system; if the degree of match is lower than a preset indicator, then adjusting the distribution ratio of corresponding types of computing operators implemented in programmable hardware on the computing system before the future time period T arrives.

[0013] Optionally, obtaining the predicted demand for the computing operator on the computing system in a future time period T based on the computing operator's computing volume data record includes: inputting the computing operator's computing volume data record into a preset computing operator demand prediction model to obtain the predicted demand for the computing operator on the computing system in the future time period T, the computing operator demand prediction model being a deep neural network model that predicts the predicted demand for the computing operator in a future time period T based on the computing operator's computing volume data record within a predetermined time period.

[0014] Optionally, the method for adjusting the distribution of computing operator types in the computing system also includes: obtaining a sample of computing operator operation data records within the predetermined time period; obtaining a sample of actual demand for the computing operator in a future time period T that matches the sample of computing operator operation data records within the predetermined time period; using the sample of computing operator operation data records within the predetermined time period and the sample of actual demand for the computing operator in the future time period T that matches the sample of computing operator operation data records within the predetermined time period to train the deep neural network model and obtain the computing operator demand prediction model.

[0015] Optionally, obtaining the computational volume data record of the computing operators in the actual data processing of the computing system specifically includes: obtaining the computational volume data record of the computing operators in the computing system used to process the data processing requests of the client in the database; correspondingly, the computing operators are all special computing programs set on the computing system in the database for data processing tasks.

[0016] Optionally, the computing system is a heterogeneous hardware accelerated computing system composed of a CPU module and a programmable hardware module for processing database data.

[0017] Optionally, the calculation amount data record of the calculation operator includes at least one of occupancy ratio data and usage time data of the calculation operator during data processing.

[0018] Optionally, the predicted demand of the computing operator is the predicted time consumed by the computing operator during data processing.

[0019] Correspondingly, the present application also provides a device for adjusting the distribution of computing operator types in a computing system, comprising: a monitoring unit, a resource manager unit and a programmable hardware management unit; the monitoring unit is used to obtain a data record of the amount of computation of the computing operator in actual data processing of the computing system, and the computing operator is used to complete a data processing task; the resource manager unit is used to obtain a predicted demand for the computing operator on the computing system in a future time period T based on the data record of the computing operator; the programmable hardware management unit is used to adjust the computing operator implemented in programmable hardware on the computing system before the future time period T arrives based on the predicted demand for the computing operator on the computing system, and the adjustment includes at least one of the following: the type of the computing operator implemented in the programmable hardware, and the distribution ratio of the computing operator implemented in the programmable hardware.

[0020] Optionally, the programmable hardware management unit is specifically used to: determine the computing operators on the computing system that reach or exceed a preset demand threshold based on the predicted demand for the computing operators on the computing system; determine the type of the computing operator that reaches or exceeds the preset demand threshold as the type of computing operator that needs to be implemented in programmable hardware on the computing system before the future time period T arrives; and adjust the type of computing operator implemented in programmable hardware preset on the computing system before the future time period T arrives based on the type of computing operator that needs to be implemented in programmable hardware on the computing system.

[0021] Optionally, the programmable hardware management unit is specifically used to: compare the type of computing operator that needs to be implemented in programmable hardware on the computing system with the actual type of computing operator implemented in programmable hardware preset on the computing system, and determine whether there is a computing operator of the corresponding type on the computing system; if not, add the type of computing operator of the corresponding type implemented in programmable hardware on the computing system before the future time period T arrives.

[0022] Optionally, the programmable hardware management unit is specifically used to: determine the computing operators on the computing system that reach or exceed a preset demand threshold based on the predicted demand for the computing operators on the computing system; determine the predicted demand for the computing operators that reach or exceed the preset demand threshold as the demand for the computing operators implemented in programmable hardware on the computing system before the future time period T arrives; and adjust the distribution ratio of the computing operators implemented in programmable hardware preset on the computing system before the future time period T arrives based on the demand for the computing operators implemented in programmable hardware on the computing system.

[0023] Optionally, the programmable hardware management unit is specifically used to: when the demand for computing operators implemented by programmable hardware on the computing system decreases, reduce the distribution ratio of the corresponding type of computing operators implemented by programmable hardware before the future time period T arrives; when the demand for computing operators implemented by programmable hardware on the computing system increases, increase the distribution ratio of the corresponding type of computing operators implemented by programmable hardware before the future time period T arrives; when the demand for computing operators implemented by programmable hardware on the computing system stabilizes within a preset range, the distribution ratio of the corresponding type of computing operators implemented by programmable hardware on the computing system remains unchanged.

[0024] Optionally, the programmable hardware management unit is specifically used to compare the demand for computing operators implemented in programmable hardware on the computing system with the distribution of computing operators implemented in programmable hardware preset on the computing system; if the degree of match is lower than a preset indicator, then adjust the distribution ratio of the corresponding type of computing operators implemented in programmable hardware on the computing system before the future time period T arrives.

[0025] Optionally, the resource manager unit is specifically used to: input the computing volume data record of the computing operator into a preset computing operator demand prediction model to obtain the predicted demand for the computing operator on the computing system in the future time period T, and the computing operator demand prediction model is a deep neural network model that predicts the predicted demand for the computing operator in the future time period T based on the computing volume data record of the computing operator in a predetermined time period.

[0026] Optionally, the device for adjusting the distribution of computing operator types in the computing system also includes: an operation volume data record sample acquisition unit, an actual demand sample acquisition unit, and a training unit; the operation volume data record sample acquisition unit is used to obtain the operation volume data record samples of the computing operators within the predetermined time period; the actual demand sample acquisition unit is used to obtain the actual demand samples for the computing operators in the future time period T that match the operation volume data record samples of the computing operators within the predetermined time period; the training unit is used to train the deep neural network model using the operation volume data record samples of the computing operators within the predetermined time period and the actual demand samples for the computing operators in the future time period T that match the operation volume data record samples of the computing operators within the predetermined time period, so as to obtain the computing operator demand prediction model.

[0027] Optionally, the monitoring unit is specifically used to: obtain the computational volume data records of the computing operators in the computing system used to process the data processing requests of the client in the database; correspondingly, the computing operators are all special computing programs set for data processing tasks on the computing system in the database.

[0028] Optionally, the computing system is a heterogeneous hardware accelerated computing system composed of a CPU module and a programmable hardware module for processing database data.

[0029] Optionally, the calculation amount data record of the calculation operator includes at least one of occupancy ratio data and usage time data of the calculation operator during data processing.

[0030] Optionally, the predicted demand quantity of the computing operator is the predicted demand time of the computing operator during data processing.

[0031] Correspondingly, the present application also provides a system for adjusting the distribution of computing operator types in a computing system, including: an apparatus for adjusting the distribution of computing operator types in a computing system as described in any one of the above.

[0032] Correspondingly, the present application also provides an electronic device, comprising: a processor and a memory, wherein the memory is used to store a program for a method for adjusting the distribution of computing operator types in a computing system. After the device is powered on and runs the program for the method for adjusting the distribution of computing operator types in the computing system through the processor, the following steps are performed: obtaining a data record of the amount of calculation of the computing operator in actual data processing of the computing system, wherein the computing operator is used to complete a data processing task; obtaining a predicted demand for the computing operator on the computing system in a future time period T based on the data record of the computing operator; and adjusting the computing operator implemented in programmable hardware on the computing system before the future time period T arrives based on the predicted demand for the computing operator on the computing system, wherein the adjustment includes at least one of the following: the type of computing operator implemented in programmable hardware, and the distribution ratio of the computing operator implemented in programmable hardware.

[0033] Correspondingly, the present application also provides a storage device, storing a program of a method for adjusting the distribution of computing operator types in a computing system, the program is executed by a processor to perform the following steps: obtaining a data record of the amount of calculation of the computing operator in the actual data processing of the computing system, the computing operator is used to complete the data processing task; according to the data record of the amount of calculation of the computing operator, obtaining the predicted demand for the computing operator on the computing system in a future time period T; according to the predicted demand for the computing operator on the computing system, adjusting the computing operator implemented in programmable hardware on the computing system before the arrival of the future time period T, the adjustment including at least one of the following: the type of computing operator implemented in programmable hardware, the distribution ratio of the computing operator implemented in programmable hardware.

[0034] Correspondingly, the present application also provides a method for adjusting the distribution of computing operator types in a computing system, characterized in that it includes: obtaining a data record of the amount of computation of at least some types of computing operators in actual data processing of the computing system, wherein the computing operators are used to complete data processing tasks; obtaining a predicted demand for computing operators on the computing system in a future time period T based on the data record of the computing operators; and adjusting the computing operators implemented by hardware accelerators on the computing system before the future time period T arrives based on the predicted demand for computing operators on the computing system, wherein the adjustment includes at least one of the following: the type of computing operators implemented by the hardware accelerator, and the distribution ratio of the computing operators implemented by the hardware accelerator.

[0035] Compared with the prior art, this application has the following advantages:

[0036] By adopting the method for adjusting the distribution of computing operator types in the computing system provided in the present application, it is possible to obtain the predicted demand for computing operators in future time periods by analyzing the data records of the computing operators' computing volume during the data processing process, and then accurately adjust the types and distribution ratios of the computing operators implemented in programmable hardware on the computing system, so that the computing power of the programmable hardware can be used as much as possible to serve the busiest data processing tasks, thereby improving the computing efficiency of the entire computing system and improving the efficiency of hardware utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A flowchart of a method for adjusting the distribution of computing operator types in a computing system provided by an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of a device for adjusting the distribution of computing operator types in a computing system provided by an embodiment of the present invention;

[0039] Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present invention;

[0040] Figure 4 A flowchart of an application scenario of a computing system provided by an embodiment of the present invention;

[0041] Figure 5a A schematic diagram of a computing operator type distribution before adjustment in a computing system provided by an embodiment of the present invention;

[0042] Figure 5b A schematic diagram of the distribution of computing operator types after adjustment in a computing system provided by an embodiment of the present invention;

[0043] Figure 6 A schematic diagram of a monitoring method for computing operator computation data on a computing system according to an embodiment of the present invention;

[0044] Figure 7a A schematic diagram of adjusting the distribution ratio of computing operator types before adjustment in a computing system provided by an embodiment of the present invention;

[0045] Figure 7b A schematic diagram of adjusting the distribution ratio of computing operator types in a computing system after adjustment provided by an embodiment of the present invention;

[0046] Figure 8 A complete operation flow chart of a system for adjusting the distribution of computing operator types in a computing system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific implementation disclosed below.

[0048] The following is a detailed description of an embodiment of the method for adjusting the distribution of computing operator types in a computing system provided by the present invention. This embodiment is described by taking a computing system for processing database management and query processing tasks as an example.

[0049] Please refer to Figure 1 As shown, it is a flowchart of a method for adjusting the distribution of computing operator types in a computing system provided by an embodiment of the present invention.

[0050] The embodiments of the present invention can be implemented based on a traditional database system that provides data storage and management services. The specific implementation process includes the following steps:

[0051] Step S101: obtaining a data record of the amount of computation of a computing operator in actual data processing of a computing system, wherein the computing operator is used to complete a data processing task.

[0052] In an embodiment of the present invention, the computing system may be a heterogeneous hardware accelerated computing system composed of a CPU module and a programmable hardware FPGA module for processing database data, and often also includes a GPU module.

[0053] The computing operators are used to complete data processing tasks. For different computing scenarios, computing operators may have different meanings. Among them, for the computing system for databases in this embodiment, it may include Filter Operator, Table Scan Operator, Hash Join Operator, Aggregation Operator, Sort Operator, and Project Operator in SQL processing tasks. Figure 4 The application scenario flow chart of a computing system shown in the figure, in the specific implementation process, the user sends a data processing request to the DS (Data Store) through the APP application, and the Load Balancer (load balancing module) implements the load balancing of the data processing request. The front-end node module (Front node Pool) receives the data processing request assigned by the load balancing module for analysis and processing, assigns it to different FNs (Front Nodes; front-end nodes), and generates a corresponding execution plan and sends it to the above CN (Compute node Pool; computing node), and then returns the corresponding data stored in the DS (DataStore; database). The execution plan is an AST (abstract syntax tree) composed of several computing operators in a set logical order.

[0054] In the process of executing the execution plan, the calculation operators will be respectively assigned to the CPU, GPU and the corresponding units in the FPGA module set as the corresponding calculation operators for execution. Since the types and distribution ratios of the preset calculation operators in the programmable hardware FPGA module are usually set in advance based on manual experience, and only the calculation tasks of specific calculation operators can be executed after setting. Therefore, in the process of accelerating the calculation of data processing, due to improper settings, the calculation operators with large actual calculation requirements may not get enough calculation power, while the operators with small actual calculation requirements occupy the calculation power resources of the FPGA module for a long time, resulting in low calculation efficiency of the entire calculation system.

[0055] To this end, the present invention provides a method for adjusting the distribution of computing operator types in a computing system. The premise of this method is to monitor and record the computing volume data of the computing operators in the actual data processing process of the entire computing system and perform statistical aggregation, so as to grasp the computing volume requirements of the computing system for each computing operator. Monitoring and recording the computing volume data of the computing operators can be carried out in different ways according to different needs. One possible way is to monitor the historical data of the computing volume of each computing operator of the computing system. This method mainly takes into account that the computing requirements in the computing system often have historical continuity, and historical data can be used to judge future computing requirements. The following introduces a specific implementation method of using this method.

[0056] like Figure 6 As shown, the figure is a schematic diagram of the computing amount data of a computing operator on a monitoring computing system provided by an embodiment of the present invention. Different types of computing operators are represented by different colors, including: Filter Operator, Table Scan Operator, Hash join Operator, Aggregation Operator, Sort Operator and Project Operator. By analyzing the computing amount data records monitored during the period from November 1st to November 7th, it can be found that among the most time-consuming types of computing operators, the demand for computing power of Filter Operator and Table Scan Operator is decreasing, while the computing power demand of Hash join Operator and Aggregation Operator is increasing, the computing power requirement of Sort Operator is relatively stable, and although the computing power demand of Project Operator fluctuates, the overall fluctuation is not large.

[0057] Step S102: Obtain the predicted demand for the computing operators on the computing system in the future time period T according to the computing amount data record of the computing operators.

[0058] The above step S101 obtains the data record of the amount of calculation of the calculation operator in the actual data processing of the calculation system within a predetermined time period. Data preparation work is done for predicting the predicted demand for the calculation operator on the calculation system in the future time period T in this step. In step S102, the predicted demand for the calculation operator on the calculation system in the future time period T can be obtained based on the data record of the amount of calculation of the calculation operator monitored above.

[0059] By continuously monitoring and collecting the trend of the load on the entire computing system during data processing, the computing operator whose predicted demand reaches or exceeds the preset demand threshold can be determined as a computing operator that requires the programmable hardware FPGA module to provide computing power support. The predicted demand of the computing operator monitored above is the predicted demand for computing volume, and the way to measure the computing volume demand, specifically, the computing time can be used as a measurement indicator, that is, the computing volume demand of a certain computing operator is the time-consuming time to execute the computing operator within the determined time period. The time to execute a computing operator includes the time for the CPU, GPU, FGPA or other programmable hardware to execute the computing operator. At the same time, quite a number of computing operators of this type can be executed at the same time, and for each executed computing operator, its running time is calculated and summed up. Of course, the computing volume demand of each computing operator can also be measured by using the proportion of the computing operator in the total computing operator time. The most time-consuming types of computing operators are determined as computing operators that require programmable hardware FPGA modules to provide computing power support.

[0060] For example, through the analysis of the computing volume data records monitored during the period from November 1 to November 7, it can be found that among the most time-consuming types of computing operators, the computing power requirements of Filter Operator and Table Scan Operator are decreasing, while the computing power requirements of Hash Join Operator and Aggregation Operator are increasing. The computing power requirements of Sort Operator are relatively stable, and although the computing power requirements of Project Operator fluctuate, the overall fluctuation is not large. Therefore, before the arrival of the future time period T, the demand trend for computing power can be obtained: 1) The computing power requirements of Hash join Operator and Aggregation Operator are constantly increasing, and the computing power needs to be replenished in time, otherwise the computing system will be busier and the computing efficiency will be lower. At this time, the distribution ratio of such computing operators in the programmable hardware FPGA module can be increased; 2) The computing power requirements of Filter Operator and Table Scan Operator are decreasing, so the distribution ratio of such computing operators in the programmable hardware FPGA module can be released, otherwise it may cause a waste of computing resources; 3) Sort Operator and Project Operator are within the normal fluctuation range, and there is no need to adjust the distribution ratio of such computing operators for the time being.

[0061] Specifically, according to the computing volume data records of the above-mentioned monitored computing operators, the predicted demand for computing operators on the computing system in the future time period T can be obtained, which can be achieved in the following way: first, obtain the computing volume data record samples of the computing operators within the predetermined time period, and the actual demand samples for the computing operators in the future time period T that match the computing volume data record samples of the computing operators in the predetermined time period; then use the computing volume data record samples of the computing operators in the predetermined time period, and the actual demand samples for the computing operators in the future time period T that match the computing volume data record samples of the computing operators in the predetermined time period to train a pre-set deep neural network model, and finally obtain a computing operator demand prediction model. The monitored computing operator computing volume data records are input into the computing operator demand prediction model in real time, and the predicted demand for the computing operators on the computing system in the future time period T can be accurately obtained. It should be noted that the computing operator demand prediction model is a deep neural network model that predicts the predicted demand for the computing operators in the future time period T based on the computing volume data records of the computing operators in the predetermined time period. It should be noted that the prediction model described in the present invention is not limited to the above-mentioned one obtained by collecting training samples to train a preset deep neural network model, and it can also be implemented by a logistic regression model. In the specific implementation process, a suitable model can be selected through performance comparison testing.

[0062] Step S103: According to the predicted demand for the computing operators on the computing system, the computing operators implemented in programmable hardware on the computing system are adjusted before the future time period T arrives, and the adjustment includes at least one of the following: the type of computing operators implemented in programmable hardware, and the distribution ratio of the computing operators implemented in programmable hardware.

[0063] After obtaining the predicted demand for computing operators on the computing system in the future time period T in the above step S102, this step can pre-adjust the type and distribution ratio of computing operators implemented in programmable hardware on the computing system before the future time period T arrives by analyzing the predicted demand.

[0064] Specifically, according to the predicted demand for computing operators on the computing system within a predetermined time period monitored in advance, the type and distribution ratio of computing operators implemented by programmable hardware can be accurately adjusted, which can be achieved in the following manner:

[0065] 1) According to the predicted demand for computing operators on the computing system, first determine the computing operators on the computing system that reach or exceed the preset demand threshold. The type of computing operators that reach or exceed the preset demand threshold is determined as the type of computing operators that need to be implemented in programmable hardware on the computing system before the future time period T arrives. According to the type of computing operators that need to be implemented in programmable hardware on the computing system, adjust the type of computing operators that are preset in programmable hardware on the computing system before the future time period T arrives. In the specific application process, the type of computing operator that needs to be implemented in programmable hardware on the computing system can be compared with the actual type of computing operator that is preset in programmable hardware on the computing system to determine whether there is a computing operator of the corresponding type on the computing system. If not, add the type of computing operator of the corresponding type that is implemented in programmable hardware on the computing system before the future time period T arrives.

[0066] Since the load of computing operators is usually constantly changing during the data processing process of the database system, not only do different types of computing operators have different requirements for computing power, but the same type of operators may also have different requirements for computing power in different time periods, that is, the computing power demand has the characteristics of temporal periodicity. Therefore, the load of the computing operator monitored within the above-mentioned determined time period may not always reach or exceed the preset load threshold. Therefore, it is sometimes necessary to adjust the type or distribution ratio of the computing operator in the FPGA module in real time. For example, Figure 5a and Figure 5b As shown in FIG. 1 , they are respectively a schematic diagram of the distribution of computing operator types before adjustment and a schematic diagram of the distribution of computing operator types after adjustment in a computing system provided by an embodiment of the present invention. In the previous execution plan, it is possible that Figure 5a The two corresponding Table Scan Operators need to be accelerated by the programmable hardware FPGA module; in the latter execution plan, due to the changes in the statistical information of different computing operators, the scheduling layer of the programmable hardware FPGA module automatically balances the types of its own computing operators. The Aggregation Operator and Join Operator with the highest actual load in the recent period are loaded into the FPGA module in a certain proportion, while one of the Table Scan Operators with lower computing power requirements is unloaded. Therefore, from the user's perspective, when the database system processes the same SQL data and executes it in different time periods, the FPGA module can dynamically adjust to provide different computing power support. It should be noted that the computing volume data record of the computing operator includes at least one of the occupancy ratio data and time-consuming time data of the computing operator during the data processing process.

[0067] 2) According to the predicted demand for the computing operators on the computing system, determine the computing operators on the computing system that reach or exceed the preset demand threshold. The predicted demand for the computing operators that reach or exceed the preset demand threshold is determined as the demand for the computing operators implemented in programmable hardware on the computing system before the future time period T arrives. According to the demand for the computing operators implemented in programmable hardware on the computing system, adjust the distribution ratio of the computing operators implemented in programmable hardware preset on the computing system before the future time period T arrives.

[0068] In a specific application process, one implementation method is: compare the demand for computing operators implemented in programmable hardware on the computing system with the distribution of computing operators implemented in programmable hardware preset on the computing system, and if the matching degree is lower than the preset index, adjust the distribution ratio of the corresponding type of computing operators implemented in programmable hardware on the computing system before the arrival of the future time period T. Another method is: when it is monitored that the demand for computing operators implemented in programmable hardware on the computing system is reduced, the distribution ratio of the corresponding type of computing operators implemented in programmable hardware is reduced before the arrival of the future time period T; when it is monitored that the demand for computing operators implemented in programmable hardware on the computing system is increased, the distribution ratio of the corresponding type of computing operators implemented in programmable hardware is increased before the arrival of the future time period T; when it is monitored that the demand for computing operators implemented in programmable hardware on the computing system is stable within the preset range, the distribution ratio of the corresponding type of computing operators implemented in programmable hardware on the computing system remains unchanged.

[0069] like Figure 7a and 7b As shown, they are respectively a schematic diagram of adjusting the distribution ratio of computing operator types in a computing system before adjustment and a schematic diagram of the distribution ratio of computing operator types after adjustment provided by an embodiment of the present invention.

[0070] By continuously monitoring and collecting the computing data of the computing operators in the entire computing system during the data processing process, we can predict the computing demand of the computing operators in the future time period T, and determine the types of computing operators with the largest computing demand as the computing operators that need the computing power support of the programmable hardware FPGA module. Figure 7aAmong the types of operators with the largest computing demand shown, the computing power demand of Orc reader is decreasing, while the computing power demand of Hash Join Operator, Json reader, Csv reader and Sorter Operator is increasing. Although the computing power demand of Filter Operator fluctuates, the overall fluctuation is not large. Therefore, a preliminary conclusion can be drawn through statistical analysis: the computing power requirements of Hash Join Operator, Json reader, Csv reader and Sorter Operator are constantly increasing, and the computing power needs to be supplemented in time, otherwise the computing system will be busier and the computing efficiency will be lower. Therefore, it is possible to consider increasing the distribution ratio of such operators in the programmable hardware FPGA module. Figure 7b As shown in the figure, the computing power requirement of the Orc reader is decreasing, so the distribution ratio of such operators in its redundant programmable hardware FPGA module can be released, otherwise it may cause a waste of computing resources; the Filter Operator belongs to the normal range to be continuously observed, and there is no need to adjust the distribution ratio of such operators for the time being.

[0071] In a complete embodiment provided by the present invention, Figure 8 As shown, it is a complete operation flow chart of a system for adjusting the distribution of computing operator types in a computing system provided by an embodiment of the present invention. First, the user creates a logical library and a logical table in the database business system and stores them in the metadata module (meta module). Then, the user submits SQL query data based on the database business system in his own program to implement corresponding data query operations. The SQL query data process mainly goes through the analysis module (Performance & Cost Analyzer), SQL optimization control module (SQL Optimizer), physical resource allocation module (Physical Resource Planer), execution scheduling module (Execution Scheduler), execution module (Stage Execution) and merge result module (Result Merger) to parse the user's data processing request, form a logical execution plan tree, and then generate an execution plan, and the computing system provides computing power for the execution plan. Finally, the data required by the user is gradually summarized and returned.

[0072] The present invention can make statistics on the amount of calculation data of the calculation operator during the calculation process through the monitoring module (Monitor), and also make statistics on the calculation time consumed by the calculation operator. This information will eventually be collected in a unified resource management module (Resource Manager), and then the FPGA management module (FPGA Manager) will adjust the types and distribution ratios of the calculation operators in the FPGA module during data processing, and synchronize the types and distribution ratios of the calculation operators in the adjusted FPGA module to the metadata module (Meta module), thereby ensuring accurate computing power support for the next execution plan.

[0073] In actual application, when using the programmable structure hardware FPGA module, firstly, according to past experience, the logic gate circuit allocation in the FPGA module is planned in advance, and the computing operator with the largest computing demand may be programmed into the FPGA module in advance. Then, the load trend of the entire computing system in the data processing process in the next period of time is monitored and collected, and the computing demand of the computing operator in the future time period T is predicted, and the types of computing operators with the largest predicted computing demand are determined as the computing operators that need the computing power support of the programmable hardware FPGA module. The database business system determines whether to dynamically adjust the computing operators in the FPGA module by comparing the difference between the preset computing operator planning in the FPGA module and the predicted computing operator situation in the actual online data processing process. If some differences are stable enough (for example, more than 10 days or 1 month) and reach or exceed the threshold range preset by the database business system (for example, 5%), the FPGA manager orchestration module (Partial Reconfiguration) of the FPGA module will be triggered to make adjustments. According to the latest computing operator distribution ratio predicted in the current actual data processing process, the arrangement of the logic gate circuits in the FPGA module is replanned, and the computing operator distribution ratio in the FPGA module is gradually adjusted and replaced. After the adjustment, when the database business system receives new SQL query data, the SQL Optimizer module of the database business system will perceive the distribution ratio information of the adjusted calculation operators and generate the latest execution plan for the user's SQL query data. If some calculation operators match the latest calculation operator list of FPGA, these calculation operators are pushed to the FPGA module for execution, otherwise they are placed in the CPU module for execution, and the statistical information of the calculation operators is continuously collected. The whole process is repeated, and the database business system continuously collects data during operation, dynamically feeds back to the FPGA manager orchestration module of the FPGA module, and then dynamically adjusts the types and distribution ratios of the calculation operators in the FPGA module, thereby gradually improving the calculation efficiency of the calculation system during the data processing of the database business system.

[0074] It should be noted that the data processing process described in the present invention is not limited to the data processing tasks of the database business system, and the computing system described is not limited to accelerating the data processing tasks of the database business system. The method of adjusting the distribution of computing operator types in the computing system described in the present invention can be applied to accelerate different types of data processing tasks and computing scenarios.

[0075] By adopting the method for adjusting the distribution of computing operator types in the computing system provided by the present invention, it is possible to obtain the predicted demand for computing operators in future time periods by analyzing the data records of the computing operators' computing volume during the data processing process, and then accurately adjust the types and distribution ratios of computing operators implemented in programmable hardware on the computing system, so that the computing power of the programmable hardware module can be used as much as possible to serve the busiest data processing tasks, thereby improving the computing efficiency of the entire computing system.

[0076] Corresponding to the method for adjusting the distribution of computing operator types in a computing system provided above, the present invention also provides a device for adjusting the distribution of computing operator types in a computing system. Since the embodiment of the device is similar to the embodiment of the method described above, the description is relatively simple. For relevant details, please refer to the description of the embodiment of the method described above. The following description of an embodiment of a device for adjusting the distribution of computing operator types in a computing system is only illustrative. Please refer to Figure 2 As shown, it is a schematic diagram of a device for adjusting the distribution of computing operator types in a computing system provided by an embodiment of the present invention.

[0077] The device for adjusting the distribution of computing operator types in a computing system according to an embodiment of the present invention includes the following parts:

[0078] The monitoring unit 201 is used to obtain the operation amount data record of the calculation operator in the actual data processing of the computing system, and the calculation operator completes the data processing task.

[0079] In an embodiment of the present invention, the computing system may be a heterogeneous hardware accelerated computing system composed of a CPU module and a programmable hardware FPGA module for processing database data, and often also includes a GPU module.

[0080] The computing operators are used to complete data processing tasks. For different computing scenarios, computing operators may have different meanings. Among them, for the computing system for databases in this embodiment, it may include Filter Operator, Table Scan Operator, Hash Join Operator, Aggregation Operator, Sort Operator, and Project Operator in SQL processing tasks. Figure 4 The application scenario flow chart of a computing system shown in the figure, in the specific implementation process, the user sends a data processing request to the DS (Data Store) through the APP application, and the Load Balancer (load balancing module) implements the load balancing of the data processing request. The front-end node module (Front node Pool) receives the data processing request assigned by the load balancing module for analysis and processing, assigns it to different FNs (Front Nodes; front-end nodes), and generates a corresponding execution plan and sends it to the above CN (Compute node Pool; computing node), and then returns the corresponding data stored in the DS (DataStore; database). The execution plan is an AST (abstract syntax tree) composed of several computing operators in a set logical order.

[0081] In the process of executing the execution plan, the calculation operators will be respectively assigned to the CPU, GPU and the corresponding units in the FPGA module set as the corresponding calculation operators for execution. Since the types and distribution ratios of the preset calculation operators in the programmable hardware FPGA module are usually set in advance based on manual experience, and only the calculation tasks of specific calculation operators can be executed after setting. Therefore, in the process of accelerating the calculation of data processing, due to improper settings, the calculation operators with large actual calculation requirements may not get enough calculation power, while the operators with small actual calculation requirements occupy the calculation power resources of the FPGA module for a long time, resulting in low calculation efficiency of the entire calculation system.

[0082] To this end, the present invention provides a method for adjusting the distribution of computing operator types in a computing system. The premise of this method is to monitor and record the computing volume data of the computing operators in the actual data processing process of the entire computing system and perform statistical aggregation, so as to grasp the computing volume requirements of the computing system for each computing operator. Monitoring and recording the computing volume data of the computing operators can be carried out in different ways according to different needs. One possible way is to monitor the historical data of the computing volume of each computing operator of the computing system. This method mainly takes into account that the computing requirements in the computing system often have historical continuity, and historical data can be used to judge future computing requirements. The following introduces a specific implementation method of using this method.

[0083] like Figure 6 As shown, the figure is a schematic diagram of the computing amount data of a computing operator on a monitoring computing system provided by an embodiment of the present invention. Different types of computing operators are represented by different colors, including: Filter Operator, Table Scan Operator, Hash join Operator, Aggregation Operator, Sort Operator and Project Operator. By analyzing the computing amount data records monitored during the period from November 1st to November 7th, it can be found that among the most time-consuming types of computing operators, the demand for computing power of Filter Operator and Table Scan Operator is decreasing, while the computing power demand of Hash join Operator and Aggregation Operator is increasing, the computing power requirement of Sort Operator is relatively stable, and although the computing power demand of Project Operator fluctuates, the overall fluctuation is not large.

[0084] The resource manager unit 202 is used to obtain the predicted demand for the computing operators on the computing system in the future time period T according to the computing volume data records of the computing operators.

[0085] The monitoring unit 201 obtains the data record of the amount of computation of the computing operator in the actual data processing of the computing system within a predetermined time period. Data preparation is done for the resource manager unit 202 to predict the predicted demand for the computing operator on the computing system in the future time period T. In the resource manager unit 202, the predicted demand for the computing operator on the computing system in the future time period T can be obtained based on the data record of the amount of computation of the computing operator monitored above.

[0086] By continuously monitoring and collecting the trend of the load on the entire computing system during data processing, the computing operator whose predicted demand reaches or exceeds the preset demand threshold can be determined as a computing operator that requires the programmable hardware FPGA module to provide computing power support. The predicted demand of the computing operator monitored above is the predicted demand for computing volume, and the way to measure the computing volume demand, specifically, the computing time can be used as a measurement indicator, that is, the computing volume demand of a certain computing operator is the time-consuming time to execute the computing operator within the determined time period. The time to execute a computing operator includes the time for the CPU, GPU, FGPA or other programmable hardware to execute the computing operator. At the same time, quite a number of computing operators of this type can be executed at the same time, and for each executed computing operator, its running time is calculated and summed up. Of course, the computing volume demand of each computing operator can also be measured by using the proportion of the computing operator in the total computing operator time. The most time-consuming types of computing operators are determined as computing operators that require programmable hardware FPGA modules to provide computing power support.

[0087] For example, through the analysis of the computing volume data records monitored during the period from November 1 to November 7, it can be found that among the most time-consuming types of computing operators, the computing power requirements of Filter Operator and Table Scan Operator are decreasing, while the computing power requirements of Hash Join Operator and Aggregation Operator are increasing. The computing power requirements of Sort Operator are relatively stable, and although the computing power requirements of Project Operator fluctuate, the overall fluctuation is not large. Therefore, before the arrival of the future time period T, the demand trend for computing power can be obtained: 1) The computing power requirements of Hash join Operator and Aggregation Operator are constantly increasing, and the computing power needs to be replenished in time, otherwise the computing system will be busier and the computing efficiency will be lower. At this time, the distribution ratio of such computing operators in the programmable hardware FPGA module can be increased; 2) The computing power requirements of Filter Operator and Table Scan Operator are decreasing, so the distribution ratio of such computing operators in the programmable hardware FPGA module can be released, otherwise it may cause a waste of computing resources; 3) Sort Operator and Project Operator are within the normal fluctuation range, and there is no need to adjust the distribution ratio of such computing operators for the time being.

[0088] Specifically, according to the computing volume data records of the above-mentioned monitored computing operators, the predicted demand for computing operators on the computing system in the future time period T can be obtained, which can be achieved in the following way: first, obtain the computing volume data record samples of the computing operators within the predetermined time period, and the actual demand samples for the computing operators in the future time period T that match the computing volume data record samples of the computing operators in the predetermined time period; then use the computing volume data record samples of the computing operators in the predetermined time period, and the actual demand samples for the computing operators in the future time period T that match the computing volume data record samples of the computing operators in the predetermined time period to train a pre-set deep neural network model, and finally obtain a computing operator demand prediction model. The monitored computing operator computing volume data records are input into the computing operator demand prediction model in real time, and the predicted demand for the computing operators on the computing system in the future time period T can be accurately obtained. It should be noted that the computing operator demand prediction model is a deep neural network model that predicts the predicted demand for the computing operators in the future time period T based on the computing volume data records of the computing operators in the predetermined time period. It should be noted that the prediction model described in the present invention is not limited to the above-mentioned one obtained by collecting training samples to train a preset deep neural network model, and it can also be implemented by a logistic regression model. In the specific implementation process, a suitable model can be selected through performance comparison testing.

[0089] The programmable hardware management unit 203 is used to adjust the computing operators implemented in programmable hardware on the computing system before the future time period T arrives according to the predicted demand for the computing operators on the computing system, and the adjustment includes at least one of the following: the type of computing operators implemented in programmable hardware, and the distribution ratio of the computing operators implemented in programmable hardware.

[0090] After the resource manager unit 202 obtains the predicted demand for computing operators on the computing system in the future time period T, the programmable hardware management unit 203 can adjust the type and distribution ratio of computing operators implemented in programmable hardware on the computing system in advance before the future time period T arrives by analyzing the predicted demand.

[0091] Specifically, according to the predicted demand for computing operators on the computing system within a predetermined time period monitored in advance, the type and distribution ratio of computing operators implemented by programmable hardware can be accurately adjusted, which can be achieved in the following manner:

[0092] 1) According to the predicted demand for computing operators on the computing system, first determine the computing operators on the computing system that reach or exceed the preset demand threshold. The type of computing operators that reach or exceed the preset demand threshold is determined as the type of computing operators that need to be implemented in programmable hardware on the computing system before the future time period T arrives. According to the type of computing operators that need to be implemented in programmable hardware on the computing system, adjust the type of computing operators that are preset in programmable hardware on the computing system before the future time period T arrives. In the specific application process, the type of computing operator that needs to be implemented in programmable hardware on the computing system can be compared with the actual type of computing operator that is preset in programmable hardware on the computing system to determine whether there is a computing operator of the corresponding type on the computing system. If not, add the type of computing operator of the corresponding type that is implemented in programmable hardware on the computing system before the future time period T arrives.

[0093] Since the load of computing operators is usually constantly changing during the data processing process of the database system, not only do different types of computing operators have different requirements for computing power, but the same type of operators may also have different requirements for computing power in different time periods, that is, the computing power demand has the characteristics of temporal periodicity. Therefore, the load of the computing operator monitored within the above-mentioned determined time period may not always reach or exceed the preset load threshold. Therefore, it is sometimes necessary to adjust the type or distribution ratio of the computing operator in the FPGA module in real time. For example, Figure 5a and Figure 5b As shown in FIG. 1 , they are respectively a schematic diagram of the distribution of computing operator types before adjustment and a schematic diagram of the distribution of computing operator types after adjustment in a computing system provided by an embodiment of the present invention. In the previous execution plan, it is possible that Figure 5a The two corresponding Table Scan Operators need to be accelerated by the programmable hardware FPGA module; in the latter execution plan, due to the changes in the statistical information of different computing operators, the scheduling layer of the programmable hardware FPGA module automatically balances the types of its own computing operators. The Aggregation Operator and Join Operator with the highest actual load in the recent period are loaded into the FPGA module in a certain proportion, while one of the Table Scan Operators with lower computing power requirements is unloaded. Therefore, from the user's perspective, when the database system processes the same SQL data and executes it in different time periods, the FPGA module can dynamically adjust to provide different computing power support. It should be noted that the computing volume data record of the computing operator includes at least one of the occupancy ratio data and time-consuming time data of the computing operator during the data processing process.

[0094] 2) According to the predicted demand for the computing operators on the computing system, determine the computing operators on the computing system that reach or exceed the preset demand threshold. The predicted demand for the computing operators that reach or exceed the preset demand threshold is determined as the demand for the computing operators implemented in programmable hardware on the computing system before the future time period T arrives. According to the demand for the computing operators implemented in programmable hardware on the computing system, adjust the distribution ratio of the computing operators implemented in programmable hardware preset on the computing system before the future time period T arrives.

[0095] In a specific application process, one implementation method is: compare the demand for computing operators implemented in programmable hardware on the computing system with the distribution of computing operators implemented in programmable hardware preset on the computing system, and if the matching degree is lower than the preset index, adjust the distribution ratio of the corresponding type of computing operators implemented in programmable hardware on the computing system before the arrival of the future time period T. Another method is: when it is monitored that the demand for computing operators implemented in programmable hardware on the computing system is reduced, the distribution ratio of the corresponding type of computing operators implemented in programmable hardware is reduced before the arrival of the future time period T; when it is monitored that the demand for computing operators implemented in programmable hardware on the computing system is increased, the distribution ratio of the corresponding type of computing operators implemented in programmable hardware is increased before the arrival of the future time period T; when it is monitored that the demand for computing operators implemented in programmable hardware on the computing system is stable within the preset range, the distribution ratio of the corresponding type of computing operators implemented in programmable hardware on the computing system remains unchanged.

[0096] By using the device for adjusting the distribution of computing operator types in a computing system provided by the present invention, the computing volume data records of the computing operators in the data processing process can be obtained through the monitoring unit, and the resource manager can analyze the computing volume data records to obtain the predicted demand for the computing operators in the future time period, and then the type and distribution ratio of the computing operators implemented in programmable hardware on the computing system can be accurately adjusted through the programmable hardware management unit, so that the computing power of the programmable hardware module can serve the busiest data processing tasks as much as possible, thereby improving the computing efficiency of the entire computing system.

[0097] Corresponding to the method for adjusting the distribution of computing operator types in the computing system provided above, the present invention also provides an electronic device. Figure 3 As shown, it is a schematic diagram of an electronic device provided by an embodiment of the present invention.

[0098] The electronic device provided by the present invention specifically includes: a processor 302 and a memory 301. Among them, the memory 301 is used to store a program of a method for adjusting the distribution of types of computing operators in a computing system. After the device is powered on and runs the program of the method for adjusting the distribution of types of computing operators in the computing system through the processor 302, step 1 is performed: obtaining a data record of the amount of calculation of the computing operator in the actual data processing of the computing system, and the computing operator is to complete the data processing task; step 2: according to the data record of the amount of calculation of the computing operator, obtaining the predicted demand for the computing operator on the computing system in the future time period T; step 3: according to the predicted demand for the computing operator on the computing system, before the arrival of the future time period T, the computing operator implemented in a programmable hardware manner on the computing system is adjusted, and the adjustment includes at least one of the following: the type of the computing operator implemented in the programmable hardware manner, and the distribution ratio of the computing operator implemented in the programmable hardware manner.

[0099] Corresponding to the method for adjusting the distribution of computing operator types in the computing system provided above, the present invention also provides a storage device, which stores a program for the method for adjusting the distribution of computing operator types in the computing system, and the program is run by a processor to perform step one: obtaining a data record of the amount of calculation of the computing operator in the actual data processing of the computing system, and the computing operator is to complete the data processing task; step two: based on the data record of the amount of calculation of the computing operator, obtaining the predicted demand for the computing operator on the computing system in the future time period T; step three: based on the predicted demand for the computing operator on the computing system, adjusting the computing operator implemented in programmable hardware on the computing system before the future time period T arrives, and the adjustment includes at least one of the following: the type of computing operator implemented in the programmable hardware, and the distribution ratio of the computing operator implemented in the programmable hardware.

[0100] Although the present invention is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the scope defined by the claims of the present invention.

[0101] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0102] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0103] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0104] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

Claims

1. A method for adjusting the distribution of computing operator types in a computing system, characterized in that: include: Obtaining a data record of the amount of computation of at least one type of computational operators in actual data processing of the computing system, wherein the computational operators are used to complete the data processing task; According to the calculation amount data record of the calculation operator, obtain the predicted demand amount of the calculation operator on the calculation system in the future time period T; According to the predicted demand for computing operators on the computing system, the computing operators implemented in programmable hardware on the computing system are adjusted before the future time period T arrives, and the adjustment includes at least one of the following: the type of computing operators implemented in programmable hardware, and the distribution ratio of the computing operators implemented in programmable hardware.

2. The method for adjusting the distribution of computing operator types in a computing system according to claim 1, characterized in that: The step of adjusting the computing operators implemented in programmable hardware on the computing system according to the predicted demand for the computing operators on the computing system before the future time period T arrives specifically includes: According to the predicted demand of the computing operators on the computing system, determining the computing operators on the computing system that reach or exceed a preset demand threshold; Determine the type of the computing operator that reaches or exceeds the preset demand threshold as the type of computing operator that needs to be implemented in a programmable hardware manner on the computing system before the future time period T arrives; According to the type of computing operator implemented in programmable hardware on the computing system as required, the type of computing operator implemented in programmable hardware preset on the computing system is adjusted before the future time period T arrives.

3. The method for adjusting the distribution of computing operator types in a computing system according to claim 2, characterized in that: The type of the computing operator implemented in a programmable hardware manner on the computing system according to the need, adjusting the type of the computing operator implemented in a programmable hardware manner preset on the computing system before the future time period T arrives, specifically includes: The type of computing operator that needs to be implemented in programmable hardware on the computing system is compared with the actual type of computing operator implemented in programmable hardware preset on the computing system to determine whether there is a computing operator of the corresponding type on the computing system; if not, the type of computing operator implemented in programmable hardware of the corresponding type on the computing system is added before the future time period T arrives.

4. The method for adjusting the distribution of computing operator types in a computing system according to claim 1, characterized in that: The step of adjusting the computing operators implemented in programmable hardware on the computing system according to the predicted demand for the computing operators on the computing system before the future time period T arrives specifically includes: According to the predicted demand of the computing operators on the computing system, determining the computing operators on the computing system that reach or exceed a preset demand threshold; Determine the predicted demand of the computing operator that reaches or exceeds the preset demand threshold as the demand of the computing operator implemented in programmable hardware on the computing system before the future time period T arrives; According to the demand for computing operators implemented in programmable hardware on the computing system, the distribution ratio of computing operators implemented in programmable hardware preset on the computing system is adjusted before the future time period T arrives.

5. The method for adjusting the distribution of computing operator types in a computing system according to claim 4, characterized in that: The adjusting, according to the demand for the computing operators implemented in the programmable hardware manner on the computing system, the distribution ratio of the computing operators implemented in the programmable hardware manner preset on the computing system before the future time period T arrives, specifically includes: When the demand for computing operators implemented in programmable hardware on the computing system decreases, reducing the distribution ratio of the corresponding type of computing operators implemented in programmable hardware before the future time period T arrives; When the demand for computing operators implemented in programmable hardware on the computing system increases, the distribution ratio of computing operators of the corresponding type implemented in programmable hardware is increased before the future time period T arrives; When the demand for computing operators implemented in programmable hardware on the computing system is stabilized within a preset range, the distribution ratio of the corresponding type of computing operators implemented in programmable hardware on the computing system remains unchanged.

6. The method for adjusting the distribution of computing operator types in a computing system according to claim 4, characterized in that: The adjusting, according to the demand for the computing operators implemented in the programmable hardware manner on the computing system, the distribution ratio of the computing operators implemented in the programmable hardware manner preset on the computing system before the future time period T arrives, specifically includes: The demand for computing operators implemented in programmable hardware on the computing system is compared with the distribution of computing operators implemented in programmable hardware preset on the computing system. If the matching degree is lower than the preset index, the distribution ratio of the corresponding type of computing operators implemented in programmable hardware on the computing system is adjusted before the future time period T arrives.

7. The method for adjusting the distribution of computing operator types in a computing system according to claim 1, characterized in that: The step of obtaining the predicted demand for the computing operators on the computing system in a future time period T according to the computing amount data record of the computing operators includes: The computing volume data record of the computing operator is input into a preset computing operator demand prediction model to obtain the predicted demand for the computing operator on the computing system in the future time period T. The computing operator demand prediction model is a deep neural network model that predicts the predicted demand for the computing operator in the future time period T based on the computing volume data record of the computing operator in a predetermined time period.

8. The method for adjusting the distribution of computing operator types in a computing system according to claim 7, characterized in that: Also includes: Obtaining a sample of the calculation amount data record of the calculation operator within the predetermined time period; Obtaining an actual demand quantity sample for the computing operator in a future time period T that matches the computing quantity data record sample of the computing operator within the predetermined time period; The deep neural network model is trained using the calculation amount data record samples of the calculation operator within the predetermined time period, and the actual demand samples of the calculation operator in the future time period T that matches the calculation amount data record samples of the calculation operator within the predetermined time period to obtain the calculation operator demand prediction model.

9. The method for adjusting the distribution of computing operator types in a computing system according to claim 1, characterized in that: The obtaining of the operation amount data record of the computing operator in the actual data processing of the computing system specifically includes: Obtaining a data record of the amount of computation of a computing operator in the computing system used to process a data processing request from a client in a database; Correspondingly, the computing operators are all special computing programs set up on the computing system in the database for data processing tasks.

10. The method for adjusting the distribution of computing operator types in a computing system according to claim 1, characterized in that: The computing system is a heterogeneous hardware accelerated computing system composed of a CPU module and a programmable hardware module for processing database data.

11. The method for adjusting the distribution of computing operator types in a computing system according to claim 1, characterized in that: The calculation amount data record of the calculation operator includes at least one of the occupancy ratio data and usage time data of the calculation operator during the data processing process.

12. The method for adjusting the distribution of computing operator types in a computing system according to claim 1, characterized in that: The predicted demand of the computing operator is the predicted time consumed by the computing operator during the data processing process.

13. A device for adjusting the distribution of computing operator types in a computing system, characterized in that: include: monitoring unit, resource manager unit, and programmable hardware management unit; The monitoring unit is used to obtain a data record of the amount of calculation of a computing operator in actual data processing of the computing system, wherein the computing operator is used to complete the data processing task; The resource manager unit is used to obtain the predicted demand for the computing operators on the computing system in the future period T according to the computing amount data record of the computing operators; The programmable hardware management unit is used to adjust the computing operators implemented in programmable hardware on the computing system before the future time period T arrives according to the predicted demand for the computing operators on the computing system, and the adjustment includes at least one of the following: the type of computing operators implemented in programmable hardware, and the distribution ratio of the computing operators implemented in programmable hardware.

14. A system for adjusting the distribution of computing operator types in a computing system, characterized in that: include: The device for adjusting the distribution of computing operator types in the computing system of claim 13.

15. An electronic device, characterized in that: include: processor; as well as A memory is used to store a program of a method for adjusting the distribution of computing operator types in a computing system. After the device is powered on and the processor runs the program of the method for adjusting the distribution of computing operator types in the computing system, the following steps are performed: Obtaining a data record of the amount of computation of a computing operator in actual data processing of a computing system, wherein the computing operator is used to complete a data processing task; According to the calculation amount data record of the calculation operator, obtain the predicted demand amount of the calculation operator on the calculation system in the future time period T; According to the predicted demand for computing operators on the computing system, the computing operators implemented in programmable hardware on the computing system are adjusted before the future time period T arrives, and the adjustment includes at least one of the following: the type of computing operators implemented in programmable hardware, and the distribution ratio of the computing operators implemented in programmable hardware.

16. A storage device, characterized in that: A program for adjusting the distribution of computing operator types in a computing system is stored, and the program is executed by a processor to perform the following steps: Obtaining a data record of the amount of computation of a computing operator in actual data processing of a computing system, wherein the computing operator is used to complete a data processing task; According to the calculation amount data record of the calculation operator, obtain the predicted demand amount of the calculation operator on the calculation system in the future time period T; According to the predicted demand for computing operators on the computing system, the computing operators implemented in programmable hardware on the computing system are adjusted before the future time period T arrives, and the adjustment includes at least one of the following: the type of computing operators implemented in programmable hardware, and the distribution ratio of the computing operators implemented in programmable hardware.

17. A method for adjusting the distribution of computing operator types in a computing system, characterized in that: include: Obtaining a data record of the amount of computation of at least one type of computational operators in actual data processing of the computing system, wherein the computational operators are used to complete the data processing task; According to the calculation amount data record of the calculation operator, obtain the predicted demand amount of the calculation operator on the calculation system in the future time period T; According to the predicted demand for computing operators on the computing system, the computing operators implemented by hardware accelerators on the computing system are adjusted before the future time period T arrives, and the adjustment includes at least one of the following: the type of computing operators implemented by the hardware accelerator, and the distribution ratio of the computing operators implemented by the hardware accelerator.

Images