Real-time data calculation method and device

Abstract

The invention provides a real-time data calculation method. The method is applied to an online business system, wherein the online business system carries a python language-based algorithm operation environment and an algorithm library. The method comprises the following steps of: obtaining real-time business data from a butted real-time data processing platform by the online business system; generating a python calculation task for the real-time business data on the basis of a target algorithm in the algorithm library; distributing calculation node equipment for the python calculation task on the basis of a preset scheduling strategy, and issuing the python calculation task to the calculation node equipment to be executed; and outputting a calculation result, returned by the calculation node equipment, of the python calculation task through a visual interface. According to the method provided by the invention, the effectiveness of real-time calculation can be enhanced, and the calculation time delay can be reduced.

Description

technical field [0001] The present application relates to the field of computer applications, in particular to a method and device for calculating real-time data. Background technique [0002] In related technologies, real-time data processing platforms are usually equipped with an independent algorithm platform, and use the general solutions provided by the algorithm platform to provide relevant algorithm services to business operators; for example, the real-time data processing platform independently developed by Ali Group Take ODPS (Open Data Processing Service) as an example. The ODPS platform is usually equipped with algorithm platforms such as XLib and PAI to provide various types of algorithm services for Ali Group. [0003] The algorithm platform carried on the real-time data processing platform usually uses the message middleware integrated in the platform to collect the original business data by collecting the log files generated by the business system, and returns...

AI Technical Summary

Problems solved by technology

[0005] On the one hand, although in the existing real-time data processing platform, the algorithm library in the algorithm platform can also be expanded by doing related algorithm development, but because the types of algorithms supported by the existing algorithm library on the platform are limited, only It is suitable for algorithm support for some stable businesses, but it is difficult to support those fast-changing business scenarios (such as large-scale promotional businesses in a short period of time)

[0006] On the other hand, in the existing real-time data processing platform, the original business data needs a long delay from collection to transmission to the real-time data processing platform; moreover, after the original business data is calculated and processed by the algorithm, the Calculation results are written back to the real-time data processing platform and called by the connected downstream business system, which will also lead to a sharp increase in processing delay. Therefore, it is difficult to guarantee the overall business stability and timeliness of the platform, and it can only adapt to some offline computing tasks.

Images