Video stream processing method and system based on qos constraints under distributed computing platform

Abstract

The invention discloses a video stream processing method based on QoS constraints under a distributed computing platform. It first defines QoS as the accuracy and delay of operators for the video stream processing characteristics, and redefines video stream data processing based on this programming model. Then publish the model to the distributed processing framework. According to the processing process submitted by the user, the platform first analyzes each operator, and predetermines the accuracy of the demand and the amount of resources required for the delay. Then, according to the data dependencies of the application, estimate the resources that should be allocated during the entire application processing process for the combination of multiple operators (serial, parallel, serial to parallel, and parallel to serial) to determine whether the requirements can be met and based on the previous The analysis of the operator can get the estimated delay of the processing time. Then the window size for data processing is obtained according to the delay and the input of the data stream, and then the total amount of resources is estimated according to the number of video stream channels estimated by the user.

Description

technical field [0001] The invention belongs to the technical field of distributed computing, and more specifically relates to a video stream processing method and system based on QoS constraints under a distributed computing platform. Background technique [0002] With the continuous improvement of video processing technology, the number of cameras deployed is also increasing, which also means that more and more real-time video streams need to be processed in a timely manner. Due to its large amount of data, large amount of calculation, and strong timeliness of data, video stream processing not only has high requirements on software and hardware platforms, but also relatively complex application development. It has always been a research field with a high threshold. [0003] Existing big data processing platforms (such as Spark, Flink, Storm, etc.) all provide data flow processing models, and provide automatic data segmentation, task scheduling, load balancing, data caching...

AI Technical Summary

Problems solved by technology

[0005] Aiming at the above defects or improvement needs of the prior art, the present invention provides a video stream processing method and system based on QoS constraints under a distributed computing platform, the purpose of which is to solve the existing data stream processing model based on a big data processing platform Due to the inability to modify the task parameters, the technical problem of being unable to meet the specific needs of different users for the task, and the technical problem of low efficiency of system resource allocation and scheduling due to the inability to allocate and schedule according to the specific needs of the task, and Since the data stream processing model is mainly based on the user's own definition of the data stream processing window for each processing, the technical problem that the user cannot get a timely response when the processing volume of the processing model is too large

Images