Video stream processing method and system based on QoS constraints under distributed computing platform

Abstract

The invention discloses a video stream processing method based on QoS constraint under a distributed computing platform, and the method comprises the following steps: defining QoS as the accuracy andtime delay of an operator according to video stream processing characteristics, and redefining a programming model for video stream data processing based on the accuracy and time delay of the operator; publishing the model to a distributed processing framework, enabling the platform to firstly analyze each operator according to the processing process submitted by a user, and determining the accuracy of the demand and the resource quantity of the time delay demand in advance; according to the data dependence relationship of the application, estimating the resources which should be allocated inthe whole application processing process according to the situation (serial, parallel, serial-to-parallel and parallel-to-serial) of the combination of the plurality of operators, judging whether therequirements can be met or not, and obtaining the estimated time delay of the processing time according to the previous analysis of the operators; obtaining the size of a data processing window according to the time delay condition and the input of the data stream, and estimating the total amount of resources according to the number of video stream paths estimated by a user.

Description

technical field [0001] The invention belongs to the technical field of distributed computing, and more specifically relates to a video stream processing method and system based on QoS constraints under a distributed computing platform. Background technique [0002] With the continuous improvement of video processing technology, the number of cameras deployed is also increasing, which also means that more and more real-time video streams need to be processed in a timely manner. Due to its large amount of data, large amount of calculation, and strong timeliness of data, video stream processing not only has high requirements on software and hardware platforms, but also relatively complex application development. It has always been a research field with a high threshold. [0003] Existing big data processing platforms (such as Spark, Flink, Storm, etc.) all provide data flow processing models, and provide automatic data segmentation, task scheduling, load balancing, data caching...

AI Technical Summary

Problems solved by technology

[0005] Aiming at the above defects or improvement needs of the prior art, the present invention provides a video stream processing method and system based on QoS constraints under a distributed computing platform, the purpose of which is to solve the existing data stream processing model based on a big data processing platform Due to the inability to modify the task parameters, the technical problem of being unable to meet the specific needs of different users for the task, and the technical problem of low efficiency of system resource allocation and scheduling due to the inability to allocate and schedule according to the specific needs of the task, and Since the data stream processing model is mainly based on the user's own definition of the data stream processing window for each processing, the technical problem that the user cannot get a timely response when the processing volume of the processing model is too large

Images