Video stream data processing system and method

Abstract

The invention provides a video stream data processing system and method, and relates to the technical field of edge computing, cloud computing and stream-oriented computing. The video stream data processing system comprises an infrastructure layer used for constructing and managing an edge computing node on which a lightweight container cluster is deployed; the stream computing engine layer is used for calling the container cluster on the edge computing node according to a to-be-processed task so as to process video stream data in parallel; and the task co-processing layer is used for adjusting the distribution of the to-be-processed task on the container cluster according to the resource utilization rate of the edge computing node when the edge computing node processes the video stream data.

Description

Technical Field

[0001] The present invention relates to technical fields such as edge computing, cloud computing, and streaming computing, and in particular to a video stream data processing system and method. Background Art

[0002] In today's digital age, the importance of video streaming application scenarios is becoming increasingly prominent. Take remote sensing data video streaming as an example. With the rapid development of the aerospace field, the number of remote sensing satellites is increasing, and drone remote sensing technology is also booming. Massive remote sensing image video streams are being generated continuously. It has a wide range of application scenarios in resources, meteorology, ocean, environment, national defense, etc. These video streaming application scenarios usually have high requirements for real-time data processing. The existing video streaming processing methods have scattered computing and storage resources, which are difficult to cope with the challenges of real-time data processing, thus hindering its development. Summary of the invention

[0003] In view of this, the present invention provides a video stream data processing system and method to at least partially solve the above problems.

[0004] On the one hand, the present invention provides a video stream data processing system, including: an infrastructure layer, which is used to build and manage edge computing nodes, on which lightweight container clusters are deployed; a stream computing engine layer, which is used to call the container clusters on the edge computing nodes to process video stream data in parallel according to the tasks to be processed; and a task collaborative processing layer, which is used to adjust the allocation of tasks to be processed on the container cluster according to the resource utilization of the edge computing nodes when processing video stream data.

[0005] According to an embodiment of the present invention, an edge computing node deploys and manages a container cluster through a lightweight container orchestration tool, wherein the lightweight container orchestration tool uses a lightweight database to store data of the container cluster and starts the container cluster through a binary file.

[0006] According to an embodiment of the present invention, the stream computing engine layer includes a stream computing component and a stream computing framework, wherein the stream computing framework is used to receive tasks to be processed, and by integrating with the stream computing component, call a container cluster to process video stream data in parallel.

[0007] According to an embodiment of the present invention, the streaming computing framework includes a job manager and at least two parallel task managers. The job manager is used to schedule and allocate tasks to be processed, and the task manager is used to receive and execute tasks to be processed allocated by the job manager, and exchange data with at least one other task manager when executing tasks to be processed.

[0008] According to an embodiment of the present invention, the task collaborative processing layer includes a resource perception component, wherein the resource perception component is used to collect and analyze indicator data of the edge computing node when processing video stream data, and obtain resource utilization based on the indicator data.

[0009] According to an embodiment of the present invention, the task collaborative processing layer also includes a task scheduling component, wherein the task scheduling component is used to utilize the resource utilization obtained by the resource awareness component to adjust the dynamic allocation of tasks to be processed on the container cluster based on a preset scheduling algorithm to improve resource utilization.

[0010] According to an embodiment of the present invention, the scheduling algorithm includes a flow topology collaborative scheduling method, wherein the flow topology collaborative scheduling method adjusts the structure of the flow topology according to resource utilization to adjust the allocation of tasks to be processed on the container cluster.

[0011] According to an embodiment of the present invention, the task scheduling component is also used to adjust the number and capacity of the container cluster to achieve elastic scaling of resources of the container cluster.

[0012] According to an embodiment of the present invention, the task scheduling component adjusts the number and capacity of the container cluster by horizontal expansion and / or vertical expansion.

[0013] On the other hand, the present invention provides a method for processing video stream data, including: receiving video stream data and tasks to be processed; calling a container cluster on an edge computing node to parallel process the video stream data according to the tasks to be processed; and adjusting the allocation of the tasks to be processed on the container cluster according to the resource utilization rate when the edge computing node processes the video stream data.

[0014] According to the video stream data processing system and method of the embodiment of the present invention, the stream computing engine layer calls the lightweight container cluster on the edge computing node to process the video stream data in parallel, which can greatly improve the speed and efficiency of data processing. Parallel processing allows a large amount of data to be processed by multiple computing resources at the same time, reducing the overall processing time. By utilizing lightweight container technology, by packaging the application and its dependencies into lightweight containers, rapid deployment and flexible expansion on edge devices are achieved, the resources of the edge devices are maximized, and the flexibility and scalability of the system are improved. At the same time, the resource-aware task scheduling mechanism can monitor the resource utilization of edge devices in real time, and perform task scheduling and resource allocation according to the dynamic changes of resources. It can realize effective distributed task processing in the edge environment, cope with the dynamic changes of resources, and improve the system's task execution efficiency and resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0016] Figure 1 A schematic diagram of a video stream data processing system according to an embodiment of the present invention is shown;

[0017] Figure 2 A K3s container structure diagram of a video stream data processing system that can be used in an embodiment of the present invention is schematically shown;

[0018] Figure 3 A Flink architecture diagram of a video stream data processing system that can be used in an embodiment of the present invention is schematically shown;

[0019] Figure 4 A processing flow chart of a video stream data processing system according to an embodiment of the present invention is schematically shown;

[0020] Figure 5 A resource utilization curve diagram of a video stream data processing system according to an embodiment of the present invention is schematically shown;

[0021] Figure 6 A diagram schematically shows the parallel processing performance of a video stream data processing system according to an embodiment of the present invention;

[0022] Figure 7 A flowchart of a method for processing video stream data according to an embodiment of the present invention is schematically shown;

[0023] Figure 8 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0024] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0025] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0026] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0028] In the process of implementing the present invention, the applicant found that the edge video stream real-time processing technology has at least the following difficulties:

[0029] 1. Edge computing resources are limited, and a single node can hardly meet the requirements. Video stream object detection applications have high processing real-time requirements, while a single node usually has limited computing, storage, and network resources, which makes it difficult to perform large-scale data processing tasks in an edge computing environment. This resource bottleneck leads to problems such as performance degradation, increased task latency, and limited application scalability.

[0030] 2. The variability of edge computing resources makes it difficult for distributed processing to succeed. The resources in the edge computing environment are highly dynamic and uncertain. Factors such as the online and offline of edge devices, fluctuations in network bandwidth, and changes in user needs will lead to dynamic changes in resources. This resource instability makes it very difficult to achieve effective distributed task processing in the edge environment. Traditional distributed task scheduling and execution models often cannot adapt to the dynamic changes of resources in the edge environment, resulting in inefficient task scheduling, low resource utilization, and large task execution delays.

[0031] In view of the above problems, the present invention provides a video stream data processing system and method. The video stream data processing system and method adopt lightweight container technology to realize resource sharing pooling and unified scheduling. By packaging the application and its dependencies into lightweight containers, rapid deployment and flexible expansion on edge devices can be achieved. This lightweight container technology can maximize the use of edge device resources and improve the flexibility and scalability of the system. At the same time, the video stream data processing system and method also monitor the resource utilization of edge devices in real time through task scheduling based on the streaming computing framework, and perform task scheduling and resource allocation according to the dynamic changes of resources. When the resources of the edge device change, the system can quickly formulate a scheduling plan, and ensure the performance and stability of the system through measures such as stream topology reconstruction, resource elastic scaling, and task migration recovery. This resource-aware task scheduling method can effectively cope with the challenges brought by resource variability in the edge computing environment and improve the system's task execution efficiency and resource utilization. By combining lightweight container technology with edge application task scheduling of streaming computing, it can effectively solve the resource limitations and dynamic problems existing in the current edge computing environment, improve the real-time processing throughput of the application and the system resource utilization, thereby meeting the high performance and low latency requirements of large-scale video stream processing application scenarios. The following is a detailed description with reference to specific embodiments.

[0032] Figure 1 The structure diagram of the video stream data processing system according to an embodiment of the present invention is schematically shown.

[0033] like Figure 1 As shown, the video stream data processing system may include an infrastructure layer 110 , a stream computing engine layer 120 , and a task coordination processing layer 130 .

[0034] The infrastructure layer 110 is used to build and manage edge computing nodes, on which lightweight container clusters are deployed.

[0035] The infrastructure layer 110, which can also be called the cloud infrastructure layer, is the foundation of the entire system and provides the basic resources and services required for the edge computing environment, such as physical servers, virtualization platforms, storage systems, etc. On the infrastructure layer 110, edge computing nodes can be built and managed to support the deployment and execution of edge computing tasks. Edge computing nodes can be computing units distributed on edge devices, responsible for receiving, processing and transmitting locally generated video stream data. These edge computing nodes can be located close to the data source and can use small, low-power, and moderately performing hardware devices, such as dedicated edge servers. Edge devices have certain computing and storage capabilities to process data from terminal devices.

[0036] A lightweight container cluster is deployed on the edge computing node. Lightweight container cluster deployment can use containerization technology, such as Docker, Kubernetes, etc. Containerization technology allows applications and their dependencies to be packaged into an independent container, which can run in any environment that supports the technology, thereby achieving rapid deployment and migration of applications.

[0037] Please continue reading Figure 1 ,The stream computing engine layer 120 is used to call the container cluster on the edge computing node to process the video stream data in parallel according to the task to be processed.

[0038] The stream computing engine layer 120 is the core part of the system. By using the parallel processing capability of the container cluster, the stream computing engine layer can process multiple video stream data tasks at the same time. The stream computing engine layer 120 can provide high-performance streaming data processing capabilities, can process video streams from edge devices, and analyze, process and respond in real time. Video stream data can be, for example, remote sensing video stream data. The tasks to be processed can be, for example, video stream recognition, target detection, etc.

[0039] The task collaborative processing layer 130 is used to adjust the allocation of tasks to be processed on the container cluster according to the resource utilization of the edge computing node when processing video stream data.

[0040] Through the above embodiments, the video stream data processing system adopts lightweight container technology to realize resource sharing pooling and unified scheduling. This lightweight container technology can maximize the use of edge device resources and improve the flexibility and scalability of the system. When processing video stream data in parallel based on the streaming computing framework, the task collaborative processing layer can monitor the resource utilization of edge devices in real time, and perform task scheduling and resource allocation according to the dynamic changes of resources. This resource-aware task scheduling method can effectively cope with the challenges brought by resource variability in the edge computing environment, improve the system's task execution efficiency and resource utilization, thereby meeting the high performance and low latency requirements of large-scale video stream processing application scenarios.

[0041] Based on the above embodiments, the edge computing node can deploy and manage the container cluster through the lightweight container orchestration tool. The lightweight container orchestration tool uses a lightweight database to store data of the container cluster and starts the container cluster through a binary file.

[0042] As an example, a lightweight container orchestration tool may be K3s, which is a lightweight Kubernetes distribution designed for edge computing scenarios. Figure 2 The following schematically shows a K3s container structure diagram of a video stream data processing system that can be used in an embodiment of the present invention. Figure 2As shown in the figure, the K3s master node (Server) can include a resource operation component (API Server), a lightweight database (SQLite), a resource manager (Controller Manager), and a scheduling module (Scheduler), while the slave node (Agent) includes a proxy component (Kubelet) for managing containers (Pod), an intra-group communication component (Flannel), and a Kube Proxy for managing service access portals. As a lightweight container orchestration tool, K3s has advantages in many aspects: first, it uses lightweight components and container runtimes, such as SQLite to replace traditional etcd, and Flannel as a network component, which greatly reduces resource consumption and is suitable for edge devices and environments; second, K3s maintains compatibility with the standard Kubernetes API and ecosystem, and can seamlessly migrate existing applications and workloads; finally, K3s provides a simplified deployment process, quickly starting the entire cluster through a single binary file, saving deployment time and improving the responsiveness and flexibility of the edge environment. In edge scenarios, K3s can be used in various application scenarios, including edge computing, edge data centers, and edge AI, providing an efficient and flexible containerized platform for edge environments.

[0043] Based on the above embodiment, the stream computing engine layer 120 may include a stream computing component and a stream computing framework, wherein the stream computing framework is used to receive tasks to be processed, and by integrating with the stream computing component, call a container cluster to process video stream data in parallel.

[0044] The streaming computing framework is the core component of the system, responsible for efficient processing and analysis of real-time data. The streaming computing framework has good fault tolerance and scalability, and can adapt to data loads of different sizes and changes. The streaming computing components include various real-time processing tasks, such as video decoding, feature extraction, target detection, etc. These components are customized according to actual application requirements, and through integration with the streaming computing framework, real-time processing and analysis of edge video streams can be achieved. Each streaming computing component is highly parallelized and scalable, and can make full use of the system's computing resources to achieve efficient real-time processing.

[0045] Based on the above embodiments, the streaming computing framework may include a job manager and at least two parallel task managers. The job manager is used to schedule and allocate tasks to be processed, and the task manager is used to receive and execute tasks to be processed allocated by the job manager, and exchange data with at least one other task manager when executing tasks to be processed.

[0046] As an example, the streaming computing framework may be a streaming computing framework based on Apache Flink. Figure 3The following schematically shows a Flink architecture diagram of a video stream data processing system that can be used in an embodiment of the present invention. Figure 3 As shown in the figure, the Flink architecture can be composed of a job manager and a task manager. For a job submitted for execution, the job manager is responsible for managing the scheduling, including the job controller, resource manager, and task distributor; the task manager is responsible for executing tasks to process data, and can have one or more, each containing multiple task slots. Slots can be registered to the resource manager, executing tasks assigned by the job controller, and exchanging data with other task managers.

[0047] Table 1 compares the Flink architecture with real-time stream data processing frameworks such as Apache Storm, Spark, and Samza.

[0048] Table 1

[0049]

[0050] Among them, Apache Storm is a distributed data processing framework that supports stream computing and performs data processing and analysis in the form of event streams. The Storm framework has two significant advantages in terms of latency and concurrency in stream data processing, but it also has disadvantages such as low throughput, high latency, slow response, and inability to meet real-time computing requirements. Spark is a distributed framework based on the MapReduce algorithm. The framework divides data into batch jobs in the form of segments according to time intervals before calculation. The disadvantage is that due to the memory-based operation, when the data volume is too large, the cluster performance will be unstable and the real-time performance will be poor. Samza is a stream data framework based on the publish-subscribe system, a higher-level abstraction of the message queue system, and implements a distributed stream data processing framework in an application mode on the message queue system. However, it can only guarantee one-time processing, which may cause data duplication.

[0051] It can be seen that Flink has significant advantages over the other three frameworks. Whether processing static data or dynamic data, Flink can perform stateful computing and process any data in the form of streams. At the same time, Flink can solve the challenges of distributed frameworks in data throughput, provide reliable data protection, and ensure that data will not be repeated or lost. It can maintain high throughput and achieve low-latency processing, which is difficult to achieve at the same time in other frameworks. At the same time, its close integration with resource managers (such as YARN, Kubernates, etc.) can simultaneously meet specific requirements such as large data scale, high real-time requirements, convenient business expansion, and post-failure recovery.

[0052] Please continue reading Figure 1In some embodiments, the task collaborative processing layer 130 may include a resource perception component, wherein the resource perception component is used to collect and analyze indicator data of the edge computing node when processing video stream data, and obtain resource utilization based on the indicator data.

[0053] For example, the resource perception component can monitor the CPU, memory, network bandwidth and other indicator data in the edge computing node, and then analyze and calculate these indicator data to obtain the resource utilization of the edge computing node. The resource perception component can provide real-time resource information by regularly collecting and analyzing the resource usage of edge nodes, helping the system to dynamically allocate and optimize resources.

[0054] As an example, in a container cluster, real-time resource awareness can rely on Prometheus to capture and store indicator data. As an open source monitoring and alarm system and time series database (TSDB), Prometheus is suitable for monitoring containerized environments, such as the above-mentioned K3s container cluster. The following is an exemplary process:

[0055] First, you can deploy Node-Exporter as a daemon process set on each container cluster node. i Run a Node-Exporter instance. Metrics collected by Node-Exporter For time series data:

[0056]

[0057] where t j It is a timestamp, which represents the time when the indicator data is collected. j is the corresponding indicator data, which is a real number type, representing the j The measured value of a metric at a point in time.

[0058] The indicators collected by Node-Exporter include but are not limited to: CPU, including CPU usage, usage of each core, etc.; memory: including memory usage, available amount, swap space usage, etc.; disk, including disk usage, I / O read and write rate, etc.; network, including the number of bytes received and sent by the network interface, the number of packets, etc.; system load, including the average load at different times such as 1 minute, 5 minutes, 15 minutes, etc. These indicators can provide cluster administrators with rich system performance data.

[0059] Next, the collected indicator data is exposed in text format through the HTTP service. The system periodically pulls monitoring data from each node running Node-Exporter according to the configuration. , for each node, Prometheus sends an HTTP GET request to node n every ∆t time i The / metrics path:

[0060]

[0061] Then, through the service discovery mechanism, Prometheus automatically registers the newly added nodes as monitoring targets, avoiding manual configuration updates. The set of nodes in the cluster at time t is N(t), and the target list dynamically maintained by Prometheus is:

[0062]

[0063] Please continue reading Figure 1 In some embodiments, the task collaborative processing layer 130 may further include a task scheduling component, wherein the task scheduling component is used to utilize the resource utilization obtained by the resource awareness component to adjust the dynamic allocation of tasks to be processed on the container cluster based on a preset scheduling algorithm to improve resource utilization.

[0064] The task scheduling component is responsible for dynamically scheduling tasks to optimize system performance based on system load and resource perception information. It uses a preset scheduling algorithm to dynamically allocate and schedule tasks, ensuring that the system can fully utilize available resources and achieve high throughput and low latency real-time data processing. The task scheduling component works closely with resource perception components, streaming computing frameworks and other components to achieve overall control and optimization of the system.

[0065] In some embodiments, the task scheduling component is also used to adjust the number and capacity of container clusters to achieve elastic scaling of resources of the container clusters.

[0066] In some embodiments, the task scheduling component can adjust the number and capacity of the container cluster by horizontal expansion and / or vertical expansion.

[0067] Horizontal expansion can adjust system capacity by increasing or decreasing the number of container instances. For example, let the current number of container instances be C and the target number of instances be C. * The controller regularly monitors the CPU and memory utilization of the container (CPU_util(t), Mem_util(t)) and calculates the target number of replicas based on predefined rules:

[0068]

[0069] Then, automatically update the replica set object:

[0070]

[0071] Vertical scaling can optimize performance by adjusting the resource configuration of a single container. For example, let the current container resource configuration be R and the target resource configuration be R * ,The target resource configuration calculation formula is:

[0072]

[0073] Where g is a function that generates resource allocation recommendations.

[0074] Then, adjust the container resource configuration to match your goals:

[0075]

[0076] When the resource configuration is updated, the container needs to be restarted to apply the new configuration:

[0077]

[0078] Through the above operations, you can achieve elastic scaling of container cluster resources, ensuring that the system can maintain the best state when processing a large number of concurrent requests and optimizing the performance of a single container.

[0079] In some embodiments, the preset scheduling algorithm may include a flow topology collaborative scheduling method, wherein the flow topology collaborative scheduling method adjusts the structure of the flow topology according to resource utilization to adjust the allocation of tasks to be processed on the container cluster.

[0080] As an example, the flow topology collaborative scheduling method may include two parts: flow topology construction and task scheduling algorithm:

[0081] (1) Flow topology construction

[0082] First, the task scheduling scenario is modeled based on the CPU and memory required by the subtask and the idle CPU and memory of the system monitored by K3s. It can be modeled as a data structure of a directed acyclic weighted graph G=(V,E). Node V represents the subtask, edge E represents the dependency, and weight w(e) represents the data transmission volume. The node attributes (CPU, memory) are expressed as:

[0083]

[0084] Then, the improved GraphSAGE framework is used for task feature classification, which includes three steps: ① Sampling neighbor vertices: a fixed number S of neighbor nodes of each node v are sampled, with replacement sampling when less than S, and without replacement sampling when more than S. ② Aggregating neighbor information: Select LSTM as the aggregation function, randomly sort the neighbors first, and then use their embedding as LSTM input; ③ Generate vertex vectors: Generate feature vectors of all nodes through k-times aggregation for downstream tasks. The formula can be expressed as:

[0085]

[0086] in is the feature vector of node v after the kth gathering.

[0087] (2) Task Scheduling Algorithm

[0088] First, a streaming data processing task allocation algorithm based on resource status is implemented through the Graph Attention Network-Advantage Actor Critic algorithm (GAT-A2C). Given a set of tasks and resource representations, the scheduling scheme gives a mapping between subtasks and computing nodes.

[0089]

[0090] Then, the tasks and resources containing features are represented as graph data to facilitate the extraction of features by the graph convolutional network. Then, the A2C algorithm is used to solve the task scheduling optimization problem. In order to implement the A2C scheduling algorithm optimization strategy, the task scheduling process should be established as a Markov decision process (MDP) consisting of state space, action space and reward:

[0091]

[0092] Where S represents the state space of the problem, A represents the action space, and P represents the state transition probability, which is defined as , which means that when action a is known t In the case of state from s t Convert to s t+1 The probability of . R represents the reward function and γ represents the discount factor.

[0093] Through the design of the above key steps, it is possible to ensure that the video stream data processing system can efficiently perform task scheduling and resource allocation in the edge computing environment, cope with dynamic changes in resources, and improve the system's task execution efficiency and resource utilization. At the same time, these designs also provide a solid foundation for the scalability and flexibility of the video stream data processing system, enabling it to adapt to changes in different scenarios and application requirements.

[0094] On the basis of the above embodiments, the video stream data processing system may further include an application component layer. The application component layer may include specific application components and business logic for implementing specific edge video stream processing applications. These application components may provide specific application services based on the functions provided by the stream computing engine, including providing specific business functions and service interfaces. Taking remote sensing data streams as an example, application services may include applications such as image recognition and target detection based on real-time video streams of remote sensing data. These application services realize the analysis and processing of real-time data through integration with streaming computing components and task scheduling services, and provide high-value application functions and service support for users and other external systems.

[0095] Based on the above embodiments, the present invention constructs an example of a video stream data processing system. Figure 4 The processing flow of the video stream data processing system is described. Figure 4 The processing flow chart of the video stream data processing system according to an embodiment of the present invention is schematically shown.

[0096] like Figure 4 As shown, in this embodiment, the video stream data processing system includes components such as a container cloud platform, a cluster management terminal, a resource perception module, and a task scheduling module. The container cloud platform can be used as Figure 1 The infrastructure layer 110, the cluster management terminal can be used as the streaming video framework layer 120, and the resource perception module and the task scheduling module can be used as the task collaborative processing layer 130. The processing flow of the entire system may include the following steps:

[0097] First, before receiving user tasks, the video stream data processing system can be initialized. For example, the node container cluster in the container cloud platform can be initialized to check the available resources of the edge device, including computing power, storage space, and network bandwidth. At the same time, the job scheduling algorithm plug-in and resource monitoring algorithm plug-in are loaded in the task collaborative processing layer to ensure that the system can accurately monitor and manage resource utilization, providing a solid foundation for subsequent task scheduling and resource allocation.

[0098] Next, the task submitted by the user is received. The user uploads the application through the interface or API and submits the video stream data processing task to the job manager. The user can specify the specific requirements of the application in detail, such as the required computing resources, memory requirements, etc. After receiving the task, the job manager passes the task information to the subsequent process and prepares for task scheduling and resource allocation.

[0099] Next, create the task flow topology. Based on the real-time resource perception results of the resource perception service, the pre-loaded scheduling algorithm can be called to generate the flow topology. In this process, the system can dynamically adjust the structure of the flow topology according to the current resource status of the edge device to maximize the use of the resources of the edge device and ensure the smooth execution of the task. In this process, the system can also consider the dependencies and priorities between tasks to ensure the stability and performance of the overall system.

[0100] Then, the computing resources are allocated. Based on the generated flow topology, the system can allocate computing resources. The system can comprehensively consider the role and current load of each node and allocate appropriate computing resources, storage space, and network bandwidth to each task. In this process, the system can flexibly adjust the resource allocation strategy according to the nature and requirements of the task to maximize the efficiency and performance of the system.

[0101] Next, start the application. Once the resource allocation is complete, the task manager of the corresponding node receives the application executable and starts the application. The job manager is responsible for managing the running status of the application, monitoring the execution progress of the task, and handling any abnormal situations. After the application is started, the tasks will start to execute in the specified order of the stream topology, processing the video stream data on the edge device in real time.

[0102] Finally, the task is completed. When the task is completed, the job manager returns the processing results to the application. At the same time, the system can perform necessary cleanup work, release used resources, and prepare to receive new tasks. This stage can also involve the aggregation and storage of results for subsequent data analysis and application.

[0103] Experimental data comparison

[0104] The video stream data processing system of the present invention is described below in conjunction with specific embodiments. Figure 5 The resource utilization curve diagram of the video stream data processing system according to an embodiment of the present invention is schematically shown. Figure 6 The figure schematically shows the parallel processing performance of the video stream data processing system according to the embodiment of the present invention.

[0105] This specific embodiment builds a simulation environment based on the pytorch network framework, and compares the minimum system configuration requirements of the lightweight architecture based on K3s+Flink (the solution of the present invention) and the traditional architecture based on K8s+Flink (the comparison solution). The CPU, memory, and storage requirements of K3s are significantly smaller than those of K8s, reflecting the lightweight advantage of this system. At the same time, by setting the CPU threshold, a curve of the system CPU usage changing over time is plotted to verify the system's elastic scaling and rescheduling capabilities, ensuring that the system can make efficient resource utilization. Finally, the system is tested using video stream data, and by recording the video stream processing time and the number of image processing frames per unit time under different degrees of parallelism, it is proved that the system has achieved efficient parallel stream processing capabilities based on Flink. The following is the specific comparison content:

[0106] The underlying hardware facilities of the simulation environment system consist of 4 x86_64 116c32G Cent0S7.8 cloud hosts and 2 x86_64 80c250G Cent0S7.8 physical machines. A lightweight container cluster is running on these physical facilities as the underlying platform of the system. In the process of building the system, a lightweight container cluster consisting of the application container engine Docker, the container cluster rapid orchestration software docker-compose, the private image warehouse Harbor, the shared storage NFS, and the database MySQL was installed and deployed. The system's front-end and back-end business containers, Prometheus monitoring-related containers, and the Flink task containers and subtask containers of the target detection application and the scheduling algorithm containers are running on the cluster.

[0107] Combining K3s and Flink provides significant advantages in edge computing scenarios. The lightweight nature of K3s makes it suitable for deploying Flink tasks on edge nodes, enabling real-time data processing and analysis. In a resource-constrained environment, the system can better utilize resources while reducing deployment and maintenance costs. As shown in the table below, the system based on K3s saves 50% in CPU and memory compared to K8s, and reduces storage costs by about 87.5%.

[0108] Table 2

[0109]

[0110] like Figure 5As shown in the figure, the system's CPU threshold is set to 80%. When the system's CPU usage reaches this threshold, the system automatically triggers the elastic scaling and rescheduling mechanism. This means that the current system resources and task scheduling will be re-evaluated and allocated to ensure the best use of system resources and efficient task execution. The experimental results show that after the system reaches the CPU usage threshold, it can quickly and effectively scale up and reschedule, and after multiple scaling up and down, the system CPU usage is stable at around 60%.

[0111] By combining image processing tasks with Flink and leveraging Flink’s powerful parallel computing capabilities, we can achieve efficient processing and analysis of large-scale image data. By optimizing the data loading and processing process into batch tasks, the overall processing efficiency is significantly improved, especially when processing large-scale data. Figure 6 As shown in the figure, the experiment uses a video stream dataset. When the degree of parallelism is set to 1, 2, 4, and 8, the system processing time is 337.5, 163.8, 80.0, and 41.7 minutes respectively, and the number of processed frames is 0.75, 1.70, 3.04, and 5.51 frames respectively, which is close to linear expansion, and the fitting value is about 0.9. This shows that with the increase of parallelism, the system's video stream processing time is significantly reduced, and the number of images processed per unit time is significantly increased. The system has good concurrent data processing performance.

[0112] Through the above simulation experiments, it can be seen that the performance of the system under different degrees of parallelism verifies that the system has the advantages of efficient task scheduling and lightweight deployment. This study not only demonstrates the application prospects of deep reinforcement learning scheduling algorithms in complex environments, but also provides methods to optimize resource utilization and improve processing efficiency in edge computing scenarios.

[0113] In summary, the present invention utilizes lightweight container technology, such as K3s lightweight container, to package applications and their dependencies into lightweight containers, thereby achieving rapid deployment and flexible expansion on edge devices, making maximum use of edge device resources, and improving the flexibility and scalability of the system. The present invention proposes a resource-aware task scheduling mechanism that can monitor the resource utilization of edge devices in real time, and perform task scheduling and resource allocation according to the dynamic changes of resources. This method can achieve effective distributed task processing in an edge environment, cope with dynamic changes in resources, and improve the task execution efficiency and resource utilization of the system. The present invention integrates Flink with K3s, utilizes Flink's powerful stream processing capabilities to process massive real-time data, and realizes real-time processing of edge video streams. Flink's parallel processing mechanism can decompose tasks into multiple subtasks and execute them in parallel on different computing nodes, thereby significantly improving the efficiency and speed of data processing.

[0114] Based on the above video stream data processing system, the present invention also provides a video stream data processing method. Figure 7 The flowchart of the video stream data processing method according to the embodiment of the present invention is schematically shown.

[0115] like Figure 7 As shown, the video stream data processing method may include steps S710 to S730.

[0116] In step S710, video stream data and tasks to be processed are received.

[0117] In step S720, according to the task to be processed, the container cluster on the edge computing node is called to process the video stream data in parallel.

[0118] In step S730, the allocation of tasks to be processed on the container cluster is adjusted according to the resource utilization rate of the edge computing node when processing the video stream data.

[0119] It should be emphasized that the above-mentioned video stream data processing method and the above-mentioned video stream data processing system have the same technical features and beneficial effects, which will not be repeated here.

[0120] The present invention also provides an electronic device, Figure 8 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present invention is schematically shown.

[0121] like Figure 8 As shown, the electronic device 800 according to an embodiment of the present invention includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 to a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory configured for cache purposes. The processor 801 may include a single processing unit or multiple processing units configured to perform different actions of the method flow according to an embodiment of the present invention.

[0122] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other via bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in one or more memories.

[0123] According to an embodiment of the present invention, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the input / output (I / O) interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 809 including a network interface card such as a LAN card, a modem, etc. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that the computer program read therefrom is installed into the storage portion 808 as needed.

[0124] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code configured to execute the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0125] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0126] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0127] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 802 and / or the RAM 803 described above and / or one or more memories other than the ROM 802 and the RAM 803 .

[0128] An embodiment of the present invention also includes a computer program product, which includes a computer program, which contains program code configured to execute the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is configured to enable the electronic device to implement the training method and detection method provided by the embodiment of the present invention.

[0129] When the computer program is executed by the processor 801, the above functions defined in the system / device of the embodiment of the present invention are executed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0130] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 809, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0131] According to an embodiment of the present invention, the program code configured to execute the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0132] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions configured to implement the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All these combinations and / or combinations fall within the scope of the present invention.

[0133] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A video stream data processing system, It is characterized in that include: The infrastructure layer is used to build and manage edge computing nodes, on which lightweight container clusters are deployed; The stream computing engine layer is used to call the container cluster on the edge computing node to process the video stream data in parallel according to the task to be processed; The task collaborative processing layer is used to adjust the allocation of the tasks to be processed on the container cluster according to the resource utilization of the edge computing node when processing the video stream data.

2. The video stream data processing system according to claim 1, It is characterized in that The edge computing node deploys and manages the container cluster through a lightweight container orchestration tool, wherein the lightweight container orchestration tool uses a lightweight database to store data of the container cluster and starts the container cluster through a binary file.

3. The video stream data processing system according to claim 1 or 2, It is characterized in that The stream computing engine layer includes a stream computing component and a stream computing framework, wherein the stream computing framework is used to receive the task to be processed, and by integrating with the stream computing component, call the container cluster to process the video stream data in parallel.

4. The video stream data processing system according to claim 3, It is characterized in that The streaming computing framework includes a job manager and at least two parallel task managers. The job manager is used to schedule and allocate the tasks to be processed. The task manager is used to receive and execute the tasks to be processed allocated by the job manager, and exchange data with at least one other task manager when executing the tasks to be processed.

5. The video stream data processing system according to claim 1, It is characterized in that The task collaborative processing layer includes a resource perception component, wherein the resource perception component is used to collect and analyze indicator data of the edge computing node when processing the video stream data, and obtain the resource utilization based on the indicator data.

6. The video stream data processing system according to claim 5, It is characterized in that The task collaborative processing layer also includes a task scheduling component, wherein the task scheduling component is used to use the resource utilization obtained by the resource perception component to adjust the dynamic allocation of the tasks to be processed on the container cluster based on a preset scheduling algorithm to improve the resource utilization.

7. The video stream data processing system according to claim 6, It is characterized in that The scheduling algorithm includes a flow topology collaborative scheduling method, wherein the flow topology collaborative scheduling method adjusts the structure of the flow topology according to the resource utilization to adjust the allocation of the to-be-processed tasks on the container cluster.

8. The video stream data processing system according to claim 6, It is characterized in that The task scheduling component is also used to adjust the number and capacity of the container cluster to achieve elastic scaling of resources of the container cluster.

9. The video stream data processing system according to claim 8, It is characterized in that The task scheduling component adjusts the quantity and capacity of the container cluster by horizontal expansion and / or vertical expansion.

10. A method for processing video stream data, It is characterized in that include: Receive video stream data and tasks to be processed; According to the tasks to be processed, calling the container cluster on the edge computing node to process the video stream data in parallel; According to the resource utilization rate of the edge computing node when processing the video stream data, the allocation of the to-be-processed tasks on the container cluster is adjusted.

Images