A parallel optimization system for mass video processing

Abstract

This invention discloses a parallel optimization system for massive video processing. The system adopts a four-layer integrated parallel and collaborative processing architecture, comprising, from top to bottom: a parallel task layer for video stream access, grouping, splitting, encapsulation, and queue management; a resource abstraction layer for unified hardware modeling, status acquisition, topology construction, and capability assessment; a parallel scheduling layer for task-hardware matching, load balancing, priority scheduling, and dynamic adjustment; and an execution optimization layer for data stream localization, parallel read / write, cache optimization, and zero-copy processing. The parallel task layer, resource abstraction layer, parallel scheduling layer, and execution optimization layer form a complete parallel processing link: task input, resource awareness, accurate scheduling, and optimized execution. This invention implements concurrent rate limiting and smooth access for the input video stream to prevent traffic surges; and sets synchronization points and timing control for parallel tasks to ensure orderly output.

Description

Technical Field

[0001] This invention belongs to the field of video data processing technology, and is mainly aimed at the needs of concurrent access, real-time decoding and parallel processing of massive high-definition videos in scenarios such as security monitoring, smart cities and traffic control. Specifically, it relates to a parallel optimization system for massive video processing. Background Technology

[0002] With the widespread adoption of video capture equipment, a single system needs to handle the parallel processing of thousands of high-definition video streams, placing extremely high demands on system throughput, concurrency, and hardware utilization. Currently, massive video processing generally relies on parallel decoding using multiple hardware components, but this approach suffers from problems such as inefficient parallel scheduling, redundant data flow, excessive CPU load, and unbalanced load, making it difficult to meet the business requirements of high concurrency, high stability, and low latency.

[0003] Existing massive video processing systems employ a multi-accelerator card parallel decoding architecture. Through the operating system's default scheduling mechanism, video stream tasks are randomly assigned to various hardware acceleration cards. Video data is uniformly scheduled and forwarded by the CPU, using a single-node memory access mode. Task distribution is allocated with fixed granularity, unaware of real-time hardware load, with no collaborative scheduling between hardware components, and no targeted optimization for data reading, writing, and transmission.

[0004] The above-mentioned existing technologies have the following disadvantages:

[0005] (1) There is no unified scheduling for multi-hardware parallel processing, the task allocation is uneven, some hardware is fully loaded and some hardware is idle, resulting in low parallel efficiency.

[0006] (2) Data flow lacks topology awareness, cross-node data transfer is frequent, CPU utilization is too high, which restricts the parallel processing performance of the whole machine.

[0007] (3) Parallel processing capability cannot be linearly expanded in high-concurrency scenarios, and performance improvement is not significant after adding hardware.

[0008] (4) Lack of parallel data flow optimization, low data read and write efficiency, high system latency, and easy to cause stuttering and frame loss. Summary of the Invention

[0009] This invention provides a parallel optimization system for massive video processing to solve the following technical problems (1) to (4):

[0010] (1) Solve the problem of uneven distribution of multiple hardware tasks and low parallel efficiency in the parallel processing of massive video.

[0011] (2) Solve the problems of unreasonable data flow and excessive CPU usage during parallel processing.

[0012] (3) Solve the problems of insufficient throughput and poor stability of video parallel processing under high concurrency.

[0013] (4) Solve the problem that parallel processing performance cannot be linearly improved after hardware expansion.

[0014] To achieve the above objectives, this invention provides a parallel optimization system for massive video processing. This system employs a four-layer integrated parallel and collaborative processing architecture, comprising, from top to bottom:

[0015] Parallel task layer: used to complete video stream access, grouping, splitting, encapsulation and queue management;

[0016] Resource Abstraction Layer: Used to complete unified hardware modeling, status acquisition, topology construction, and capability assessment;

[0017] Parallel scheduling layer: used to perform task matching, load balancing, priority scheduling, and dynamic adjustment; and

[0018] Execution optimization layer: used to perform data stream localization, parallel read and write, cache optimization, and zero-copy processing;

[0019] The parallel task layer, resource abstraction layer, parallel scheduling layer, and execution optimization layer form a complete parallel processing link with task input, resource knowledge, scheduling accuracy, and execution optimization.

[0020] In one embodiment of the present invention, optionally, the parallel task layer includes a parallel task construction and standardization module, which is the input source of the parallel processing link and provides standard, ordered, and parallelizable task units for subsequent scheduling. The parallel task construction and standardization module performs the following (1) to (4):

[0021] (1) Perform bitstream parsing, format verification, resolution / frame rate / bitrate identification on massive video streams and remove abnormal streams;

[0022] (2) Classify video streams into multiple levels according to channel, service type and real-time level to form mutually independent parallel task groups;

[0023] (3) Divide the task group into the smallest parallelizable decoding unit and uniformly encapsulate the task ID, resource requirements, priority, and time limit requirements;

[0024] (4) Construct a multi-level priority parallel task queue to ensure that high-priority tasks enter the scheduling process first.

[0025] In one embodiment of the present invention, optionally, the resource abstraction layer includes a heterogeneous hardware unified abstraction and global awareness module. The heterogeneous hardware unified abstraction and global awareness module serves as the decision basis for parallel scheduling and provides the parallel scheduling layer with a real-time, accurate, and complete hardware view. The heterogeneous hardware unified abstraction and global awareness module performs the following (1) to (4):

[0026] (1) Unify the abstract modeling of all decoding hardware in the system, shield the hardware differences, and form a standardized resource pool;

[0027] (2) Collect hardware load, bandwidth, temperature, number of available channels, and processing capacity in real time to generate a global resource status table;

[0028] (3) Automatically identify the physical topology relationship between hardware and CPU, memory and bus, and establish a hardware-node topology mapping table;

[0029] (4) Classify and label the hardware capabilities to provide a basis for optimal task allocation.

[0030] In one embodiment of the present invention, optionally, the parallel scheduling layer includes a global parallel scheduling and dynamic load balancing module, which connects the parallel task layer and the execution optimization layer, determines the parallel efficiency, and performs the following (1) to (4):

[0031] (1) Perform task-hardware optimal matching calculation based on task attributes and hardware status;

[0032] (2) With the goal of global load balancing, tasks are evenly distributed to each hardware to avoid local overload;

[0033] (3) Supports priority preemption and dynamic scheduling, and high real-time tasks can be allocated resources first;

[0034] (4) Monitor hardware load in real time, dynamically migrate overloaded tasks, and always maintain the best parallel efficiency.

[0035] In one embodiment of the present invention, optionally, the parallel scheduling layer includes a parallel data flow topology optimization module. The parallel data flow topology optimization module is a key module that determines performance improvement and directly solves the technical problems of high CPU usage, high latency, and low throughput. The parallel data flow topology optimization module performs the following (1) to (4):

[0036] (1) Based on the topology mapping table, establish hardware-memory node-CPU core affinity binding;

[0037] (2) Force local data processing first, and minimize data transfer across nodes and buses;

[0038] (3) Adopt parallel read / write, multi-level caching, data prefetching, and zero-copy / memory sharing technologies to reduce bandwidth pressure;

[0039] (4) Optimize the data flow path so that the decoding calculation is completed with the shortest path and the least overhead.

[0040] In one embodiment of the present invention, optionally, the execution optimization layer includes a high-concurrency parallel control and stability guarantee module, which ensures that the system does not crash, lose frames, or run out of order under full load and is scalable. The high-concurrency parallel control and stability guarantee module performs the following (1) to (4):

[0041] (1) Implement concurrent rate limiting and smooth access for the input video stream to prevent traffic surges;

[0042] (2) Set synchronization points and timing control for parallel tasks to ensure orderly output;

[0043] (3) Automatic task migration in case of hardware failure to achieve self-healing;

[0044] (4) Supports hardware hot expansion and automatic management, and the parallel capability is linearly improved after expansion.

[0045] The parallel optimization system for massive video processing provided by this invention has the following beneficial technical effects:

[0046] (1) Significantly improved parallel efficiency: The unified scheduling and balancing mechanism enables the hardware utilization rate to approach full load, and the overall parallel throughput of the machine is greatly improved;

[0047] (2) CPU usage is significantly reduced: Data flow localization and path optimization reduce cross-node data transfer, resulting in a significant decrease in CPU usage;

[0048] (3) More stable under high concurrency: The rate limiting, synchronization and self-healing mechanisms ensure that there is no lag, no frame drop and no out-of-order when the load is full;

[0049] (4) Enhanced scalability: Hardware hot expansion can achieve a linear increase in parallelization capability;

[0050] (5) Lower processing latency: Parallel read and write, cache optimization, and zero copy significantly reduce end-to-end latency;

[0051] (6) Enhanced system compatibility: The unified hardware abstraction layer can adapt to different types of accelerator cards, resulting in low deployment costs. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart of a parallel optimization system for massive video processing according to an embodiment of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] This invention proposes an integrated parallel optimization architecture for massive video, with unified scheduling, hierarchical parallelism, topology awareness, local processing, and dynamic balancing as its core. It forms a complete closed-loop system of task construction → resource awareness → parallel scheduling → data flow optimization → stable execution → capacity expansion guarantee. The technical solution is highly structured and logically rigorous.

[0056] Figure 1 This is a flowchart of a parallel optimization system for massive video processing according to an embodiment of the present invention, as shown below. Figure 1 As shown, this invention provides a parallel optimization system for massive video processing. The system employs a four-layer integrated parallel and collaborative processing architecture, comprising, from top to bottom:

[0057] Parallel task layer: used to complete video stream access, grouping, splitting, encapsulation and queue management;

[0058] Resource Abstraction Layer: Used to complete unified hardware modeling, status acquisition, topology construction, and capability assessment;

[0059] Parallel scheduling layer: used to perform task matching, load balancing, priority scheduling, and dynamic adjustment; and

[0060] Execution optimization layer: used to perform data stream localization, parallel read and write, cache optimization, and zero-copy processing;

[0061] The parallel task layer, resource abstraction layer, parallel scheduling layer, and execution optimization layer form a complete parallel processing link with task input, resource knowledge, scheduling accuracy, and execution optimization.

[0062] In one embodiment of the present invention, optionally, the parallel task layer includes a parallel task construction and standardization module, which is the input source of the parallel processing link and provides standard, ordered, and parallelizable task units for subsequent scheduling. The parallel task construction and standardization module performs the following (1) to (4):

[0063] (1) Perform bitstream parsing, format verification, resolution / frame rate / bitrate identification on massive video streams and remove abnormal streams;

[0064] (2) Classify video streams into multiple levels according to channel, service type and real-time level to form mutually independent parallel task groups;

[0065] (3) Divide the task group into the smallest parallelizable decoding unit and uniformly encapsulate the task ID, resource requirements, priority, and time limit requirements;

[0066] (4) Construct a multi-level priority parallel task queue to ensure that high-priority tasks enter the scheduling process first.

[0067] In one embodiment of the present invention, optionally, the resource abstraction layer includes a heterogeneous hardware unified abstraction and global awareness module. The heterogeneous hardware unified abstraction and global awareness module serves as the decision basis for parallel scheduling and provides the parallel scheduling layer with a real-time, accurate, and complete hardware view. The heterogeneous hardware unified abstraction and global awareness module performs the following (1) to (4):

[0068] (1) Unify the abstract modeling of all decoding hardware in the system, shield the hardware differences, and form a standardized resource pool;

[0069] (2) Collect hardware load, bandwidth, temperature, number of available channels, and processing capacity in real time to generate a global resource status table;

[0070] (3) Automatically identify the physical topology relationship between hardware and CPU, memory and bus, and establish a hardware-node topology mapping table;

[0071] (4) Classify and label the hardware capabilities to provide a basis for optimal task allocation.

[0072] In one embodiment of the present invention, optionally, the parallel scheduling layer includes a global parallel scheduling and dynamic load balancing module, which connects the parallel task layer and the execution optimization layer, determines the parallel efficiency, and performs the following (1) to (4):

[0073] (1) Perform task-hardware optimal matching calculation based on task attributes and hardware status;

[0074] (2) With the goal of global load balancing, tasks are evenly distributed to each hardware to avoid local overload;

[0075] (3) Supports priority preemption and dynamic scheduling, and high real-time tasks can be allocated resources first;

[0076] (4) Monitor hardware load in real time, dynamically migrate overloaded tasks, and always maintain the best parallel efficiency.

[0077] In one embodiment of the present invention, optionally, the parallel scheduling layer includes a parallel data flow topology optimization module. The parallel data flow topology optimization module is a key module that determines performance improvement and directly solves the technical problems of high CPU usage, high latency, and low throughput. The parallel data flow topology optimization module performs the following (1) to (4):

[0078] (1) Based on the topology mapping table, establish hardware-memory node-CPU core affinity binding;

[0079] (2) Force local data processing first, and minimize data transfer across nodes and buses;

[0080] (3) Adopt parallel read / write, multi-level caching, data prefetching, and zero-copy / memory sharing technologies to reduce bandwidth pressure;

[0081] (4) Optimize the data flow path so that the decoding calculation is completed with the shortest path and the least overhead.

[0082] In one embodiment of the present invention, optionally, the execution optimization layer includes a high-concurrency parallel control and stability guarantee module, which ensures that the system does not crash, lose frames, or run out of order under full load and is scalable. The high-concurrency parallel control and stability guarantee module performs the following (1) to (4):

[0083] (1) Implement concurrent rate limiting and smooth access for the input video stream to prevent traffic surges;

[0084] (2) Set synchronization points and timing control for parallel tasks to ensure orderly output;

[0085] (3) Automatic task migration in case of hardware failure to achieve self-healing;

[0086] (4) Supports hardware hot expansion and automatic management, and the parallel capability is linearly improved after expansion.

[0087] The parallel optimization system for massive video processing provided by this invention has the following beneficial technical effects:

[0088] (1) Significantly improved parallel efficiency: The unified scheduling and balancing mechanism enables the hardware utilization rate to approach full load, and the overall parallel throughput of the machine is greatly improved;

[0089] (2) CPU usage is significantly reduced: Data flow localization and path optimization reduce cross-node data transfer, resulting in a significant decrease in CPU usage;

[0090] (3) More stable under high concurrency: The rate limiting, synchronization and self-healing mechanisms ensure that there is no lag, no frame drop and no out-of-order when the load is full;

[0091] (4) Enhanced scalability: Hardware hot expansion can achieve a linear increase in parallelization capability;

[0092] (5) Lower processing latency: Parallel read and write, cache optimization, and zero copy significantly reduce end-to-end latency;

[0093] (6) Enhanced system compatibility: The unified hardware abstraction layer can adapt to different types of accelerator cards, resulting in low deployment costs.

[0094] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0095] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A parallel optimization system for massive video processing, characterized in that, The system adopts a four-layer integrated parallel and collaborative processing architecture, which includes, from top to bottom: Parallel task layer: used to complete video stream access, grouping, splitting, encapsulation and queue management; Resource Abstraction Layer: Used to complete unified hardware modeling, status acquisition, topology construction, and capability assessment; Parallel scheduling layer: used to perform task matching, load balancing, priority scheduling, and dynamic adjustment; and Execution optimization layer: used to perform data stream localization, parallel read and write, cache optimization, and zero-copy processing; The parallel task layer, resource abstraction layer, parallel scheduling layer, and execution optimization layer form a complete parallel processing link with task input, resource knowledge, scheduling accuracy, and execution optimization.

2. The parallel optimization system for massive video processing according to claim 1, characterized in that, The parallel task layer includes a parallel task construction and standardization module, which is the input source of the parallel processing link and provides standard, ordered, and parallelizable task units for subsequent scheduling. The parallel task construction and standardization module performs the following (1) to (4): (1) Perform bitstream parsing, format verification, resolution / frame rate / bitrate identification on massive video streams and remove abnormal streams; (2) Classify video streams into multiple levels according to channel, service type and real-time level to form mutually independent parallel task groups; (3) Divide the task group into the smallest parallelizable decoding unit and uniformly encapsulate the task ID, resource requirements, priority, and time limit requirements; (4) Construct a multi-level priority parallel task queue to ensure that high-priority tasks enter the scheduling process first.

3. The parallel optimization system for massive video processing according to claim 1, characterized in that, The resource abstraction layer includes a heterogeneous hardware unified abstraction and global awareness module. The heterogeneous hardware unified abstraction and global awareness module is the decision basis for parallel scheduling and provides a real-time, accurate and complete hardware view for the parallel scheduling layer. The heterogeneous hardware unified abstraction and global awareness module performs the following (1) to (4): (1) Unify the abstract modeling of all decoding hardware in the system, shield the hardware differences, and form a standardized resource pool; (2) Collect hardware load, bandwidth, temperature, number of available channels, and processing capacity in real time to generate a global resource status table; (3) Automatically identify the physical topology relationship between hardware and CPU, memory and bus, and establish a hardware-node topology mapping table; (4) Classify and label the hardware capabilities to provide a basis for optimal task allocation.

4. The parallel optimization system for massive video processing according to claim 1, characterized in that, The parallel scheduling layer includes a global parallel scheduling and dynamic load balancing module. The global parallel scheduling and dynamic load balancing module connects the parallel task layer and the execution optimization layer, and determines the parallel efficiency. The global parallel scheduling and dynamic load balancing module performs the following (1) to (4): (1) Perform task-hardware optimal matching calculation based on task attributes and hardware status; (2) With the goal of global load balancing, tasks are evenly distributed to each hardware to avoid local overload; (3) Supports priority preemption and dynamic scheduling, and high real-time tasks can be allocated resources first; (4) Monitor hardware load in real time, dynamically migrate overloaded tasks, and always maintain the best parallel efficiency.

5. The parallel optimization system for massive video processing according to claim 1, characterized in that, The parallel scheduling layer includes a parallel data flow topology optimization module, which is the key module that determines performance improvement. It directly solves the technical problems of high CPU usage, high latency, and low throughput. The parallel data flow topology optimization module performs the following (1) to (4): (1) Based on the topology mapping table, establish hardware-memory node-CPU core affinity binding; (2) Force local data processing first, and minimize data transfer across nodes and buses; (3) Adopt parallel read / write, multi-level caching, data prefetching, and zero-copy / memory sharing technologies to reduce bandwidth pressure; (4) Optimize the data flow path so that the decoding calculation is completed with the shortest path and the least overhead.

6. The parallel optimization system for massive video processing according to claim 1, characterized in that, The execution optimization layer includes a high-concurrency parallel control and stability guarantee module. The high-concurrency parallel control and stability guarantee module ensures that the system does not crash, drop frames, or cause out-of-order execution under full load and is scalable. The high-concurrency parallel control and stability guarantee module executes the following (1) to (4): (1) Implement concurrent rate limiting and smooth access for the input video stream to prevent traffic surges; (2) Set synchronization points and timing control for parallel tasks to ensure orderly output; (3) Automatic task migration in case of hardware failure, achieving self-healing of faults; (4) Supports hardware hot expansion and automatic management, and the parallel capability is linearly improved after expansion.

Images