Video stream adaptive low-latency real-time transmission method based on edge computing

Abstract

The application discloses a video stream adaptive low-delay real-time transmission method and system based on edge computing, and the method comprises the following steps: receiving a real-time video stream from a network camera, creating a pipe queue, adding timestamp information to each video frame, and setting a queue protection mechanism; taking out the encoded video frame from the input queue, using hardware acceleration decoding to improve the processing of the video frame; monitoring the system resource usage in real time; performing adaptive frame skipping decision according to the performance monitoring result; generating a timestamp, dynamically calculating the timestamp interval according to the actual processing frame rate; receiving the decoded original video frame and the corresponding timestamp information, using hardware acceleration decoding to perform a video encoding operation; packaging and transmitting the encoded video data, and providing a standard protocol interface to connect a client for playing. The scheme can maintain stable low delay and low resource occupation, and significantly improve the video quality.

Description

Technical Field

[0001] This invention relates to the fields of video stream processing and edge computing technology, and in particular to a method and system for adaptive low-latency real-time transmission of video streams based on edge computing. Background Technology

[0002] With the rapid development of real-time video applications such as remote control, immersive interaction, industrial vision inspection, and automated driving, achieving stable low-latency transmission in edge computing environments with limited computing resources and dynamically changing network conditions has become a key technical challenge. Current mainstream solutions are mainly based on fixed frame processing mechanisms, static parameter configuration, or local optimization strategies. When faced with dynamic loads and heterogeneous hardware conditions in edge computing environments, it is difficult to effectively balance the multiple constraints of latency, quality, and resource efficiency.

[0003] (1) Latency accumulation problem caused by fixed frame rate processing: Existing technologies generally use fixed frame rate or static pipeline architecture for data processing and transmission, lacking a dynamic adjustment mechanism based on real-time computing capabilities and queue status. When the input frame rate does not match the system processing capacity, frame data queues up at each processing stage, causing end-to-end latency to accumulate linearly with increasing load, while latency jitter increases significantly. This problem is particularly pronounced on edge devices with limited computing resources and memory bandwidth, severely restricting the ability to meet the strict latency requirements of interactive applications.

[0004] (2) Static frame skipping strategies lack adaptability: Traditional frame skipping algorithms typically make frame dropping decisions based on preset thresholds or fixed ratios, lacking a feedback adjustment mechanism based on the real-time state of the system. Under fluctuating load conditions, suboptimal phenomena such as excessive or insufficient frame skipping are prone to occur, making it difficult to maintain a stable performance balance in diverse application scenarios. In addition, existing static strategies generally do not comprehensively consider key performance indicators such as encoding processing latency, buffer depth, and network round-trip latency, resulting in a lack of precision in the overall control mechanism.

[0005] (3) Robustness defects of timestamp management and clock synchronization mechanisms: Existing solutions mostly use linear increment or simple extrapolation methods for timestamp management. When the system experiences frame skipping, retransmission, out-of-order processing or processing delay fluctuations, the timestamp sequence is prone to discontinuity, rollback or abnormal crossing, which leads to the clock reference of downstream decoders and rendering components being out of order, and thus causes service quality problems such as video desynchronization, excessive display delay or playback stuttering. Moreover, the mechanism lacks the ability to systematically constrain the monotonicity, continuity and boundary conditions of timestamps.

[0006] (4) Lack of closed-loop feedback and adaptive scheduling mechanism: Most existing technologies lack an observable-controllable closed-loop architecture for real-time performance. They cannot continuously collect and effectively utilize key performance indicators such as processing latency, buffer depth, packet loss rate, CPU / NPU utilization, and network congestion to achieve automatic optimization of system parameters. Therefore, core control parameters such as encoding bitrate, resolution, frame skipping interval, tile partitioning granularity, and thread parallelism remain statically configured or adjusted only according to heuristic rules under different load conditions, resulting in significant fluctuations in system stability and low-latency performance as the application scenario changes.

[0007] (5) Insufficient utilization and collaborative optimization of heterogeneous edge hardware resources: Although some existing solutions have achieved a certain degree of latency optimization through protocol layer collaboration or edge-side preprocessing and tile-based technologies, they lack fine-grained, perception-driven scheduling and collaboration mechanisms for heterogeneous hardware resources in edge computing devices. This is mainly reflected in the lack of adaptive optimization of thresholds and queue depths in hardware codecs and image signal processing links; the inability to dynamically adjust software and hardware parallelism and task sharding strategies according to load changes, resulting in the inability to stably convert the acceleration potential of heterogeneous hardware into end-to-end low latency and high-quality output performance.

[0008] Comprehensive analysis shows that while existing solutions have made some progress in individual technologies such as protocol coordination, accelerated rendering, and edge processing, they generally lack system-level integrated optimization capabilities, including a "data-driven closed-loop feedback control mechanism + adaptive scheduling algorithm + robust timestamp and clock management scheme + heterogeneous hardware-aware collaborative optimization." This makes it difficult to maintain the collaborative goals of low latency, low jitter, and high image quality in complex and ever-changing edge computing environments over the long term. Therefore, it is necessary to develop a real-time video streaming transmission method and system architecture for edge computing that can automate parameter scheduling and timing management based on performance monitoring and network status awareness at runtime, fully leveraging the potential of heterogeneous hardware to ensure stable, low-latency, and high-reliability end-to-end transmission paths. Summary of the Invention

[0009] To address the aforementioned technical issues, this invention provides a method and system for adaptive low-latency real-time transmission of video streams based on edge computing. It relates to a real-time transmission latency optimization technology for video streams implemented on an edge computing development board, which is particularly suitable for application scenarios with extremely high real-time requirements for video transmission, such as intelligent monitoring, industrial inspection, and autonomous driving.

[0010] This invention is achieved using the following technical solution: Firstly, the edge computing-based adaptive low-latency real-time video stream transmission system is constructed using a closed-loop feedback control architecture, including the following modules: Video input processing module: Inputs and receives real-time video streams based on the RTSP protocol, buffers and controls the flow of the received real-time video streams through the input streaming media pipeline queue, and decodes the real-time video streams in real time through a hardware decoder; Performance feedback monitoring module: collects multi-dimensional performance indicators of the system in real time, calculates the system load index and provides a basis for decision-making; Adaptive control decision module: Determines whether the current video frame needs to be skipped based on the system load status. If the system load is high, skip inference and directly generate the timestamp. If the system load is normal, send the video frame data to the inference processing module. Inference processing module: Performs image inference on video frame data that has not been skipped, prioritizing the use of computing resources to complete preprocessing and inference; Timestamp synchronization module: Generates monotonically increasing timestamps for all output video frames; Video output processing module: It uses a hardware encoder to encode the video frames processed by the timestamp synchronization module, and pushes the encoded video stream to the streaming media server for video stream distribution via the RTSP protocol.

[0011] Specifically, the multi-dimensional performance indicators in the performance feedback monitoring module include CPU utilization, memory usage, processing time, and queue depth.

[0012] Specifically, the system also includes a multi-core parallel processing architecture for streaming media pipelines based on edge computing devices, built using the Gstramer framework. The multi-core parallel processing architecture includes three independent parallel processing links: input decoding, edge processing, and output encoding. Among them, the input decoding link uses a hardware decoder to perform video stream decoding in parallel on a dedicated decoding core, and achieves non-blocking transmission of video frame data through asynchronous queue buffering. The edge processing link performs video frame preprocessing and image target recognition on the processor core through an asynchronous callback mechanism. The output encoding link uses a hardware encoder to implement video encoding and network streaming on a dedicated encoding core.

[0013] Specifically, the performance feedback monitoring module also sets up a memory optimization management mechanism, uses a bounded cache queue to manage performance monitoring data, utilizes streaming media pipeline memory mapping technology to achieve zero-copy data access, and directly reads video frame data from the hardware buffer; by setting downstream leakage and frame discarding policies, it automatically releases outdated data when the system load is too high.

[0014] On the other hand, the edge computing-based adaptive low-latency real-time transmission method for video streams includes the following steps: Step S1: Start processing, initialize system parameters and configurations, receive real-time video streams from network cameras via RTSP protocol, connect to camera devices to obtain video stream metadata information, and configure an automatic reconnection mechanism after network interruption. Step S2: Create a pipeline queue, store the received video frames into the buffer queue, set the queue depth, add timestamp information to each video frame, and set up a queue protection mechanism; Step S3: Retrieve the encoded video frames from the input queue, use hardware acceleration decoding to improve the processing of video frames, and record the processing time for each frame; Step S4: Monitor system resource usage in real time; Step S5: Frame skipping determination, and perform adaptive frame skipping decisions based on performance monitoring results; Step S6: Timestamp generation, dynamically calculate the timestamp interval based on the actual processing frame rate; Step S7: Receive the decoded original video frames and corresponding timestamp information, use hardware acceleration to decode, and perform video encoding operations; encapsulate and transmit the encoded video data according to the RTCP protocol, and connect to the client for playback through the standard protocol interface provided by the RTSP server.

[0015] Specifically, step S4 includes the following sub-steps: Step S41: Obtain CPU utilization; Step S42: Monitor the total system memory and available memory; Step S43: Calculate the depth of the input queue; Step S44: Calculate the comprehensive load index at time [time missing]. System load index Represented as: ; in, CPU utilization; Memory usage; This represents the standardized average processing time, and , Indicates the current time Average processing time This indicates the preset standard processing time; Indicates the normalized queue depth, and , Indicates the current time queue depth, Indicates the maximum queue depth; , , and Let be the weight coefficient, and satisfy... ; Step S45: and define the first The frame-level performance of a frame is represented as follows: ; System-level performance is represented as: ; in, Indicates the processing time for a single frame. Indicates the encoding time; This indicates the system temperature status.

[0016] Specifically, step S5 includes the following sub-steps: Step S51: Calculate the frame skipping interval based on the processing capability assessment. The calculation formula is expressed as: ; in, Indicates the target frame interval time; The average processing time within the sliding window is expressed by the following formula: ; in, Indicates the first Frame to the The actual processing time of a frame. To adjust the sliding window size, Indicates the first frame; Step S52: Maintain a global frame counter. The update of the frame counter is represented as follows: ; Step S53: Perform frame skipping judgment. The frame skipping decision is represented as follows: ; in, This means skipping the first... frame, This indicates that the processing of the first... frame.

[0017] Specifically, the timestamp in step S6 is generated based on an adaptive timestamp generation and synchronization mechanism, including: Set up a monotonically increasing timestamp generator to generate the first... The timestamp of a frame is represented as: ; in, Indicates the start timestamp; Indicates the standard frame interval; The timestamp interval is adaptively adjusted, dynamically calculated based on the actual processing frame rate, and expressed as follows: ; ; in, Indicates the first Frame timestamps Indicates the current system timestamp; Indicates the minimum timestamp interval; Indicates the adaptive time interval; The time step threshold satisfies .

[0018] The beneficial effects of this invention are as follows: This invention achieves an integrated "observation-control-correction" technical capability for real-time video transmission links. In complex edge computing environments and under dynamic load conditions, it can significantly reduce end-to-end transmission latency and latency jitter, improve system adaptability, operational stability, and video quality consistency, thereby meeting the stringent technical requirements of real-time interactive applications for low-latency transmission and high-reliability services. It has the following advantages in real-time video stream transmission under edge computing environments: (1) Improved pipelined parallel processing efficiency: The three links of "input decoding - edge processing - output encoding" are decoupled and parallelized at the edge computing device. Based on GStreamer's asynchronous queue and backpressure control, producer-consumer decoupling and stage balancing are achieved, enabling hardware decoding, NPU / CPU inference and hardware encoding to be executed concurrently, significantly reducing cross-stage waiting. Compared with schemes that only carry out TCP / UDP collaboration or GPU rendering acceleration on the terminal side, or mainly perform tile encoding and compression domain splicing at the edge nodes, this invention constructs a more granular parallel topology and queue management strategy at the device side, making it easier to maintain overall low latency and stable output.

[0019] (2) Enhanced system adaptability and operational stability: Closed-loop adaptive scheduling is driven by multi-dimensional performance telemetry (CPU / memory utilization, processing time, queue depth, etc.). The system is designed with hysteresis, anti-oscillation, and upper limit protection to dynamically adjust the frame skipping ratio and processing strategy. It actively reduces load during high load and achieves smooth recovery when the load decreases. Unlike the traditional approach of fixed frame rate / static parameters, or the scheme of static optimization only at the network / tile level, this invention can continuously adapt during operation and achieve a controllable trade-off between latency and image quality under load and network disturbances.

[0020] (3) The utilization efficiency and energy efficiency of heterogeneous resources are significantly improved. Asynchronous frame callback and task slicing are adopted, zero-copy path is given priority, and hardware encoding and decoding framework (MPP) is introduced to give full play to hardware acceleration capabilities. Combined with the NPU's collaborative acceleration of preprocessing and inference, fine-grained matching and parallel balancing of "computing power-bandwidth-caching" are achieved. Compared with the solution that only relies on the terminal-side GPU rendering or mainly builds tile pipelines on the edge cloud side, this invention emphasizes the collaborative scheduling of heterogeneous acceleration on the device side, which can significantly reduce CPU usage, improve throughput and energy efficiency, and show higher robustness under sudden high load conditions.

[0021] (4) Enhanced time synchronization and timing management capabilities: The system employs a "monotonically increasing + adaptive interval" timestamp management mechanism and sets minimum intervals and crossing limits for boundary protection against frame skipping, out-of-order issues, and processing delay fluctuations, ensuring continuous and backtrack-free output timelines. Compared to conventional practices that only align at the acquisition or rendering side, this invention maintains a unified time base even when skipping frames in the output, preventing downstream parsing anomalies. The system can stably control timestamp deviations within a very small range, thereby ensuring smooth playback.

[0022] (5) Memory usage efficiency and long-term stability optimization: Bounded queues and memory pool management are adopted in the pipeline, combined with zero-copy / low-copy cross-module transfer and "expired frame discarding" strategy, to effectively suppress memory usage increase and fragmentation. Unlike the approach of relying solely on general memory pools or simple FIFO caches, this invention incorporates "buffer level - queue length - discarding rules" into closed-loop control, controlling memory usage while constraining end-to-end latency, ensuring stable and efficient operation even after long-term operation.

[0023] (6) Broad hardware compatibility and portability, adaptable to multiple mainstream edge computing platforms, enabling rapid porting and on-demand optimization through modular pipelines and parameterized scheduling. Unlike platform-independent but device-aware heterogeneous solutions, this invention achieves a balance between "platform universality + hardware-aware optimization". The same framework can maintain hardware acceleration capabilities and low latency characteristics on different board types with only a few parameter adjustments. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0025] Figure 1 This is a basic flowchart of the adaptive low-latency real-time transmission method for video streams based on edge computing in this embodiment of the invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0027] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0028] The following is in conjunction with the appendix Figure 1 The following describes some embodiments of the present invention in detail. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0029] This invention proposes an adaptive low-latency real-time video stream transmission method and system based on edge computing, aiming to overcome the systematic shortcomings of existing technologies in real-time video stream transmission in edge computing environments. The technical problems to be solved include, but are not limited to: (1) Stable low-latency transmission under edge resource constraints: In scenarios where processor memory bandwidth is limited and load and network conditions fluctuate dynamically, how to avoid continuous backlog of processing queues and amplification of jitter, keep end-to-end latency within a controllable range and achieve stable convergence, while taking into account the smoothness of interaction and the continuity of the screen. (2) Adaptive frame processing based on real-time performance status: How to design an adaptive scheduling algorithm with hysteresis compensation, oscillation suppression and boundary protection mechanisms based on observable performance indicators such as processing latency, buffer depth, CPU / NPU utilization and packet loss rate, so as to achieve dynamic optimization of frame skipping ratio and sampling interval, avoid excessive or conservative deviation of frame skipping strategy, and achieve a controllable trade-off between transmission delay and video quality under diverse load conditions.

[0030] (3) Full-link performance monitoring and closed-loop feedback control architecture: How to build a closed-loop control architecture covering data acquisition, performance evaluation, decision generation and parameter execution, and establish an automated optimization mechanism from observable performance indicators to controllable system parameters so that the system can continuously adapt and maintain a stable operating state as the application scenario changes.

[0031] (4) Timestamp continuity and monotonicity management in frame scenarios: How to ensure that the output timestamps are strictly monotonically increasing on a unified time base and maintain continuity during frame jumps, avoid backtracking and abnormal jumps, and ensure that the downstream decoder and rendering end can correctly parse and align the timing, thereby ensuring the smoothness of video playback.

[0032] (5) Hardware-aware resource collaboration optimization and efficient utilization: How to make full use of the hardware acceleration capabilities of the edge computing development board to improve overall processing efficiency.

[0033] By addressing the above technical challenges, this invention significantly reduces end-to-end latency in video stream transmission, improves the system's adaptability and stability under varying load conditions, and achieves integrated "observation-control-correction" capabilities for real-time video transmission links. In complex edge computing environments and under dynamic load conditions, it significantly reduces end-to-end transmission latency and jitter, enhancing the system's adaptability, operational stability, and video quality consistency, thereby meeting the stringent technical requirements of real-time interactive applications for low-latency transmission and high-reliability services.

[0034] In a preferred embodiment, the edge computing-based adaptive low-latency real-time video stream transmission method is as follows: Figure 1 As shown, the specific steps include: Step 1: Start processing, initialize system parameters and configurations, including setting the target frame rate, sliding window size, and load index calculation weight coefficient; receive real-time video streams from network cameras via the RTSP protocol, establish a connection with the camera devices, obtain metadata information such as the video stream's encoding format, resolution, and frame rate, and configure an automatic reconnection mechanism after network interruption.

[0035] Step 2: Create a pipeline queue, store the received video frames in the buffer queue, set the queue depth, add timestamp information to each video frame, and set up a queue protection mechanism.

[0036] Step 3: Retrieve the encoded video frames from the input queue, use hardware acceleration decoding to improve the processing of video frames, and record the processing time of each frame, including the time consumed by all operations such as frame retrieval, decoding calculation, and memory copying.

[0037] Step 4, monitor system resource usage in real time, specifically including: (1) Obtain CPU utilization; (2) Monitoring system total memory and available memory; (3) Calculate the depth of the input queue; (4) Calculate the comprehensive load index according to the formula: ; in, :time The system load index; CPU utilization; Memory usage rate; : This represents the standardized average processing time, and , Indicates the current time Average processing time This indicates the preset standard processing time; Indicates the normalized queue depth, and , Indicates the current time queue depth, Indicates the maximum queue depth; , , , Weighting coefficients, satisfying .

[0038] In this embodiment, system monitoring also includes: Frame-level performance is defined as: ; Indicates the processing time for a single frame. Indicates the encoding time; System-level performance is defined as: ; This indicates the system temperature status.

[0039] In one embodiment, it also includes load index. Progressive smoothing and hysteresis handling: for load exponents Each sub-indicator uses a sliding window and exponential smoothing, and sets up / down thresholds and hold times to suppress frequent switching caused by jitter.

[0040] Step 5, frame skipping determination: Based on performance monitoring results, adaptive frame skipping decisions are executed. This is the core innovation of this invention, specifically including: (1) Calculate the frame skipping interval: ; in: Average processing time within the sliding window; : No. The actual processing time of the frame; : Slide window size; : The current video frame number.

[0041] Frame skipping interval calculation: ; in: Frame skipping interval; Target frame interval time; (2) Maintain a global frame counter. The frame counter update is represented as follows: ; (3) Perform frame skipping judgment and processing capability assessment: Frame skipping decision: ; in, Indicates skipping the first frame; Indicates processing the first frame.

[0042] Step 6, Timestamp Generation: The timestamp interval is dynamically calculated based on the actual processing frame rate to address timestamp continuity issues in frame skipping scenarios, ensuring the temporal continuity of the output video stream. An adaptive timestamp generation and synchronization mechanism is established, including: Monotonically increasing timestamp generator: ; in, Indicates the start timestamp; Indicates the standard frame interval; Adaptive adjustment of timestamp interval: ; in: : The system's current timestamp; Minimum timestamp interval; Adaptive time interval; .

[0043] The timestamp interval is dynamically calculated based on the actual frame rate to address the timestamp continuity issue in frame skipping scenarios.

[0044] Step 7: Receive the decoded original video frames and corresponding timestamp information, use hardware acceleration for decoding, and perform video encoding operations.

[0045] Step 8: Encapsulate and transmit the encoded video data according to the RTCP protocol, and provide a standard protocol interface through the RTSP server for client connection and playback. After completing the entire video stream processing flow, return to Step 1 to continue processing subsequent frame data.

[0046] This invention also proposes a video stream adaptive low-latency real-time transmission system based on edge computing, applied to a video stream adaptive low-latency real-time transmission method based on edge computing. The system includes: The video input processing module receives real-time video streams based on the RTSP protocol. The input pipeline queue buffers and controls the flow of the received video streams, and the hardware decoder decodes the video streams in real time. The performance feedback monitoring module is used to collect multi-dimensional performance indicators such as system CPU utilization, memory usage, processing time and queue depth in real time, calculate the system load index and provide decision-making basis. The adaptive control decision module is used to determine whether the current video frame needs to be skipped based on the system load status. If the system load is high, it skips the inference and directly enters the timestamp generation stage. If the system load is normal, the video frame data is sent to the inference processing module. The inference processing module is used to perform image target recognition calculations (target detection / recognition / tracking) on ​​normally processed video frames, prioritizing the use of computing resources to complete preprocessing and inference. The timestamp synchronization module is used to generate monotonically increasing timestamps for all output video frames (including skipped frames and normal processing frames) to ensure the temporal continuity of the video stream in skipped frame scenarios. The video output processing module is used to encode the processed video frames using a hardware encoder to H.264, and then push the encoded video stream to the streaming media server via the RTSP protocol to complete the final video stream distribution.

[0047] In this embodiment, the system utilizes Gstramer to build a multi-core parallel processing architecture for the streaming media pipeline. By constructing three independent parallel processing links—input decoding, edge processing, and output encoding—it fully utilizes the multi-core resources of edge computing. The input link uses a hardware decoder to perform video stream decoding in parallel on a dedicated decoding core, and achieves non-blocking transmission of frame data through asynchronous queue buffering. The edge processing link performs frame preprocessing and image target recognition on the processor core through an asynchronous callback mechanism. The output encoding link uses a hardware encoder to implement video encoding and network streaming on a dedicated encoding core.

[0048] In this embodiment, the system also builds a Gstramer pipeline, an efficient memory optimization management mechanism, and uses a bounded cache queue to manage performance monitoring data to prevent memory leaks; it uses pipeline memory mapping technology to achieve zero-copy data access, directly reading video frame data from the hardware buffer to avoid additional memory allocation and data movement; and through downstream leakage and frame dropping strategies, it automatically releases outdated data when the system load is too high to maintain stable memory usage.

[0049] This invention also provides performance comparison experimental results under different load scenarios, as shown in Table 1. The experiments were conducted on the RK3576 development board under different load scenarios, comparing the processing performance of the commonly used FFmpeg streaming solution with that of this invention. The experimental results show that the solution of this invention significantly outperforms the FFmpeg solution in all key indicators: Compared with commonly used FFmpeg streaming solutions on the RK3576 platform, this invention reduces end-to-end latency by approximately 92.6%–94.0%, CPU utilization by approximately 59.6%–72.8%, and PSNR by approximately 146%–320% under different loads.

[0050] Table 1 Comparison of Experimental Results

[0051] Experimental data shows that as the load increases, the performance of the traditional FFmpeg solution drops sharply, while the adaptive frame skipping scheduling scheme of this invention can maintain stable low latency and low resource consumption, while significantly improving video quality, thus verifying the superiority of the technical solution.

[0052] For the foregoing embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0053] The above embodiments describe the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Modifications and variations made by those skilled in the art without departing from the spirit and scope of the invention should be within the protection scope of the appended claims.

Claims

1. A video stream adaptive low-latency real-time transmission method based on edge computing, implemented based on a video stream adaptive low-latency real-time transmission system, characterized in that... Includes the following steps: Step S1: Start processing, initialize system parameters and configurations, receive real-time video streams from network cameras via RTSP protocol, connect to camera devices to obtain video stream metadata information, and configure an automatic reconnection mechanism after network interruption. Step S2: Create a pipeline queue, store the received video frames into the buffer queue, set the queue depth, add timestamp information to each video frame, and set up a queue protection mechanism; Step S3: Retrieve the encoded video frames from the input queue, use hardware acceleration decoding to improve the processing of video frames, and record the processing time for each frame; Step S4: Monitor system resource usage in real time and adjust load index. The system resource sub-indicators are smoothed using a sliding window and exponential smoothing, and uplink and downlink thresholds and hold times are set to suppress frequent switching caused by jitter; step S4 includes the following sub-steps: Step S41: Obtain CPU utilization; Step S42: Monitor the total system memory and available memory; Step S43: Calculate the depth of the input queue; Step S44: Calculate the overall load index; Step S45: Define the first The frame-level performance of a frame is represented as follows: ； System-level performance is represented as: ； in, CPU utilization; Memory usage; Indicates the processing time for a single frame. Indicates the encoding time; Indicates the system temperature status; Step S5: Frame skipping determination. Based on performance monitoring results, an adaptive frame skipping decision is made, including: Step S51: Calculate the frame skipping interval based on the processing capability assessment. The calculation formula is expressed as: ； in, Indicates the target frame interval time; The average processing time within the sliding window is expressed by the following formula: ； in, Indicates the first Frame to the The actual processing time of a frame. To adjust the sliding window size, Indicates the first frame; Step S52: Maintain a global frame counter. The update of the frame counter is represented as follows: ； Step S53: Perform frame skipping judgment. The frame skipping decision is represented as follows: ； in, This means skipping the first... frame, This indicates that the processing of the first... frame; Step S6: Develop timestamps based on an adaptive timestamp generation and synchronization mechanism, dynamically calculating the timestamp interval according to the actual processing frame rate, including: Set up a monotonically increasing timestamp generator to generate the first... The timestamp of a frame is represented as: ； in, Indicates the start timestamp; Indicates the standard frame interval; The timestamp interval is adaptively adjusted, dynamically calculated based on the actual processing frame rate, and expressed as follows: ； ； in, Indicates the first Frame timestamp; Indicates the current system timestamp; Indicates the minimum timestamp interval; Indicates the adaptive time interval; The time step threshold satisfies ; Step S7: Receive the decoded original video frames and corresponding timestamp information, use hardware acceleration to decode, and perform video encoding operations; encapsulate and transmit the encoded video data according to the RTCP protocol, and connect to the client for playback through the standard protocol interface provided by the RTSP server; The system includes a multi-core parallel processing architecture for streaming media pipelines based on edge computing devices, built using the Gstreamer framework. This architecture comprises three independent parallel processing links: input decoding, edge processing, and output encoding. The input decoding link utilizes a hardware decoder to perform video stream decoding in parallel on a dedicated decoding core, achieving non-blocking transmission of video frame data through asynchronous queue buffering. The edge processing link performs video frame preprocessing and image target recognition on the processor core via an asynchronous callback mechanism. The output encoding link utilizes a hardware encoder to implement video encoding and network streaming on a dedicated encoding core.

2. The video stream adaptive low-latency real-time transmission method as described in claim 1, characterized in that, The adaptive low-latency real-time video stream transmission system is built using a closed-loop feedback control architecture and includes the following modules: Video input processing module: Inputs and receives real-time video streams based on the RTSP protocol, buffers and controls the flow of the received real-time video streams through the input streaming media pipeline queue, and decodes the real-time video streams in real time through a hardware decoder; Performance feedback monitoring module: Real-time collection of multi-dimensional performance indicators of the system, including CPU utilization, memory usage, processing time and queue depth, to calculate system load index and provide decision-making basis; It also incorporates a memory optimization management mechanism, employing a bounded cache queue to manage performance monitoring data, and utilizing streaming media pipeline memory mapping technology to achieve zero-copy data access, directly reading video frame data from the hardware buffer; by setting a frame discarding policy, it automatically releases outdated data when the system load is too high; the system load index... At any moment Represented as: ； in, CPU utilization; Memory usage; This represents the standardized average processing time, and , Indicates the current time Average processing time This indicates the preset standard processing time; Indicates the normalized queue depth, and , Indicates the current time queue depth, Indicates the maximum queue depth; , , and Let be the weight coefficient, and satisfy... ; Adaptive control decision module: Determines whether the current video frame needs to be skipped based on the system load status. If the system load is high, skip inference and directly generate the timestamp. If the system load is normal, send the video frame data to the inference processing module. Inference processing module: Performs image inference on video frame data that has not been skipped, prioritizing the use of computing resources to complete preprocessing and inference; Timestamp synchronization module: Generates monotonically increasing timestamps for all output video frames; Video output processing module: It uses a hardware encoder to encode the video frames processed by the timestamp synchronization module, and pushes the encoded video stream to the streaming media server for video stream distribution via the RTSP protocol.

Images