A time delay compensation control method and system for 5G uplink real-time video transmission

Abstract

The application discloses a kind of 5G uplink real-time video transmission oriented time delay compensation control method and system, belong to wireless network real-time video transmission optimization technical field.The application is to solve the problem of time delay inaccuracy and control misjudgment caused by base station scheduling waiting, batch resource release and link layer retransmission in 5G uplink real-time video transmission, by sending end disabling step module to realize video frame burst transmission, and using the flow pattern analysis of receiving end and uplink scheduling estimation to identify scheduling additional influence, construct scheduling additional occupancy ratio to generate correction time delay and feedback to sending end for controller adaptation.The application can estimate and compensate the additional time delay component related to wireless scheduling waiting, batch resource release and potential link layer retransmission, improve the accuracy of sending end in judging real link state, with the advantages of low engineering modification cost, strong commercial adaptability and low computational complexity.

Description

Technical Field

[0001] This invention discloses a latency compensation control method and system for 5G uplink real-time video transmission, belonging to the field of wireless network real-time video transmission optimization technology. Background Technology

[0002] With the rapid deployment of 5G communication networks, real-time uplink video services on mobile terminals are showing a continuous growth trend. Typical applications include mobile live streaming, video conferencing, remote collaboration, industrial inspection, drone image transmission, cloud gaming, and augmented reality / virtual reality. These services typically require both high transmission bitrates and low end-to-end latency, ensuring both video quality and real-time interactivity. Therefore, the transmitting end usually needs to dynamically adjust the video encoding rate or transmission rate based on network feedback to achieve a balance between latency and throughput.

[0003] In existing real-time video control methods, many algorithms rely on RTT (Round-Trip Time), one-way latency, frame completion time, or time metrics related to the receiving rate to infer network congestion and perform speed-up or speed-down operations accordingly. These methods typically implicitly assume that the observed latency changes primarily originate from queuing changes in bottleneck links. However, this assumption often does not hold true in 5G uplinks. 5G uplink transmission is controlled by a centralized scheduling mechanism at the base station. Before sending data, terminal devices typically need to report their buffer status and wait for the base station to allocate uplink radio resources. After resources arrive, packets are often sent in short batches, rather than leaving the transmitter almost continuously and instantly as in wired networks. Furthermore, when radio link transmission fails, link-layer retransmission further prolongs the packet completion time. These mechanisms collectively result in the latency observed at the device side including not only the actual queuing latency but also significant scheduling waiting latency and retransmission latency.

[0004] In this context, directly inputting the raw latency into the existing controller can easily lead to the following problems: First, when wireless scheduling wait increases but the bottleneck is not actually congested, the transmitter may mistakenly interpret the increase in latency as worsening congestion, prematurely reducing the bit rate and causing underutilization of available wireless bandwidth. Second, when scheduling wait decreases for a period of time but the actual queuing does not improve, the transmitter may incorrectly increase the transmission rate, thereby triggering new queue backlogs and control oscillations. In other words, in a 5G uplink environment, latency and congestion no longer maintain a simple correspondence; the raw latency signal has been "contaminated" by the wireless scheduling process.

[0005] Existing technologies can be broadly categorized into two types. One type is the pure end-to-end control method, which is easy to deploy, but its observables are limited to information visible at the transport or application layers, making it difficult to explicitly separate queuing delay from scheduling delay. The other type relies on cross-layer feedback, base station collaboration, or network-side modifications. While theoretically more accurate wireless status information can be obtained, these methods often require modifications to access network equipment, underlying protocol stacks, or network infrastructure, resulting in high deployment complexity and making them unsuitable for large-scale application in commercial 5G networks.

[0006] While studying 5G uplink real-time video transmission, the inventors discovered that even without significant network link congestion, the round-trip time and packet arrival intervals within video frames observed at the endpoint still exhibit significant fluctuations. These fluctuations typically possess recurring discrete temporal characteristics rather than being completely random. Further analysis revealed that this phenomenon is closely related to radio-side scheduling waiting, batch resource release, and link-layer retransmission processes. Based on this analysis, it is evident that existing technologies lack a solution that neither relies on explicit base station coordination nor requires modification of the 5G network infrastructure, while simultaneously utilizing observable timing characteristics at the endpoint to correct the original latency. Therefore, there is an urgent need to propose a latency compensation control method and device for 5G uplink real-time video transmission to reduce the interference of radio scheduling waiting, batch resource release, and link-layer retransmission on endpoint latency measurements, improve the accuracy of the transmitter's control decisions in representing the actual link state, and thereby enhance the system's robustness and adaptability in complex wireless environments. Summary of the Invention

[0007] The purpose of this invention is to solve the problems of inaccurate latency and control misjudgment introduced by base station scheduling waiting, batch resource release and link layer retransmission in 5G uplink real-time video transmission. It proposes a latency compensation control method and system for 5G uplink real-time video transmission, which realizes the identification and removal of non-congestion-related components in the original latency signal, improves the accuracy of the transmitter's perception of the real link status, and optimizes the performance of low-latency real-time video transmission in wireless network environment.

[0008] To achieve the above objectives, the present invention adopts the following technical solution.

[0009] A latency compensation control method for 5G uplink real-time video transmission includes the following steps: The video content is acquired and encoded to obtain video frame data; The video frame data is sent to the network burst based on a pacing disable policy; The feedback information is obtained and the correction delay parameter is parsed, and the correction delay parameter is mapped to the congestion controller to obtain the control input signal; The video encoding parameters and data packet transmission parameters are determined based on the control input signals.

[0010] Furthermore, based on a pacing-based disabling strategy, video frame data is sent to network bursts, including: After the video frame is encoded, the restriction on the uniform transmission interval of all data packets belonging to the same video frame is lifted, allowing the data packets to enter the transmission path in a frame-level grouped manner.

[0011] Further, feedback information is acquired and parsed to obtain correction delay parameters, and the correction delay parameters are mapped to the congestion controller to obtain control input signals, including: Analyze the control mechanism type of the congestion controller; Based on the control mechanism type, the correction delay parameter is adapted to the data. The original duration is replaced by the compensated completion time, or the original delay parameter is replaced by the correction delay parameter, or the delay cost and window adjustment amount are recalculated using the correction delay parameter to generate the corresponding control input signal.

[0012] A latency compensation control method for 5G uplink real-time video transmission includes the following steps: Record the arrival time of received data packets and determine the traffic pattern representation sequence corresponding to the video frame; Based on the traffic pattern characterization sequence, the actual scheduling occupancy of video frames and the effective carrying capacity of a single scheduling are estimated. Based on the actual scheduling occupancy and the effective carrying capacity, a scheduling additional occupancy ratio is constructed; Based on the scheduling additional occupancy ratio and the original delay, corrected delay information is generated, and the corrected delay information is encapsulated and sent to the sending end as a feedback data packet.

[0013] Furthermore, the arrival time of the received data packets is recorded, and the traffic pattern representation sequence corresponding to the video frame is determined, including: Based on the frame affiliation and arrival time of the data packets, preliminary clustering is performed according to the packet interval to obtain a preliminary data cluster sequence; Identify cross-frame continuous scheduling processes at the boundaries of adjacent video frames; Based on the cross-frame continuous scheduling process, the preliminary data cluster sequence is boundary-corrected to generate a traffic pattern representation sequence that reflects the characteristics of underlying radio resource release.

[0014] Further, based on the traffic pattern characterization sequence, the actual scheduling occupancy of video frames is estimated, including: Representative scheduling cycles are determined based on statistical characteristics of traffic patterns, the main peak value of inter-cluster interval histograms, candidate cycle matching within sliding windows, or preset time slot parameters of network configuration. The total number of scheduling units that a video frame actually traverses is calculated based on the representative scheduling period and the distance between each data cluster in the traffic pattern.

[0015] Further, estimating the effective carrying capacity of a single scheduling based on the traffic pattern characterization sequence includes: Candidate carrying capacity values ​​are determined based on the volume balance relationship between adjacent data clusters or the volume of a single cluster. The candidate carrying capacity value and the historical effective carrying capacity are weighted and averaged based on a smoothing factor to update the effective carrying capacity.

[0016] Further, based on the actual scheduling occupancy and the effective carrying capacity, a scheduling additional occupancy ratio is constructed, including: The theoretical minimum number of scheduling units is determined based on the total payload of the video frames and the effective carrying capacity. Calculate the difference between the total number of scheduling units actually crossed and the theoretical minimum number of scheduling units, and construct the additional scheduling occupancy ratio based on the ratio of the difference to the total number of scheduling units actually crossed.

[0017] Further, based on the scheduling additional occupancy ratio and the original delay, corrected delay information is generated, including: By using a proportional adjustment coefficient or a preset mapping relationship, the additional scheduling occupancy ratio or representative scheduling cycle is mapped to the original delay to determine the estimated value of the additional scheduling delay. The scheduling-added delay component is removed from the original delay to generate corrected delay information that reflects the actual link congestion state.

[0018] A latency compensation control system for 5G uplink real-time video transmission includes a transmitter and a receiver; The sending end includes: A video encoder is used to acquire video content and encode it to obtain video frame data. The Pacing module is disabled to provide a pacing disable strategy to weaken fine-grained packet-by-packet control. The transmission module is used to send the video frame data to the network burst based on the step-disable strategy; The controller adaptation module is used to acquire feedback information and parse the correction delay parameters, map the correction delay parameters to the congestion controller to obtain control input signals, and make decisions on video encoding parameters and data packet transmission parameters based on the control input signals. The receiving end includes: A video decoder is used to decode and restore received video data packets; The traffic pattern analysis module is used to record the arrival time of received data packets and determine the traffic pattern representation sequence corresponding to the video frame. The uplink scheduling estimation module is used to estimate the actual scheduling occupancy of video frames and the effective carrying capacity of a single scheduling based on the traffic pattern characterization sequence, and to construct the additional scheduling occupancy ratio based on the actual scheduling occupancy and the effective carrying capacity. The delay compensation generation module is used to generate corrected delay information based on the scheduling additional occupancy ratio and the original delay, and to encapsulate the corrected delay information and send a feedback data packet to the sending end.

[0019] The present invention has achieved the following beneficial effects.

[0020] 1. This invention can be deployed simply by enhancing the software of the sending and receiving ends, without modifying the 5G base station, core network equipment or underlying protocol stack. Furthermore, by connecting the correction delay to the original control loop through the controller adaptation module, it does not require replacing the main structure of the existing congestion controller. It has strong commercial network adaptability, good compatibility and low engineering transformation cost.

[0021] 2. This invention, through the collaborative processing of multiple functional modules at the receiving end, can perform time-series analysis on the packet arrival time structure within a video frame, identify and remove non-congestion-related additional delays caused by base station scheduling waiting, batch resource release, and link layer retransmission, thereby improving the accuracy of the sending end in judging the true link status in principle and effectively solving the problem of control misjudgment caused by delay distortion.

[0022] 3. This invention employs a coordinated design that disables the pacing module at the sending end and uses a delay compensation mechanism at the receiving end. This reduces invalid waiting caused by local scheduling at the sending end and mitigates the impact of delay misjudgment on rate control or congestion control, thereby achieving accurate perception and optimization of end-to-end transmission delay.

[0023] 4. The overall solution of this invention only relies on the data packet arrival time, frame attribution relationship and a small number of statistics that can be obtained from the end side for processing. The algorithm implementation logic is clear and the computational complexity is low, which can be widely deployed in mobile terminals, edge devices and various real-time video transmission systems. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of 5G uplink scheduling and retransmission time slots on which the present invention is based; Figure 2 This is a flowchart of a latency compensation control method for 5G uplink real-time video transmission located at the transmitting end in the embodiment. Figure 3 This is a flowchart of a latency compensation control method for 5G uplink real-time video transmission located at the receiving end in the embodiment. Figure 4 This is a schematic diagram of the overall structure of the latency compensation control system for 5G uplink real-time video transmission in the embodiment. Figure 5 This is a comparison diagram of the effects of the method of the present invention and existing methods. Detailed Implementation

[0025] To make the various technical features, advantages, or effects of the present invention more apparent and understandable, detailed descriptions are provided below through embodiments.

[0026] Figure 1 This diagram illustrates the specific sources of additional uplink latency in 5G networks. In a 5G network environment, the end-to-end transmission link consists of both wireless and wired links. Data sent from the terminal must wait for an uplink transmission slot to arrive before it can be transmitted via the wireless side, while feedback from the receiving end must be sent back via a downlink transmission slot. The diagram further illustrates the concepts of flexible time slot scheduling units, retransmission slots, and retransmission counts. When a wireless transmission fails or link conditions fluctuate, the data transmission process, which should have been completed in one uplink scheduling session, may be repeated across multiple subsequent time slots, resulting in additional waiting. Here, the uplink transmission slot represents the scheduling process of data transmission from the terminal to the network, the downlink transmission slot represents the process of feedback information from the network to the terminal, and the retransmission slot and retransmission count represent the situation where data transmission is completed across multiple scheduling units. Therefore, the original delay observed by the sending or receiving end not only includes the queuing delay caused by actual congestion, but also includes additional delay components introduced by uplink scheduling waiting, resource allocation delay, and link layer retransmission. This additional part is not equivalent to the actual bottleneck congestion. If it is directly used as the congestion control input, it is easy to cause system misjudgment.

[0027] The purpose of this invention is to identify and estimate this additional part at the receiving end, and then correct the control input at the transmitting end through a feedback mechanism, thereby providing the control loop with a delay observation basis that is closer to the real link state and enhancing the system's adaptability to the complex wireless environment of 5G uplink.

[0028] This invention provides a latency compensation control method for 5G uplink real-time video transmission, used at the transmitting end, such as... Figure 2 As shown, the processing steps include the following: Step S1: Acquire video content and encode it to obtain video frame data.

[0029] In an optional embodiment of the present invention, step S1 may include: Step S11: Encode the video content to obtain video data organized by frame.

[0030] Specifically, the video encoder at the sending end encodes the video content to be transmitted, generating video data organized by frame sequence.

[0031] Step S2: The video frame data is sent to the network burst based on the pacing strategy.

[0032] In an optional embodiment of the present invention, step S2 may include: Step S21: Identify all data packets belonging to the same video frame.

[0033] Step S22: After the video frame encoding is completed, the restriction on the uniform transmission interval of all data packets belonging to the same video frame is lifted, so that the data packets enter the transmission path in a frame-level grouped manner.

[0034] Specifically, the transmitting end adjusts the transmission rhythm control mode to cancel or weaken the traditional fine-grained packet-by-packet uniform transmission restriction. This invention configures a pacing-based disabling transmission strategy based on 5G uplink scheduling characteristics, and the specific strategy is described below: (1) Adjustment mechanism of transmission strategy: It does not require the sending end to apply a strict and uniform packet transmission rhythm to each data packet. Instead, it adopts a frame-level group transmission method, that is, when a video frame is encoded, the data packets belonging to that video frame are allowed to enter the network in a short period of time.

[0035] (2) The beneficial effects of adopting the stepping (Pacing) disabling design: First, it reduces local rhythm control interference. Since the actual air interface transmission time of data packets in the 5G uplink scenario is mainly controlled by base station authorization and radio resource scheduling, uniform stepping per packet usually cannot really control the departure transmission order of the radio side. Instead, it may introduce additional queuing and waiting at the transmitting end, reducing the additional impact of local scheduling at the transmitting end on delay observation. Second, it optimizes traffic feature identification. This design makes it easier for the receiving end to identify the traffic pattern features formed by radio scheduling from the arrival time of data packets.

[0036] Step S23: Burst transmission of video frame data packets.

[0037] Specifically, each frame of video data is further divided into multiple data packets and sent into the network transmission path. After the video frame is encoded, the data packets belonging to that video frame are allowed to enter the network in a group in a short period of time in the form of a Burst.

[0038] Step S3: Obtain feedback information and parse it to obtain the correction delay parameter, and map the correction delay parameter to the congestion controller to obtain the control input signal.

[0039] Specifically, this step provides a more realistic latency observation basis for the existing control loop through feedback coordination between the sending and receiving ends. This enhances the system's adaptability to the complex wireless environment of 5G uplink without reconstructing the original controller's main logic, demonstrating good compatibility and engineering deployability.

[0040] In an optional embodiment of the present invention, step S3 may include: Step S31: Obtain feedback data packets from the reverse link and parse them to obtain the correction delay parameters.

[0041] Specifically, the sending end receives feedback information from the receiving end, parses and extracts the compensated correction delay parameters from the data packets carrying the correction delay.

[0042] Step S32: Analyze the control mechanism type of the congestion controller.

[0043] Step S33: Based on the control mechanism type, perform data adaptation on the correction delay parameters to generate the corresponding control input signal.

[0044] Specifically, the correction delay parameter is integrated into the existing congestion controller to replace or correct the original delay signal that was previously used directly. The specific application method in step S33 is explained below, depending on the implementation mechanism of different controllers: (1) Rate estimation adaptation method: For controllers that use frame completion time to estimate the receiving rate, the compensated completion time is used to replace the original time.

[0045] (2) Trend judgment adaptation method: For controllers that judge congestion trends based on round-trip time (RTT) or one-way time delay, the corrected time delay parameters are used as new time delay inputs.

[0046] (3) Control logic adaptation method: For control logic that uses delay to construct delay cost, bandwidth utilization or window adjustment amount, the compensated delay parameters are used for the corresponding calculation.

[0047] Step S4: Decision on video encoding parameters and data packet transmission parameters based on the control input signal.

[0048] In an optional embodiment of the present invention, step S4 may include: Step S41: Based on the corrected input signal, a decision processing is performed to obtain the adjusted encoding and transmission parameters.

[0049] Specifically, step S41 includes: (1) System gain description: This invention enhances rather than replaces the existing congestion control logic through feedback coordination between the sending end and the receiving end.

[0050] (2) Strategy decision content: Adjust the video encoding rate, transmission window or target bit rate based on the corrected signal.

[0051] This invention provides a latency compensation control method for 5G uplink real-time video transmission, used at the receiving end, such as... Figure 3 As shown, the processing steps include the following: Step S1: Record the arrival time of the received data packets and determine the traffic pattern representation sequence corresponding to the video frame.

[0052] Specifically, this step involves performing timing analysis on the data packets at the receiving end to identify the timing characteristics during the arrival of the data packets, providing input for subsequent uplink scheduling estimation.

[0053] In an optional embodiment of the present invention, step S1 may include: Step S11: Based on the frame affiliation and arrival time of the data packets, perform preliminary clustering according to the packet interval to obtain a preliminary data cluster sequence.

[0054] Specifically, the receiving end records the arrival time of each data packet and assigns the received data packets to the corresponding video frame based on the frame identifier, timestamp, or sequence number in the packet header. For any video frame, the data cluster structure reflecting the timing characteristics of 5G uplink transmission is extracted by processing the data packet arrival time sequence within that video frame. The specific process is as follows: calculate the arrival time interval between two adjacent data packets, i.e., the difference between the arrival time of the current data packet and the arrival time of the previous data packet. Set an intra-frame traffic pattern partitioning threshold. If the time interval is less than the partitioning threshold, the current data packet and the previous data packet are considered to belong to the same near-continuously arriving data cluster; if the time interval is greater than or equal to the partitioning threshold, the current data packet is considered to have started a new data cluster. After the above partitioning, a video frame is represented as multiple data cluster sequences arranged in chronological order. The traffic pattern partitioning threshold can be set to a small threshold on the order of milliseconds, or it can be adaptively adjusted according to timestamp accuracy, link characteristics, or historical statistical results.

[0055] Step S12: Identify the cross-frame continuous scheduling process at the boundary of adjacent video frames.

[0056] Specifically, data clusters at the boundaries of adjacent video frames are detected to identify cross-frame resource release processes and eliminate false boundary interference. Since a single underlying resource release process may span two adjacent video frames in a 5G uplink scenario, simply truncating the process at video frame boundaries could easily missegment a continuous radio-side release process into two independent arrival clusters, thus amplifying the estimation of subsequent scheduling spans. Therefore, this embodiment calculates a cross-frame continuity determination factor, which is the difference between the start time of the first cluster segment in the current video frame and the end time of the last cluster segment in the previous video frame.

[0057] Step S13: Based on the cross-frame continuity determination quantity, the preliminary data cluster sequence is boundary corrected to generate a traffic pattern characterization sequence that reflects the characteristics of the underlying radio resource release.

[0058] Specifically, a continuity threshold is set. If the cross-frame continuity determination quantity is less than this threshold, the boundary is considered a false boundary and treated as a continuous part of the same underlying release process in subsequent uplink scheduling estimation. This processing only serves the scheduling structure analysis and does not change the semantic attribution and decoding relationship of the video frames themselves. The traffic pattern representation sequence finally output in step S1 records the number of data packets or the amount of data contained in each data cluster, as well as the inter-cluster interval of each data cluster relative to the previous data cluster.

[0059] Step S2: Estimate the actual scheduling occupancy of video frames and the effective carrying capacity of a single scheduling based on the traffic pattern characterization sequence.

[0060] Specifically, this step infers the additional scheduling impact in the 5G uplink, using time-slotted scheduling characteristics and based on traffic pattern analysis results to estimate the actual scheduling span experienced by video frames on the wireless side. In a 5G network environment, the end-to-end transmission link consists of both wireless and wired links. Data sent by the terminal must wait for the uplink transmission time slot to arrive before it can be sent via the wireless side, while feedback from the receiving end must be returned via the downlink transmission time slot. The figure also shows the flexible time-slot scheduling unit, as well as the retransmission time slot and the number of retransmissions. When a transmission fails on the wireless side or link conditions fluctuate, a data transmission process that should have been completed in one uplink scheduling session may be repeated across multiple subsequent time slots, resulting in additional waiting.

[0061] In an optional embodiment of the present invention, step S2 may include: Step S21: Determine a representative scheduling period based on the statistical characteristics of traffic patterns, the main peak value of the inter-cluster interval histogram, candidate period matching within the sliding window, or the preset time slot parameters of the network configuration.

[0062] Specifically, the representative scheduling period is used to map the inter-cluster interval monitored by the receiver to a count of the number of scheduling events experienced at the underlying level.

[0063] Step S22: Calculate the total number of scheduling units that the video frame actually traverses based on the representative scheduling cycle and the spacing between data clusters in the traffic pattern.

[0064] Specifically, the quotient of each inter-cluster interval in the video frame divided by a representative scheduling period is rounded down and incremented by one to obtain the scheduling occupancy count corresponding to that interval. The scheduling occupancy counts corresponding to all valid cluster intervals in the video frame are then summed to obtain the total number of scheduling units actually traversed by the video frame on the radio side. This total number indicates approximately how many radio-side scheduling periods elapsed from the release of the first data packet in the video frame to the arrival of the last data packet at the receiver. If the total number of scheduling units actually traversed is large, it indicates that the current video frame may have experienced a long waiting period, retransmission, or multiple distributed release processes on the radio side.

[0065] Step S23: Determine the candidate carrying capacity value based on the volume balance relationship of adjacent data clusters or the volume of a single cluster.

[0066] Specifically, this invention requires estimating the effective carrying capacity of a single scheduling operation, i.e., under current wireless conditions, how much effective video data can be carried in an ideal uplink scheduling operation? Candidate carrying capacity values ​​are dynamically determined based on the size patterns of adjacent data clusters: when there is a significant imbalance between two adjacent data clusters, the smaller cluster is used as the candidate carrying capacity value; when two adjacent clusters have similar sizes at small intervals, their combined statistical value is used as the candidate carrying capacity value; when a video frame contains only a single data cluster and there is no cross-frame continuity issue, the size of that single cluster is used as the candidate carrying capacity value.

[0067] Step S24: Based on the smoothing factor, perform a weighted average of the candidate carrying capacity value and the historical effective carrying capacity, and update the effective carrying capacity.

[0068] Specifically, to avoid drastic fluctuations in the estimation process, an exponential smoothing method is used to update the effective carrying capacity. The smoothing factor is multiplied by the historical effective carrying capacity, and then the difference between one and the smoothing factor is multiplied by the current candidate carrying capacity value. Finally, the two products are added together to obtain the updated effective carrying capacity. The smoothing factor is preferably between zero and one.

[0069] Step S3: Based on the actual scheduling occupancy and effective carrying capacity, construct the additional scheduling occupancy ratio.

[0070] Specifically, by comparing the actual scheduling span with the theoretical minimum scheduling requirement, the degree to which delay is affected by non-congestion factors is quantified.

[0071] In an optional embodiment of the present invention, step S3 may include: Step S31: Determine the theoretical minimum number of scheduling units based on the total payload of the video frames and the effective carrying capacity.

[0072] Specifically, the quotient of the total payload of the video frame divided by the effective payload is rounded up to obtain the theoretical minimum number of scheduling units required to complete the transmission of the video frame under the current wireless conditions.

[0073] Step S32: Calculate the difference between the total number of scheduling units actually crossed and the theoretical minimum number of scheduling units, and construct the additional scheduling occupancy ratio based on the ratio of the difference to the total number of scheduling units actually crossed.

[0074] Specifically, the actual total number of scheduling units spanned is compared with the theoretical minimum number of scheduling units. The difference between the maximum of the two and the theoretical minimum number of scheduling units is obtained. This difference is then divided by the maximum of the two to obtain the scheduling additional occupancy ratio. When the two are close, the scheduling additional occupancy ratio is close to zero, indicating that most of the time span of the current video frame can be explained by normal and effective transmission. When the actual occupancy number is significantly greater than the theoretical requirement, the scheduling additional occupancy ratio increases, indicating that a higher proportion of the time span in this video frame comes from additional factors such as scheduling waiting, distributed release, or link layer retransmission.

[0075] Step S4: Based on the scheduling additional occupancy ratio and the original delay, generate correction delay information, encapsulate the correction delay information, and send a feedback data packet to the sender.

[0076] Specifically, the receiving end generates corrected delay information through delay compensation. This allows for the purification of the input signal without altering the existing congestion controller implementation, thereby reducing the interference of non-congestion scheduling factors on control decisions.

[0077] In an optional embodiment of the present invention, step S4 may include: Step S41: Map the additional scheduling occupancy ratio or representative scheduling cycle to the original delay using a proportional adjustment coefficient or a preset mapping relationship, and determine the estimated value of the additional scheduling delay.

[0078] Specifically, the proportional adjustment coefficient is used to adjust the mapping strength of the scheduling additional occupancy ratio to the original delay. In a preferred embodiment, the proportional adjustment coefficient, the scheduling additional occupancy ratio, and the original delay are multiplied together to obtain the estimated scheduling additional delay. In actual deployments, linear mapping functions, lookup table functions, piecewise functions, or lightweight regression functions based on representative scheduling cycles can also be used to generate the estimated scheduling additional delay.

[0079] Step S42: Remove the scheduling-related delay component from the original delay to generate corrected delay information that reflects the actual link congestion status.

[0080] Specifically, the estimated scheduling-related delay component is subtracted from the original measurement value to obtain the corrected delay. The corrected delay information is then written into the feedback information to form a data packet carrying the corrected delay, which is transmitted back to the sending end via the downlink transmission link. This corrected delay information can be used to replace the original frame delay, correct the round-trip delay measurement value, or participate in the calculation of control variables such as receive rate, delay cost, and congestion status judgment.

[0081] This invention also provides a latency compensation control system for 5G uplink real-time video transmission, the overall structure of which is as follows: Figure 4 As shown, the system mainly includes a transmitting end-side functional module, a receiving end-side functional module, and a data packet and feedback data packet interaction path deployed between the two. Deployed in a 5G uplink real-time video transmission environment, the overall architecture encompasses the transmitting end, the receiving end, and a transmission path composed of wireless and wired sides. The transmitting end is equipped with a congestion controller, which includes a video encoder, a transmission module, a pacing disabling module, and a controller adaptation module. The receiving end is equipped with a video decoder, a traffic pattern analysis module, an uplink scheduling estimation module, and a latency compensation generation module.

[0082] On the transmitting end, the video encoder generates video frame data to be sent based on current control parameters. The transmission module encapsulates the encoded video frames into data packets and sends them to the network. Disabling the pacing module weakens fine-grained packet-by-packet control, allowing data packets of the same video frame to enter the network in groups within a shorter time, thereby reducing the interference of local pacing control at the transmitting end on delay observation. The controller adaptation module receives corrected delay information from the receiving end and integrates this information into the control input of the existing congestion controller to replace or correct the original delay signal used directly, thus enhancing rather than replacing the existing congestion control logic.

[0083] At the receiving end, data packets output by the transmitting end are transmitted to the receiving end via the 5G wireless link and subsequent wired links. The traffic pattern analysis module performs timing analysis on the received data packets, identifying the timing characteristics during their arrival. The uplink scheduling estimation module infers the additional impact of wireless scheduling based on the traffic pattern analysis results. The delay compensation generation module combines the original delay measurement value with scheduling additional occupancy information to estimate the additional delay components introduced by 5G uplink internal scheduling waiting, batch release, and link layer retransmission, and generates corrected delay information accordingly. This corrected delay information is fed back to the transmitting end via data packets carrying the corrected delay, allowing the controller adaptation module in the transmitting end to correct the input of the existing congestion controller, thereby guiding subsequent video encoding and transmission processes.

[0084] The specific processing procedure of this system is as follows.

[0085] Step S1, congestion controller processing on the sending end side.

[0086] This step uses a congestion controller on the sending end to convert the raw video stream into controlled batch data transmission. The congestion controller includes a video encoder, a transmission module, a pacing disable module, and a controller adapter module.

[0087] Step S11: Encode the video content to obtain video data organized by frame.

[0088] The video encoder at the sending end encodes the video content to be transmitted, generating video data organized by frame sequence.

[0089] Step S12: Configure the pacing transmission strategy to be disabled based on 5G uplink scheduling characteristics.

[0090] The sending end adjusts the sending rhythm control mode by disabling the pacing module, thereby canceling or weakening the traditional fine-grained packet-by-packet uniform sending limitation. The specific strategy is explained below: (1) Adjustment mechanism of the sending strategy: Disabling the pacing module does not require the sending end to apply a strict and uniform packet-by-packet sending rhythm to each data packet, but instead adopts a frame-level group sending method.

[0091] (2) The beneficial effects of adopting the pacing-disabled design: First, it reduces local rhythm control interference. Since the actual air interface transmission time of data packets in the 5G uplink scenario is mainly controlled by base station authorization and radio resource scheduling, uniform pacing per packet usually cannot really control the departure transmission order of the radio side. Instead, it may introduce additional buffering and waiting at the transmitting end. By disabling the pacing module, the additional impact of local scheduling at the transmitting end on delay observation can be reduced. Second, it optimizes traffic feature identification. This design makes it easier for the receiving end to identify the traffic pattern features formed by radio scheduling from the arrival time of data packets.

[0092] Step S13: Perform group encapsulation processing on the video frame data packets to obtain the data packets sent by Burst.

[0093] Each frame of video data is further divided into multiple data packets by the transmission module and sent into the network transmission path. After the video frame is encoded, the data packets belonging to that video frame are allowed to enter the network in a group in a short time in the form of a Burst.

[0094] Step S2: Receiver-side traffic pattern analysis and processing.

[0095] This step uses the traffic pattern analysis module on the receiving end to perform timing analysis on the data packets, identify the timing characteristics of the data packets during their arrival process, and provide input basis for subsequent uplink scheduling estimation.

[0096] Step S21: Record the arrival time and classify the frame of the incoming data packets to obtain the arrival time sequence organized by frame.

[0097] The receiving end records the arrival time of each data packet and, based on the frame identifier, timestamp, or sequence number in the packet header, assigns the received data packets to the corresponding video frame. For any video frame, let its corresponding data packet arrival time sequence be: in, This indicates the first [frame] in the video. The arrival time of each data packet.

[0098] Step S22: Cluster the arrival sequence based on the packet interval to obtain the traffic pattern characterization sequence.

[0099] The traffic pattern analysis module processes the arrival time series of data packets within the same video frame to extract the data cluster structure that reflects the timing characteristics of 5G uplink transmission. The specific process is described below: (1) Calculate the arrival time interval between two adjacent data packets: in, This indicates the arrival time interval between two adjacent data packets; This indicates the arrival time of the previous data packet.

[0100] (2) Perform data clustering determination. Set the intra-frame traffic pattern division threshold to... If satisfied Then it is considered that the first The data packet belongs to the same nearly consecutively arriving data cluster as the previous data packet; if the following conditions are met... Then it is considered that the first Each data packet initiates a new data cluster.

[0101] (3) Constructing a flow pattern representation sequence. After the above division, a video frame is represented as a sequence of multiple data clusters arranged in chronological order, i.e.: in, This represents the data cluster sequence obtained after the current video frame has been divided; Indicates the first The number of data packets or the corresponding amount of data contained in a data cluster; Indicates the first The inter-cluster interval of each data cluster relative to the previous data cluster; This represents the total number of data clusters. In actual implementation, It can be set to a small threshold in the millisecond range (generally configurable to 1ms), or it can be adaptively adjusted based on timestamp precision, link characteristics, or historical statistical results.

[0102] Step S23: Perform cross-frame continuity identification on adjacent frame boundaries to obtain the corrected scheduling structure information.

[0103] The traffic pattern analysis module detects data clusters at the boundaries of adjacent video frames to identify cross-frame resource release processes and eliminate false boundary interference. Details are as follows: (1) Necessity of performing cross-frame continuity identification: In the 5G uplink scenario, a single underlying resource release process may span two adjacent video frames. If it is simply truncated according to the video frame boundary, it is easy to mistakenly divide a continuous wireless side release process into two independent arrival clusters, thereby amplifying the estimation of the subsequent scheduling span.

[0104] (2) Calculate the cross-frame continuity determination factor: in, Indicates the cross-frame continuity determination quantity; Indicates the start time of the first cluster segment in the current video frame; This indicates the end time of the last cluster segment in the previous video frame.

[0105] (3) Cross-frame continuity determination and correction. Set the continuity threshold as follows: If satisfied If the boundary is false, it is considered a pseudo-boundary and treated as a continuous part of the same underlying release process in subsequent uplink scheduling estimation. It should be noted that this processing only serves the scheduling structure analysis and does not change the semantic attribution or decoding relationship of the video frame itself.

[0106] Step S3: Uplink scheduling estimation process on the receiving end side.

[0107] This step uses the uplink scheduling estimation module to infer the additional scheduling impact in the 5G uplink, and uses time-slotted scheduling characteristics and traffic pattern analysis results to estimate the actual scheduling span experienced by video frames on the wireless side.

[0108] Step S31: Determine representative scheduling cycles based on historical statistical characteristics. .

[0109] The receiver determines the representative scheduling period of the 5G uplink through statistical analysis of inter-cluster intervals. The specific methods for determining parallel arrangements are explained below: (1) Determination method based on histogram statistics.

[0110] Representative scheduling cycles are obtained by performing histogram statistics on a large number of inter-cluster interval samples and selecting the main peak value.

[0111] (2) Estimation method based on sliding window matching.

[0112] Estimation is performed within a sliding window using candidate period matching.

[0113] (3) Preset method based on network configuration parameters.

[0114] A representative scheduling period is pre-set given the network time slot configuration parameters.

[0115] in, Used to map the inter-cluster interval monitored by the receiver to a count of the number of scheduling events experienced at the underlying level.

[0116] Step S32: Calculate the actual scheduling usage of video frames based on the data cluster distribution. .

[0117] The receiving end calculates the number of scheduling units actually occupied or traversed by the current video frame on the wireless side based on the traffic pattern characterization results. The calculation process is explained below: (1) Calculate the scheduling occupancy count corresponding to a single inter-cluster interval: in, Indicates the first The scheduling occupancy count corresponding to the interval between data clusters; Indicates the first in the traffic pattern The inter-cluster interval of a data cluster relative to the previous data cluster.

[0118] (2) The total number of scheduling units actually spanned by the video frame is obtained by summing: in, This represents the total number of radio-side scheduling cycles from the release of the first data packet of a video frame to the arrival of the last data packet at the receiver.

[0119] Step S33: Estimate and update the dynamic scheduling carrying capacity based on the adjacent cluster size pattern. .

[0120] The receiver calculates in real time the effective payload that a single scheduling unit can carry under the current wireless conditions. The specific methods for estimation and updating are explained below: (1) A method for obtaining candidate carrying values ​​based on cluster volume relationship.

[0121] When two adjacent data clusters are imbalanced in size, the smaller cluster size is used as the candidate value; when two adjacent data clusters are close in size at a small interval, the combined statistic of the two is used as the candidate value; or when there is no cross-frame continuity, the size of a single cluster is used as the candidate value.

[0122] (2) A carrying capacity update method based on exponential smoothing.

[0123] in, This indicates the updated effective carrying capacity; This represents the smoothing factor, and its value is between 0 and 1. This represents the candidate carrying capacity value obtained from the current observation.

[0124] Step S34: Construct the additional occupancy ratio for scheduling based on the difference between actual and theoretical occupancy. .

[0125] The receiving end quantifies the degree to which non-congestion factors affect latency by comparing the actual scheduling span with the theoretical minimum scheduling requirement. The specific process is explained below: (1) Calculate the theoretical minimum number of scheduling units: in, This represents the theoretical minimum number of scheduling units required to complete video frame transmission under current wireless conditions. This indicates the total payload size of the current video frame.

[0126] (2) Constructing the additional occupancy ratio for scheduling: in, The additional scheduling occupancy ratio reflects the degree of impact from factors such as wireless scheduling waiting, resource allocation delays, and link layer retransmissions.

[0127] Step S4: Delay compensation generation process at the receiving end.

[0128] This step combines the original latency measurement with the additional occupancy information of the scheduling through the latency compensation generation module to estimate the additional latency component introduced by 5G uplink internal scheduling waiting, batch release and link layer retransmission in the original latency. Without changing the main implementation of the existing congestion controller, the interference of non-congestion scheduling factors on control decisions is reduced through signal purification processing.

[0129] Step S41: Calculate the scheduling additional delay component based on the original delay and the scheduling additional occupancy ratio.

[0130] The receiving end generates a scheduling-added delay estimate based on the traffic pattern analysis results and the uplink scheduling estimation results. The specific method for generating this estimate is explained below: (1) Calculation method based on linear proportional mapping.

[0131] The additional scheduling delay is obtained by weighting the original delay according to the additional scheduling occupancy ratio. The calculation formula is as follows: in, This represents the estimated additional delay during scheduling; This is a proportional adjustment coefficient used to adjust the mapping strength of the scheduling additional occupancy ratio to the original delay; Additional occupancy percentage for scheduling; This represents the original latency of the current video frame.

[0132] (2) Conversion method based on preset mapping relationship.

[0133] In actual deployment, a representative scheduling cycle is adopted. Linear mapping functions, lookup table functions, piecewise functions, or lightweight regression functions can be used to generate additional scheduling delays.

[0134] Step S42: Perform component removal processing on the original time delay to obtain the corrected time delay information.

[0135] The receiving end subtracts the estimated non-congestion add-on from the original measurement value to obtain the delay characteristic reflecting the true link congestion state. The calculation formula is as follows: in, This indicates the corrected time delay.

[0136] Step S43: Encapsulate the corrected delay information to obtain and send a feedback data packet.

[0137] The receiving end writes the generated corrected delay information into the feedback signaling, forming a data packet carrying the corrected delay, and then transmits it back to the sending end via the downlink transmission link. The corrected delay information has several applications in subsequent processing: (1) Replace the original frame delay or correct the RTT measurement value.

[0138] (2) Participate in the calculation of control variables such as receiving rate, delay cost and congestion status judgment.

[0139] Step S5: The controller adaptation and feedback interaction processing process at the sending end.

[0140] This step provides a more accurate time delay observation basis for the existing control loop by coordinating feedback between the transmitter and receiver. This enhances the system's adaptability to the complex wireless environment of 5G uplink without reconstructing the original controller's main logic, demonstrating good compatibility and engineering deployability.

[0141] Step S51: Obtain feedback data packets from the reverse link and parse them to obtain the correction delay parameters.

[0142] The controller adaptation module in the sending end receives feedback information from the receiving end, parses and extracts the compensated correction delay parameters from the data packet carrying the correction delay.

[0143] Step S52: Perform adaptation mapping processing on the parsed correction delay to obtain the congestion controller input signal.

[0144] The controller adaptation module integrates the correction delay into the existing congestion controller to replace or correct the original delay signal that was directly used. The specific application methods are explained below, depending on the implementation mechanism of different controllers: (1) Rate estimation and adaptation method.

[0145] For controllers that use frame completion time to estimate the receive rate, the compensated completion time is used instead of the original duration.

[0146] (2) Trend judgment and adaptation method.

[0147] For controllers that determine congestion trends based on round-trip or one-way delay, the corrected delay parameters are used as new delay inputs.

[0148] (3) Control logic adaptation method.

[0149] For control logic that uses delay to construct delay cost, bandwidth utilization, or window adjustment amount, the compensated delay parameters are used for the corresponding calculations.

[0150] Step S53: Based on the corrected input signal, a decision processing is performed to obtain the adjusted encoding and transmission parameters.

[0151] The transmitting end enhances the input signal layer through the controller adaptation module, guiding the congestion controller to adjust the network state input used by the video encoder and transmission module based on the purified signal. This enables precise control over the subsequent video transmission process, specifically including: (1) System gain description: This invention enhances rather than replaces the existing congestion control logic through feedback coordination between the sending end and the receiving end.

[0152] (2) Strategy decision content: Adjust the video encoding rate, transmission window or target bit rate based on the corrected signal.

[0153] Performance testing: like Figure 5As shown, the horizontal axis represents the 95th percentile frame delay, and the vertical axis represents the average bit rate. The arrows indicate the direction of better overall performance. The figure shows the performance differences in latency and bit rate between the original algorithm, the algorithm with pacing disabled, and the algorithm with further latency correction. The figure uses the 95th percentile frame delay as the horizontal axis and the average bit rate as the vertical axis to show the performance distribution of the existing algorithm, the algorithm with pacing disabled, and the algorithm with latency correction in real-time video transmission scenarios. The data points and their connections in the figure represent the balance between latency and bit rate under different control methods, and the arrows indicate the direction of better performance. This figure mainly illustrates that the present invention, through the coordinated design of the pacing-disabling module at the transmitting end and the latency compensation mechanism at the receiving end, can achieve a better ratio of average bit rate to 95th percentile frame delay, thereby improving the overall transmission performance of the 5G uplink real-time video transmission system.

[0154] from Figure 5 It can be seen that disabling pacing at the sending end alone improves system performance to some extent, indicating that reducing packet-by-packet rhythm control at the sending end helps reduce the impact of local waiting. Furthermore, by introducing a traffic pattern analysis module, an uplink scheduling estimation module, and a delay compensation generation module at the receiving end, the system achieves a better balance between 95th percentile frame delay and average bit rate. This demonstrates that the collaborative design of this invention—combining a sending end pacing disabling module, a receiving end delay compensation mechanism, and a sending end controller adaptation module—enables the delay information relied upon by the sending end to more closely approximate the actual link state, thereby achieving a better ratio of average bit rate to 95th percentile frame delay.

[0155] This invention can also be implemented by an electronic device, which includes a processor, a memory, and program instructions stored in the memory and executable by the processor, wherein the program instructions, when executed, perform the above-described method steps. Accordingly, this invention also includes a computer-readable storage medium storing the program instructions, which, when executed by a processor, implement the above-described method steps.

[0156] Although the present invention has been disclosed above with reference to embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions made by those skilled in the art to the technical solutions of the present invention should be covered within the protection scope of the present invention, which is defined by the claims.

Claims

1. A latency compensation control method for 5G uplink real-time video transmission, characterized in that, Includes the following steps: The video content is acquired and encoded to obtain video frame data; The video frame data is burst-sent to the network based on a step-by-step disabling strategy; The feedback information is obtained and the correction delay parameter is parsed, and the correction delay parameter is mapped to the congestion controller to obtain the control input signal; The video encoding parameters and data packet transmission parameters are determined based on the control input signals.

2. The method as described in claim 1, characterized in that, The video frame data is burst-sent to the network based on a step-disable strategy, including: After the video frame is encoded, the restriction on the uniform transmission interval of all data packets belonging to the same video frame is lifted, allowing the data packets to enter the transmission path in a frame-level grouped manner.

3. The method as described in claim 1, characterized in that, The feedback information is obtained and the correction delay parameter is parsed. The correction delay parameter is then mapped to the congestion controller to obtain the control input signal, including: Analyze the control mechanism type of the congestion controller; Based on the control mechanism type, the correction delay parameter is adapted to the data. The original duration is replaced by the compensated completion time, or the original delay parameter is replaced by the correction delay parameter, or the delay cost and window adjustment amount are recalculated using the correction delay parameter to generate the corresponding control input signal.

4. A latency compensation control method for 5G uplink real-time video transmission, characterized in that, Includes the following steps: Record the arrival time of received data packets and determine the traffic pattern representation sequence corresponding to the video frame; Based on the traffic pattern characterization sequence, the actual scheduling occupancy of video frames and the effective carrying capacity of a single scheduling are estimated. Based on the actual scheduling occupancy and the effective carrying capacity, a scheduling additional occupancy ratio is constructed; Based on the scheduling additional occupancy ratio and the original delay, corrected delay information is generated, and the corrected delay information is encapsulated and sent to the sending end as a feedback data packet.

5. The method as described in claim 4, characterized in that, Record the arrival time of received data packets and determine the traffic pattern representation sequence corresponding to the video frame, including: Based on the frame affiliation and arrival time of the data packets, preliminary clustering is performed according to the packet interval to obtain a preliminary data cluster sequence; Identify cross-frame continuous scheduling processes at the boundaries of adjacent video frames; Based on the cross-frame continuous scheduling process, the preliminary data cluster sequence is boundary-corrected to generate a traffic pattern representation sequence that reflects the characteristics of underlying radio resource release.

6. The method as described in claim 4, characterized in that, Based on the traffic pattern characterization sequence, the actual scheduling occupancy of video frames is estimated, including: Representative scheduling cycles are determined based on statistical characteristics of traffic patterns, the main peak value of inter-cluster interval histograms, candidate cycle matching within sliding windows, or preset time slot parameters of network configuration. The total number of scheduling units that a video frame actually traverses is calculated based on the representative scheduling period and the distance between each data cluster in the traffic pattern.

7. The method as described in claim 4, characterized in that, Based on the traffic pattern characterization sequence, the effective carrying capacity of a single scheduling is estimated, including: Candidate carrying capacity values ​​are determined based on the volume balance relationship between adjacent data clusters or the volume of a single cluster. The candidate carrying capacity value and the historical effective carrying capacity are weighted and averaged based on a smoothing factor to update the effective carrying capacity.

8. The method as described in claim 4, characterized in that, Based on the actual scheduling occupancy and the effective carrying capacity, a scheduling additional occupancy ratio is constructed, including: The theoretical minimum number of scheduling units is determined based on the total payload of the video frames and the effective carrying capacity. Calculate the difference between the total number of scheduling units actually crossed and the theoretical minimum number of scheduling units, and construct the additional scheduling occupancy ratio based on the ratio of the difference to the total number of scheduling units actually crossed.

9. The method as described in claim 4, characterized in that, Based on the aforementioned additional occupancy ratio and the original latency, corrected latency information is generated, including: By using a proportional adjustment coefficient or a preset mapping relationship, the additional scheduling occupancy ratio or representative scheduling cycle is mapped to the original delay to determine the estimated value of the additional scheduling delay. The scheduling-added delay component is removed from the original delay to generate corrected delay information that reflects the actual link congestion state.

10. A latency compensation control system for 5G uplink real-time video transmission, characterized in that, Includes the sending end and the receiving end; The sending end includes: A video encoder is used to acquire video content and encode it to obtain video frame data. Disable step module to provide a step disable strategy to weaken fine-grained packet-by-packet control; The transmission module is used to burst-send the video frame data to the network based on the step-disable strategy; The controller adaptation module is used to acquire feedback information and parse the correction delay parameters, map the correction delay parameters to the congestion controller to obtain control input signals, and make decisions on video encoding parameters and data packet transmission parameters based on the control input signals. The receiving end includes: A video decoder is used to decode and restore received video data packets; The traffic pattern analysis module is used to record the arrival time of received data packets and determine the traffic pattern representation sequence corresponding to the video frame. The uplink scheduling estimation module is used to estimate the actual scheduling occupancy of video frames and the effective carrying capacity of a single scheduling based on the traffic pattern characterization sequence, and to construct the additional scheduling occupancy ratio based on the actual scheduling occupancy and the effective carrying capacity. The delay compensation generation module is used to generate corrected delay information based on the scheduling additional occupancy ratio and the original delay, and to encapsulate the corrected delay information and send a feedback data packet to the sending end.

Images