Adaptive bitrate real-time transmission system for security monitoring streams

By optimizing the allocation of encoding resources for security surveillance videos through motion vector analysis and difference degree segmentation, the problem of unbalanced encoding between static backgrounds and moving target areas was solved, achieving efficient transmission and network stability for key target areas.

CN123123567APending Publication Date: 2026-11-17CHONGQING JIZHONG TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202611304717.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-26
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In existing security surveillance video coding, the distribution of coding resources between static backgrounds and moving target areas is uneven, resulting in a decrease in coding quality for critical target areas when the network is congested. Existing methods that increase computational load or rely on bandwidth improvements cannot effectively solve this problem.

Method used

The two-dimensional spatial displacement vector of the macroblock is obtained by the motion vector extraction module. Combined with the spatial difference analysis module, the macroblock sets are divided into high dynamic and low dynamic sets. When the network is congested, more bit resources are allocated to the high dynamic macroblock set first, and it is encapsulated into an enhanced stream for priority transmission, thereby reducing the bit consumption of static background areas.

Benefits of technology

It reduces computational load, maintains the clarity and texture discernibility of critical target areas, reduces the risk of degradation of whole-frame image information during network congestion, and improves bandwidth utilization and transmission stability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123567A_ABST
    Figure CN123123567A_ABST
Patent Text Reader

Abstract

This invention belongs to the field of video coding and compression technology, and relates to an adaptive bitrate real-time transmission system for security monitoring streams. It includes a motion vector extraction module, a spatial difference analysis module, a bitrate adjustment control module, and a heterogeneous bitstream generation module. The motion vector extraction module extracts the displacement vectors of macroblocks in the encoded frames during the inter-frame prediction stage of the video encoder. The spatial difference analysis module determines the spatial difference of the motion vectors based on the difference in displacement components between the target macroblock and its adjacent macroblocks. When channel bandwidth is limited, the bitrate adjustment control module divides the encoded frames into a high-dynamic macroblock set and a low-dynamic macroblock set based on the spatial difference, and superimposes redundant bits from the low-dynamic macroblock set onto the high-dynamic macroblock set. The heterogeneous bitstream generation module constrains the macroblock bitrate through differentiated quantization parameters. This invention utilizes the native motion vectors of the video encoder to achieve asymmetric bit allocation, eliminating local feature ambiguity caused by frame-level unified quantization.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to an adaptive bitrate real-time transmission system for security monitoring streams, belonging to the field of video encoding and video compression technology. Background Technology

[0002] Existing adaptive bitrate adjustment typically employs a feedback control approach. The system generates encoding control commands based on the occupancy of the transmission buffer or the packet loss rate in the transmission channel. The video encoding engine then adjusts the quantization parameters of the video frames according to a preset step size, making the output bitrate adapt to the dynamic changes in bandwidth. In security monitoring footage, static backgrounds usually occupy a large area, and their pixel information changes little over time. However, moving targets with actual monitoring value often only appear in local areas, and the image content in these areas changes more significantly. When the transmission channel bandwidth is limited, the existing method of adjusting the uniform quantization parameters across the entire frame will simultaneously reduce the retention of high-frequency information in the entire frame image, easily causing blurring of edge details of key targets. Therefore, this bitrate adjustment method that does not distinguish between image regions is not compatible with the information changes in different areas of security monitoring footage.

[0003] To address the aforementioned issues, existing technologies typically employ deep learning models to identify regions of interest (ROIs) or directly increase access bandwidth. However, deep learning models increase computational load and processing latency, while increasing access bandwidth is limited by deployment costs and infrastructure conditions. These methods still fail to resolve the contradiction between uniform adjustment across the entire frame and rapid changes in local content within the video encoding process itself. For example, Chinese invention patent application CN116708790A discloses a HEVC video encoding method based on motion and texture region detection. This method utilizes the Sobel operator to statistically analyze pixel gradients and combines the grayscale difference between adjacent frames to divide motion and texture regions, then adjusts the quantization parameters. In real-time transmission of security videos, this type of pixel-level feature detection imposes additional computational load on embedded front-end devices with limited computing power. The resulting processing latency affects the real-time performance of video transmission. Furthermore, pixel grayscale is easily affected by changes in illumination and sensor thermal noise, making it difficult to stably and accurately determine regional features in complex environments. In addition, this type of method does not directly reallocate bit resources in the transmission link, making it difficult to distinguish critical target-related data from other data and prioritize its transmission during instantaneous network congestion.

[0004] Therefore, the technical problem to be solved by this invention is how to utilize the existing motion vector data of the video encoder, allocate bit resources according to the changes in different image regions, and prioritize the encoding quality of key target regions when the channel is congested. Summary of the Invention

[0005] To address the problems in the background art, the technical solution of the present invention is as follows: An adaptive bitrate real-time transmission system for security monitoring streams, comprising: The motion vector extraction module obtains the two-dimensional spatial displacement vectors of each macroblock in the current encoded frame through block matching, and stores the two-dimensional spatial displacement vectors into the buffer register array; The spatial difference analysis module synchronously reads the reference displacement vectors of the target macroblock and its eight adjacent macroblocks from the cache register array. It determines the spatial difference of the motion vector by calculating the sum of the magnitudes of the horizontal and vertical component differences of the target macroblock relative to the reference displacement vectors. The bitrate adjustment control module connects the spatial difference analysis module and the quantizer of the video coding engine through a non-uniform bit allocation weight feedback link. It collects the congestion status indicators fed back by the network hardware interface. When the congestion status indicator indicates that the transmission channel bandwidth is limited, it divides the current encoded frame into a high dynamic macroblock set and a low dynamic macroblock set according to the motion vector spatial difference. It removes redundant bits from the baseline bit quota originally allocated to the low dynamic macroblock set and adds the redundant bits to the allocation limit of the high dynamic macroblock set. The heterogeneous bitstream generation module controls the quantizer to set the first quantization parameter of the low dynamic macroblock set to 51, so that the coding residual coefficient of the low dynamic macroblock set is zero, and calculates the second quantization parameter of the high dynamic macroblock set using the superimposed allocation limit to maintain the edge texture sharpness of the high dynamic macroblock set; wherein, the first quantization parameter is greater than the second quantization parameter. It also includes a heterogeneous stream encapsulation module; the heterogeneous stream encapsulation module is connected to the heterogeneous bit stream generation module, used to encapsulate high-frequency residual data in the high dynamic macroblock set into an enhanced stream with a preemptive scheduling identifier, and encapsulate data from the low dynamic macroblock set and basic motion vectors into a basic stream with a long image group period; it also includes a preemptive scheduling unit; the preemptive scheduling unit is connected to the heterogeneous stream encapsulation module, used to monitor the packet loss rate of the transmission queue, and when the packet loss rate of the transmission queue reaches a preset threshold, it ensures a constant transmission rate of the enhanced stream data packets by discarding basic stream data packets.

[0006] Preferably, it also includes a global motion compensation module; the global motion compensation module is connected between the motion vector extraction module and the spatial difference analysis module, and is used to extract the common mode motion components of the current coded frame, subtract the common mode motion components from the two-dimensional spatial displacement vector of the target macroblock, and use the corrected two-dimensional spatial displacement vector as the input for the spatial difference analysis module to calculate the spatial difference of the motion vector.

[0007] Preferably, the bitrate adjustment control module is also connected to a state machine controller; the state machine controller is used to suspend the non-uniform bit allocation weight feedback link and switch the system to a low frame rate transmission mode when the proportion of macroblocks in the high dynamic macroblock set exceeds a first threshold for a continuous preset number of frames; and to restore the normal transmission mode after the proportion drops below a second threshold for a preset duration.

[0008] Preferably, the spatial difference analysis module includes a comparison array consisting of multiple parallel subtractors for calculating the algebraic differences between the target macroblock and its eight neighboring macroblocks in the horizontal and vertical components.

[0009] Preferably, the header of the enhanced flow includes a Quality of Service (QoS) flag, which instructs the transmission network device to prioritize forwarding the enhanced flow when the basic flow and the enhanced flow compete for physical resources.

[0010] Preferably, the motion vector extraction module uses a 16×16 macroblock size for block matching.

[0011] Preferably, the quantizer includes a quantization factor mapping table; the bitrate adjustment control module constrains the differential bitrate between the high dynamic macroblock set and the low dynamic macroblock set by modifying the macroblock-level scalar quantization values ​​in the quantization factor mapping table.

[0012] Compared with the prior art, the beneficial effects of the present invention are: First, it directly utilizes the macroblock motion vectors generated by the video encoder during the inter-frame prediction stage and distinguishes between high-dynamic macroblock sets and low-dynamic macroblock sets based on the spatial difference of motion vectors. This eliminates the need to introduce an additional visual recognition model to determine the degree of change in different regions of the image, thereby reducing the computational load caused by complex semantic analysis. Furthermore, it prioritizes the allocation of bit quotas saved from static background regions to moving target regions. Even when the network is congested, key target regions can still obtain more coding resources, which helps maintain the clarity of target edges and the recognizability of textures, and avoids blurring of local key features caused by uniformly adjusting quantization parameters across the entire frame.

[0013] Secondly, high-frequency residual data in high-dynamic macroblock sets are encapsulated into enhanced streams, while data from low-dynamic macroblock sets and basic motion vectors are encapsulated into basic streams. This allows data from critical target areas to be transmitted and scheduled separately from other data. Through preemptive scheduling, enhanced streams can be sent first when basic streams and enhanced streams compete for transmission resources. When network congestion worsens, the transmission rate of enhanced stream data packets can be maintained at a constant rate by discarding basic stream data packets. This avoids data from different areas from degrading in the same way when bandwidth decreases, allowing limited transmission resources to be prioritized for critical target areas and reducing the risk of irrecoverable loss of important image information when network throughput drops sharply.

[0014] Third, by extracting the common-mode motion component of the current encoded frame and subtracting it from the two-dimensional spatial displacement vector of the macroblock, the interference of overall image motion such as camera mechanical micro-vibration on target motion judgment can be reduced, thereby reducing the probability of false triggering of the enhanced stream and improving bandwidth utilization. When the number of macroblocks in the high dynamic range macroblock set is too high, the system can also pause non-uniform bit allocation and switch to a low frame rate transmission mode to maintain the basic clarity of a single frame by reducing the frame update frequency. Therefore, even under extreme weather conditions, physical pan-tilt rotation, and other conditions that cause large-scale image changes, the transmission link can still maintain basic video backhaul, and the system operation is more stable. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the system architecture of the present invention based on motion vector space difference degree and heterogeneous flow encapsulation; Figure 2 This is a flowchart illustrating the logical interaction between network status and video encoding engine in this invention. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0017] Example 1: This example relates to an adaptive bitrate real-time transmission system for security monitoring streams, including: The motion vector extraction module obtains the two-dimensional spatial displacement vectors of each macroblock in the current encoded frame through block matching, and stores the two-dimensional spatial displacement vectors into the buffer register array; The spatial difference analysis module synchronously reads the reference displacement vectors of the target macroblock and its eight adjacent macroblocks from the cache register array. It determines the spatial difference of the motion vector by calculating the sum of the magnitudes of the horizontal and vertical component differences of the target macroblock relative to the reference displacement vectors. The bitrate adjustment control module connects the spatial difference analysis module and the quantizer of the video coding engine through a non-uniform bit allocation weight feedback link. It collects the congestion status indicators fed back by the network hardware interface. When the congestion status indicator indicates that the transmission channel bandwidth is limited, it divides the current encoded frame into a high dynamic macroblock set and a low dynamic macroblock set according to the motion vector spatial difference. It removes redundant bits from the baseline bit quota originally allocated to the low dynamic macroblock set and adds the redundant bits to the allocation limit of the high dynamic macroblock set. The heterogeneous bitstream generation module controls the quantizer to set the first quantization parameter of the low dynamic macroblock set to 51, thereby setting the coding residual coefficients of the low dynamic macroblock set to zero, and calculates the second quantization parameter of the high dynamic macroblock set using the superimposed allocation limit to maintain the edge texture sharpness of the high dynamic macroblock set; wherein, the first quantization parameter is greater than the second quantization parameter.

[0018] This embodiment also includes a global motion compensation module; the global motion compensation module is connected between the motion vector extraction module and the spatial difference analysis module, and is used to extract the common mode motion components of the current coded frame, subtract the common mode motion components from the two-dimensional spatial displacement vector of the target macroblock, and use the corrected two-dimensional spatial displacement vector as the input for the spatial difference analysis module to calculate the spatial difference of the motion vector.

[0019] This embodiment also includes a heterogeneous stream encapsulation module; the heterogeneous stream encapsulation module is connected to the heterogeneous bit stream generation module and is used to encapsulate the high-frequency residual data in the high dynamic macroblock set into an enhanced stream with a preemptive scheduling identifier, and to encapsulate the data of the low dynamic macroblock set and the basic motion vector into a basic stream with a long image group period.

[0020] This embodiment also includes a preemptive scheduling unit; the preemptive scheduling unit is connected to the heterogeneous stream encapsulation module and is used to monitor the packet loss rate of the sending queue. When the packet loss rate of the sending queue reaches a preset threshold, the unit ensures a constant sending rate of the enhanced stream data packets by discarding the basic stream data packets.

[0021] The bitrate adjustment control module in this embodiment is also connected to a state machine controller; the state machine controller is used to suspend the non-uniform bit allocation weight feedback link and switch the system to a low frame rate transmission mode when the proportion of macroblocks in the high dynamic macroblock set exceeds a first threshold for a continuous preset number of frames; and restore the normal transmission mode after the proportion drops below a second threshold for a preset duration.

[0022] The spatial difference analysis module of this embodiment includes a comparison array composed of multiple parallel subtractors, used to calculate the algebraic difference between the target macroblock and its eight neighboring macroblocks in the horizontal and vertical components.

[0023] In this embodiment, the header of the enhanced flow is provided with a Quality of Service (QoS) flag, which is used to instruct the transmission network device to prioritize forwarding the enhanced flow when the basic flow and the enhanced flow compete for physical resources.

[0024] The motion vector extraction module in this embodiment uses a 16×16 macroblock size for block matching.

[0025] The quantizer in this embodiment includes a quantization factor mapping table; the bitrate adjustment control module constrains the differential bitrate between the high dynamic macroblock set and the low dynamic macroblock set by modifying the macroblock-level scalar quantization value in the quantization factor mapping table.

[0026] Example 2: Large-scale city-level video surveillance topology networks typically contain a large number of concurrent monitoring nodes. When the basic bandwidth is limited, the normal monitoring images still have high spatiotemporal redundancy, while sudden abnormal events will significantly increase the local information entropy. When the monitoring node in the industrial park detects the movement of a target and simultaneously receives the congestion status indicator fed back by the network hardware interface, the traditional adaptive bitrate mechanism reduces the bitrate by uniformly increasing the quantization parameter at the frame level. This can easily cause the edge details of the moving target to be quantized and truncated, resulting in semantic feature ambiguity.

[0027] The motion vector extraction module performs block matching using a 16×16 macroblock size to obtain the two-dimensional spatial displacement vectors of each macroblock in the current encoded frame, and stores these vectors in a buffer register array. The spatial difference analysis module synchronously reads the reference displacement vectors of the target macroblock and its eight adjacent macroblocks from the buffer register array, and determines the spatial difference of the motion vectors by calculating the sum of the magnitudes of the horizontal and vertical component differences of the target macroblock relative to the reference displacement vectors. : ,in, For the difference in motion vector space, This is the two-dimensional spatial displacement vector of the current macroblock. This represents the two-dimensional spatial displacement vector between adjacent macroblocks, with the absolute value sign indicating the summation after taking the modulo of the difference between the horizontal and vertical components, respectively. The rate adjustment control module collects congestion status indicators fed back from the network hardware interface. When the congestion status indicator indicates that the transmission channel bandwidth is limited, it intercepts the global quantization parameter update instruction and updates it based on the spatial difference of the motion vector. The current encoded frame is divided into a set of high dynamic range macroblocks and a set of low dynamic range macroblocks.

[0028] The heterogeneous bitstream generation module controls the quantizer to set the first quantization parameter of the low-dynamic macroblock set to 51. This allows the quantizer to quantize the coding residual coefficients of the low-dynamic macroblock set at the saturation quantization level corresponding to the first quantization parameter 51, and then set the quantized coding residual coefficients to zero to release the base bit quota. The rate adjustment control module removes redundant bits from the base bit quota originally allocated to the low-dynamic macroblock set and adds the redundant bits to the allocation limit of the high-dynamic macroblock set. The heterogeneous bitstream generation module then uses the added allocation limit to calculate the second quantization parameter of the high-dynamic macroblock set, where the first quantization parameter is greater than the second quantization parameter. The quantizer has a built-in quantization factor mapping table. The rate adjustment control module writes the first and second quantization parameters as macroblock-level scalar quantization values ​​for the low-dynamic macroblock set and the high-dynamic macroblock set, respectively, into the quantization factor mapping table. Based on this, the heterogeneous bitstream generation module controls the quantizer to constrain the differentiated bit rates of the two macroblock sets, enabling the high-dynamic macroblock set to obtain more bit quotas under limited channel resources, maintaining the peak signal-to-noise ratio and edge texture sharpness of the moving target.

[0029] The global motion compensation module connects the motion vector extraction module and the spatial difference analysis module to extract the common-mode motion components of the current encoded frame. During extraction, the two-dimensional spatial displacement vectors of all macroblocks in the current encoded frame are first acquired. Local high-frequency abrupt change vectors with a vector magnitude change rate greater than a preset distribution threshold are removed, resulting in a subset of reference displacement vectors covering the static background area. This preset distribution threshold is determined based on real-time statistical results of the displacement vector magnitudes of all macroblocks in the current encoded frame. The built-in arithmetic unit calculates the expected value and standard deviation of the vector magnitudes of all macroblocks in the frame, and uses the sum of the expected value and three times the standard deviation as the filtering boundary to filter out high-energy discrete noise generated by non-rigid moving targets within the image. Subsequently, the algebraic median of the reference displacement vector subset on the horizontal and vertical coordinate axes is calculated, and the resulting two-dimensional median vector is determined as the common-mode motion component. The common-mode motion component is then subtracted from the two-dimensional spatial displacement vector of the target macroblock, and the corrected two-dimensional spatial displacement vector is used as the input to the spatial difference analysis module to distinguish between camera mechanical micro-vibrations and the actual target motion trajectory.

[0030] When the number of macroblocks in the high dynamic macroblock set exceeds 40% of the total number of macroblocks in the current encoded frame for a consecutive preset number of frames, the state machine controller suspends the non-uniform bit allocation weight feedback link and switches the system to a low frame rate transmission mode of 1Hz. This reduces the frame update frequency to maintain the basic texture recognition of a single frame image. The heterogeneous stream encapsulation module reads the partitioning results of the high dynamic macroblock set and the low dynamic macroblock set. During the encoding data output process, it encapsulates the high-frequency residual data in the high dynamic macroblock set into an enhanced stream with a preemptive scheduling flag, and encapsulates the data of the low dynamic macroblock set and the basic motion vector into a basic stream with a long image group period. The basic motion vector is a two-dimensional spatial displacement vector used by the low dynamic macroblock to perform inter-frame prediction. The basic stream organizes the encoded data according to the long image group period, sets an intra-frame encoded frame at the beginning of each image group period, and uses inter-frame prediction for the remaining encoded frames. The heterogeneous stream encapsulation module writes the preemptive scheduling flag into the header of the enhanced stream and configures it as a quality of service (QoS) flag. When the basic stream and the enhanced stream compete for physical resources, the transmission network device prioritizes forwarding the enhanced stream according to the QoS flag.

[0031] The preemptive scheduling unit reads enhanced flow data packets from the sending queue according to a fixed sending period and continuously monitors the packet loss rate of the sending queue. When the packet loss rate of the sending queue reaches a preset threshold, the preemptive scheduling unit discards the basic flow data packets in the sending queue, while maintaining the reading and sending periods of the enhanced flow data packets unchanged, in order to maintain a constant sending rate of the enhanced flow data packets and reduce the impact of network throughput degradation on the core semantic information of the monitoring screen. The spatial difference analysis module polls the cache register array stored in the parallel bus. Macroblock neighborhood motion vector matrix, extracting the horizontal components of the target macroblock and surrounding macroblocks. and vertical components The comparison array in this module consists of multiple parallel subtractors and accumulators. After the motion estimation engine of the video encoder outputs candidate motion vectors, it synchronously triggers the difference operation instruction to calculate the algebraic difference between the target macroblock and its eight neighboring macroblocks in the horizontal and vertical components. The modulo of each component difference is taken and accumulated to obtain the motion vector spatial difference. The calculation results are then written back to the decision register of the state machine controller.

[0032] Example 3: This example uses a video transmission test simulating a sudden incident on an urban arterial road. The test platform is configured with a field-programmable gate array (FPGA) hardware encoding environment, and the channel bandwidth is reduced from 8Mbps to 1.5Mbps using a network simulator. The test data is taken from standard security monitoring sequences, and Gaussian white noise with a signal-to-noise ratio of 20dB is superimposed during the motion estimation stage to simulate thermal noise interference generated by the camera's image sensor. The motion vector space difference threshold is... The value of this threshold considers both background disturbance suppression and target edge sharpness, and is used to limit bit resource consumption caused by displacement noise misjudgment when channel bandwidth is reduced. This threshold is calibrated through a pre-conducted discrete cosine transform energy distribution experiment. In a sample monitoring sequence with superimposed test thermal noise, the threshold range of 5.0 to 20.0 is scanned in steps of 0.5, and the redundant bit stripping benefit corresponding to each threshold is compared. When the threshold is below 12, the high-frequency AC coefficient truncation caused by pseudo-difference increases, and the target structure similarity index decreases significantly. When the threshold is above 12, the increase in static bit release gradually decreases. Based on the extreme inflection point of the derivative of energy change, the threshold is... Set to 12.

[0033] In this embodiment, the rate adjustment control module collects the congestion status indicator fed back by the network hardware interface. The heterogeneous bitstream generation module applies the first quantization parameter 51 to the low dynamic macroblock set. The measured value of the output residual coefficient of the low dynamic macroblock set is 0, thereby releasing redundant bits accounting for 85.4% of the original code rate. The rate adjustment control module redistributes the redundant bits to the high dynamic macroblock set, reducing its second quantization parameter from the initial 28 to 22. During a 10-second test period with a channel bandwidth of 1.5 Mbps, the control group uses frame-level uniform quantization, and the peak signal-to-noise ratio of the moving target drops from 36.5 dB to 27.2 dB, resulting in a blocky effect in the image. In this embodiment, after using non-uniform bit allocation, the peak signal-to-noise ratio of the moving target is 35.8 dB, and the measured value of the edge structure similarity index is 0.92. The bit quota obtained by the high dynamic macroblock set is used to preserve the edge texture of the moving target.

[0034] The experiment also gradually increased the proportion of macroblocks in the high dynamic range macroblock set from 20.5% to 45.0%. When the proportion was no more than 40%, the system maintained target sharpness through non-uniform bit allocation. When the proportion was greater than 42.5%, the redundant bits that the low dynamic range macroblock set could release were insufficient to meet the low quantization requirements of the high dynamic range macroblock set, and its peak signal-to-noise ratio dropped by 6.8dB. Based on this, the state machine controller suspended the non-uniform bit allocation weight feedback link and switched the system frame rate from 25Hz to 1Hz. The 42.5% execution node in the test was obtained by adding an engineering margin to the 40% basic architecture setting value. This margin was used to absorb the 2.5% motion vector space difference evaluation offset caused by the uneven distribution of high-frequency white noise in the test sample. After switching to the 1Hz frame rate mode, the peak signal-to-noise ratio of the single frame texture of the high dynamic target rebounded to 34.5dB. When the proportion of high dynamic range macroblocks was high, the single frame texture information was preserved by reducing the frame update frequency.

[0035] Motion vector space difference The saturated quantization corresponding to the first quantization parameter 51 provides a redistributable bit quota for reducing the second quantization parameter. Under the condition of reduced channel bandwidth and superimposed noise interference, the peak signal-to-noise ratio and edge structure similarity index of the sample group for moving targets in this embodiment are higher than those of the frame-level uniform quantization control group.

[0036] Example 4: This example combines Figures 1 to 2 This section describes an adaptive bitrate real-time transmission system for security monitoring streams, such as... Figure 1 As shown, the security monitoring video stream inputs the current encoded frame. The motion vector extraction module obtains the two-dimensional spatial displacement vector through block matching and transmits the two-dimensional spatial displacement vector to the global motion compensation module. The global motion compensation module deducts the common-mode motion component and corrects the displacement vector, then transmits the corrected two-dimensional spatial displacement vector to the spatial difference analysis module. The spatial difference analysis module determines the spatial difference of the motion vector and outputs the spatial difference of the motion vector to the bitrate adjustment control module. The state machine controller acts on the bitrate adjustment control module by suspending the feedback link and switching to a low frame rate transmission mode. The bitrate adjustment control module divides the data into high / low dynamic macroblock sets according to the difference and superimposes the bits, then transmits the control result to the heterogeneous bitstream generation module. The heterogeneous bitstream generation module calculates the differential quantization parameters to constrain the macroblock bitrate and transmits the processed bitstream data to the heterogeneous stream encapsulation module. The preemptive scheduling unit works in conjunction with the heterogeneous stream encapsulation module by discarding basic stream data packets to ensure a constant transmission rate of the enhanced stream. The heterogeneous stream encapsulation module encapsulates the basic stream and the enhanced stream with preemptive scheduling identifier, and finally outputs the basic stream with a long image group period and the enhanced stream with preemptive scheduling identifier, respectively.

[0037] like Figure 2 As shown, the video coding engine provides video coding data for acquiring the two-dimensional spatial displacement vector and calculating the second quantization parameter. Subtracting common-mode motion components and determining the motion vector spatial difference are applied to acquiring the two-dimensional spatial displacement vector. The network hardware interface feeds back the network status to the acquisition congestion status indicator. The acquisition congestion status indicator and suspending the feedback link and switching to a low frame rate transmission mode are applied to dividing the high dynamic range macroblock set and the low dynamic range macroblock set, respectively. Dividing the high dynamic range macroblock set and the low dynamic range macroblock set triggers the determination of the motion vector spatial difference and the removal of redundant bits and their superposition to the allocation limit. The removal of redundant bits and their superposition to the allocation limit transmits the adjusted allocation limit to the calculation of the second quantization parameter. The calculation of the second quantization parameter inputs the determined quantization parameter into the encapsulated enhanced stream and the basic stream with preemptive scheduling. The discarding of basic stream data packets works synergistically with the encapsulation of the enhanced stream and the basic stream with preemptive scheduling. The encapsulated enhanced stream and the basic stream with preemptive scheduling output the encapsulated data stream to the transmission network device. The discarding of basic stream data packets performs data packet discarding control on the sending queue.

[0038] Example 5: When outdoor high-traffic road monitoring transmits video back via a wireless bridge, environmental climate changes and multipath effects can cause high-frequency logic pulses in the congestion status indicators fed back by the network hardware interface. The system performs mid-range filtering on the congestion status indicator sequence with a sampling depth of 5 frames in a sliding window. When the proportion of congestion status indicators within the sliding window exceeds 0.8, the rate adjustment control module determines the current channel physical state as severely congested. Based on the current channel physical state, the rate adjustment control module calls the allocation weight mapping table stored in the register array of the state machine controller and calculates the mean magnitude of the two-dimensional spatial displacement vector of the high dynamic macroblock set. Using the interval where this mean value is located as an index, the dynamic gain correction factor is read from the allocation weight mapping table. It is used as a correction factor for the bit allocation weight of high dynamic macroblock sets.

[0039] The spatial difference analysis module obtains the spatial difference of motion vectors for each macroblock. Then, the difference threshold between this value and the motion vector space calculated by the scene base entropy evaluation unit is determined. A comparison was made; specifically for road monitoring scenarios with complex background textures. The initial value is set to 15.2 to suppress misjudgments of displacement noise caused by tree swaying or changes in light and shadow. The scene base entropy assessment unit runs during the idle time window at the initial stage of system link establishment. It sequentially reads the two-dimensional spatial displacement vector magnitude data of the first 50 frames of full-frame macroblocks through a parallel shift window, and then performs time-based integral averaging and high-pass bypass filtering to obtain the background discrete baseline generated by the continuous dynamic disturbance of the background within the target field of view. After receiving the superimposed allocation limit, the heterogeneous bitstream generation module calculates the second quantization parameter of the high dynamic macroblock set according to the bit allocation model. This process is achieved through bitrate quota. With quantization step size Implementation of the inverse proportional mapping function: ,in, Quantize the step size for the target. The total pixel size of the high dynamic range macroblock set. To allocate quotas, These are the feature coefficients related to the complexity of image texture.

[0040] The heterogeneous bitstream generation module obtains the target quantization step size. Then, according to the logarithmic scaling relationship of the video coding standard, it is converted into scalar quantization parameters that conform to the underlying hardware register specification: ,in, This represents the second quantization parameter. The system extracts the integer part of the calculation result, and the bitrate adjustment control module writes this integer as the macroblock-level scalar quantization value corresponding to the high dynamic macroblock set into the quantizer's quantization factor mapping table, and simultaneously writes it into the quantizer parameter addressing space, forming a conversion process from allocation limit to quantizer control parameters. To prevent the state machine controller from repeatedly switching frame rates near the critical bandwidth, the system adopts hysteresis judgment logic. When the proportion of macroblocks in the high dynamic macroblock set exceeds 42.5% for 3 consecutive frames, the state machine controller issues a 1Hz low frame rate switching command. When this proportion drops below 35% and lasts for more than 2 seconds, the system returns to the normal encoding mode of 25Hz.

[0041] The above calibration and adjustment process converts the channel physical state into a second quantization parameter that the quantizer can execute, and reduces parameter mutations caused by network fluctuations through hysteresis determination. When the system is in a high-dynamic scenario and the channel bandwidth drops to 800kbps, the bit rate occupancy of the high-dynamic macroblock set is kept within 75% of the available bandwidth. The heterogeneous bitstream generation module increases the first quantization parameter of the low-dynamic macroblock set to 51, so that the available bit quota is concentratedly allocated to the high-dynamic macroblock set, maintaining the quantization output of the edge contour of the moving target. The system maintains video transmission in unstable backhaul links through non-uniform bit allocation and frame rate hysteresis control.

[0042] Example 6: In the initial deployment of a cross-regional security system, the channel quality benchmark has not yet been determined. The system constructs an allocation weight mapping table in a controlled offline environment and reads the data from the target system. A sample set consisting of standard video sequences corresponding to preset complexity gradients is used to calculate the ratio of bitrate gain to target peak signal-to-noise ratio loss for each video sequence using different combinations of quantization parameters. For positive integers, the system establishes a binary lookup table between the channel congestion level and the bit stripping weight based on the calculation results, and calculates the channel gain coefficient during the calibration process. The system collects packet loss rate data from 1000 sampling points, and obtains a weighting factor characterizing the channel packet loss characteristics through least squares fitting. This weighting factor is then used as... Based on this, the upper limit of redundant bit stripping corresponding to different bandwidth drop intervals is determined. The fitting process is iterated 50 times in an offline environment to reduce measurement random errors. The obtained quantization gain regression coefficients are stored in the read-only memory of the bit rate adjustment control module.

[0043] In an indoor underground parking garage with low illumination and severe light noise interference, the system first acquires a 60-second static video stream of the background under no-load conditions. It then extracts the mean value of the pseudo-two-dimensional spatial displacement vector generated by sensor shot noise in each macroblock. The spatial difference analysis module uses the background difference offset corresponding to this mean value as the scene noise compensation amount. Then, a single uniformly moving target is introduced into the scene for dynamic feature matching, and the initial threshold is corrected according to the following formula: ,in, As a preset baseline difference constant, The threshold for the difference in motion vector space is used to compensate for the scene noise floor. This is used to distinguish between the pseudo-two-dimensional spatial displacement vector corresponding to the thermal noise generated by random charge fluctuations and the two-dimensional spatial displacement vector corresponding to the actual macroblock movement trajectory.

[0044] Example 7: During system calibration, a preset is made by A sample set consisting of a sequence of reference images showing brightness gradients, wherein... For positive integers, the motion vector extraction module obtains the macroblock residual energy distribution under static background, and determines the basic information entropy offset corresponding to different illumination intensities. For each set of reference image sequences, the heterogeneous bitstream generation module iterates through the quantization step size step by step. Obtain the corresponding measured output bitrate And calculate the image texture complexity feature coefficients according to the following formula. : ,in, To quantize the step size, To measure the actual output bitrate, Given the total number of pixels in the current image sequence, the system performs least-squares fitting on 20 sequences of different complexities to determine the image texture complexity feature coefficients. The center value is fixed in the read-only memory of the bitrate adjustment control module for use in the calculation of the second quantization parameter of the high dynamic macroblock set. This calibration process is only performed offline. The quantization step size is used as the independent variable for step-by-step testing. The image texture complexity feature coefficient, which characterizes the background attributes of the video source, is obtained by fitting the measured output bitrate. In the real-time communication stage, the system directly reads the coefficient as a known fixed value and solves the target quantization step size in reverse.

[0045] When the congestion status indicator fed back by the network hardware interface shows logical flip-flops, the bitrate adjustment control module is activated for a specified duration. The frame shift register window calculates the logical state flipping frequency of the congestion state identifier within the window. ,when The electromagnetic interference frequency exceeds the preset threshold. At that time, the system will determine the current hardware feedback as fault data, whereby... It is a positive integer. This represents the number of state transitions per unit of time. This is the frequency threshold.

[0046] After hardware feedback fails, the system suspends the corresponding feedback link, reads the physical backlog of the sending queue through a flow counter, and calculates the logical throughput based on its rate of change. Based on the flow conservation principle during the queuing process, the system synchronously collects the average code distribution rate within the target time window. Calculate the algebraic difference of physical accumulation within adjacent sampling periods. Then, the arithmetic logic unit solves the problem using the following formula: ,in, The physical backlog is the time span between two consecutive data samples, taken from the difference between the memory address pointer cursors of the front-end chip's static cache register. Its internal bus data flow occurs before the external wireless bridge's RF transceiver link, and is not directly affected by air interface handshake failures or retransmission pulses. The rate adjustment control module uses logical throughput. The invalidated congestion status identifier is used as the input criterion for updating the second quantization parameter.

[0047] When the heterogeneous stream encapsulation module encapsulates the enhanced stream, it writes a Quality of Service (QoS) flag in the extended field of the Real-Time Transport Protocol (RTP) header as a preemptive scheduling identifier. Enhanced stream data packets corresponding to the high dynamic macroblock set set the QoS flag to 1, while basic stream data packets set the QoS flag to 0. The preemptive scheduling unit retrieves the sending queue based on the QoS flag and continuously monitors the packet loss rate and sending buffer water level of the sending queue. When the packet loss rate of the sending queue reaches a preset threshold, the preemptive scheduling unit suspends the reading process of basic stream data packets with a QoS flag of 0 and discards the basic stream data packets to maintain a constant sending rate of enhanced stream data packets. When the sending buffer water level reaches 85% of the physical capacity and the network congestion status indicator remains valid, the network adapter's direct memory access channel is used first to send enhanced stream data packets with a QoS flag of 1.

Claims

1. An adaptive bitrate real-time transmission system for security monitoring streams, characterized in that, include: The motion vector extraction module obtains the two-dimensional spatial displacement vectors of each macroblock in the current encoded frame through block matching, and stores the two-dimensional spatial displacement vectors into the buffer register array; The spatial difference analysis module synchronously reads the reference displacement vectors of the target macroblock and its eight adjacent macroblocks from the cache register array. It determines the spatial difference of the motion vector by calculating the sum of the magnitudes of the horizontal and vertical component differences of the target macroblock relative to the reference displacement vectors. The bitrate adjustment control module connects the spatial difference analysis module and the quantizer of the video coding engine through a non-uniform bit allocation weight feedback link. It collects the congestion status indicators fed back by the network hardware interface. When the congestion status indicator indicates that the transmission channel bandwidth is limited, it divides the current encoded frame into a high dynamic macroblock set and a low dynamic macroblock set according to the motion vector spatial difference. It removes redundant bits from the baseline bit quota originally allocated to the low dynamic macroblock set and adds the redundant bits to the allocation limit of the high dynamic macroblock set. The heterogeneous bitstream generation module controls the quantizer to set the first quantization parameter of the low dynamic macroblock set to 51, so that the coding residual coefficient of the low dynamic macroblock set is zero, and calculates the second quantization parameter of the high dynamic macroblock set using the superimposed allocation limit to maintain the edge texture sharpness of the high dynamic macroblock set; wherein, the first quantization parameter is greater than the second quantization parameter. It also includes a heterogeneous stream encapsulation module; the heterogeneous stream encapsulation module is connected to the heterogeneous bit stream generation module, used to encapsulate high-frequency residual data in the high dynamic macroblock set into an enhanced stream with a preemptive scheduling identifier, and encapsulate data from the low dynamic macroblock set and basic motion vectors into a basic stream with a long image group period; it also includes a preemptive scheduling unit; the preemptive scheduling unit is connected to the heterogeneous stream encapsulation module, used to monitor the packet loss rate of the transmission queue, and when the packet loss rate of the transmission queue reaches a preset threshold, it ensures a constant transmission rate of the enhanced stream data packets by discarding basic stream data packets.

2. The adaptive bitrate real-time transmission system for security monitoring streams according to claim 1, characterized in that, It also includes a global motion compensation module; the global motion compensation module is connected between the motion vector extraction module and the spatial difference analysis module, and is used to extract the common mode motion components of the current coded frame, subtract the common mode motion components from the two-dimensional spatial displacement vector of the target macroblock, and use the corrected two-dimensional spatial displacement vector as the input for the spatial difference analysis module to calculate the spatial difference of the motion vector.

3. The adaptive bitrate real-time transmission system for security monitoring streams according to claim 1, characterized in that, The bitrate adjustment control module is also connected to a state machine controller; the state machine controller is used to suspend the non-uniform bit allocation weight feedback link and switch the system to a low frame rate transmission mode when the proportion of macroblocks in the high dynamic macroblock set exceeds the first threshold for a continuous preset number of frames; and restore the normal transmission mode after the proportion drops below the second threshold and continues for a preset duration.

4. The adaptive bitrate real-time transmission system for security monitoring streams according to claim 1, characterized in that, The spatial difference analysis module includes a comparison array consisting of multiple parallel subtractors, used to calculate the algebraic difference between the target macroblock and its eight neighboring macroblocks in the horizontal and vertical components.

5. The adaptive bitrate real-time transmission system for security monitoring streams according to claim 1, characterized in that, The header of an enhanced flow contains a Quality of Service (QoS) flag, which instructs the transmission network device to prioritize forwarding the enhanced flow when there is a physical resource contention between the basic flow and the enhanced flow.

6. The adaptive bitrate real-time transmission system for security monitoring streams according to claim 1, characterized in that, The motion vector extraction module uses a 16×16 macroblock size for block matching.

7. The adaptive bitrate real-time transmission system for security monitoring streams according to claim 1, characterized in that, The quantizer includes a quantization factor mapping table; the rate adjustment control module constrains the differential bitrate between the high dynamic macroblock set and the low dynamic macroblock set by modifying the macroblock-level scalar quantization values ​​in the quantization factor mapping table.

Citation Information

Patent Citations

  • HEVC video coding method based on motion and texture region detection

    CN116708790A