FPGA-based visual lossless video transmission system

Abstract

The application relates to the technical field of lossless video transmission, and discloses a visual lossless video transmission system based on FPGA, which comprises a video acquisition and processing module, a visual lossless processing module, a data transmission module and a decoding and display module.

Description

Technical Field

[0001] This invention relates to the field of lossless video transmission technology, and more specifically, to a visual lossless video transmission system based on FPGA. Background Technology

[0002] Traditional video transmission systems often face a trade-off between transmission bandwidth and video quality. Pursuing high video quality results in massive amounts of raw video data, demanding extremely high bandwidth and making real-time transmission difficult. Conversely, excessive video compression to accommodate limited bandwidth can lead to loss of image detail and distortion, failing to achieve visually lossless quality. This is particularly problematic in fields with stringent video quality requirements, such as medical image transmission, industrial monitoring, and high-definition video conferencing, where traditional video compression and transmission solutions struggle to balance transmission efficiency with visual losslessness. Furthermore, existing software-based video processing solutions also suffer from bottlenecks in processing speed and real-time performance, making it difficult to meet the real-time processing and transmission needs of high-speed video streams.

[0003] Therefore, it is necessary to design an FPGA-based lossless video transmission system to solve the problems existing in the current technology. Summary of the Invention

[0004] In view of this, the present invention proposes a visual lossless video transmission system based on FPGA, which aims to solve the problems of difficulty in balancing transmission bandwidth and video quality and insufficient real-time performance of software processing schemes in traditional video transmission systems.

[0005] This invention proposes a lossless visual video transmission system based on FPGA, comprising: The video acquisition and processing module is used to acquire the video to be processed, and to perform format conversion and frame structure processing on the video to be processed in order to obtain the video to be transmitted. The visual lossless processing module is used to compress the video to be transmitted under a preset delay constraint based on an FPGA hardware architecture and a parallel pipeline structure to obtain a compressed video to be transmitted; it is also configured to adjust the encoding parameters of the compressed video to be transmitted based on the image visual features so that the encoded compressed video to be transmitted meets a preset visual lossless threshold to generate a compressed video bitstream. The data transmission module is used for cache management and link transmission of the compressed video stream; The decoding and display module is used to decode and reconstruct the received compressed video stream and output the video signal.

[0006] Further, when compressing the video to be transmitted under a preset delay constraint to obtain the compressed video to be transmitted, the process includes: In the FPGA hardware architecture, a parallel pipeline structure with multi-level processing units is constructed, wherein each level of the multi-level processing unit corresponds to a different processing stage in the video compression process. A preset delay limit is determined, and based on the preset delay limit, the operation clock frequency and data throughput bandwidth of each level processing unit are allocated and adjusted by configuring the internal logic resources of the FPGA so that the overall pipeline processing delay does not exceed the preset delay limit. The video to be transmitted is divided into multiple data blocks by frame and input into the parallel pipeline through multiple parallel channels. The multi-level processing unit sequentially completes the transformation processing, quantization processing and encoding processing, and outputs compressed encoded data. The compressed encoded data is integrated to generate the compressed video to be transmitted.

[0007] Furthermore, when allocating and adjusting the clock frequency and data throughput bandwidth of each processing unit by configuring the internal logic resources of the FPGA, this includes: Collect the real-time data processing volume and corresponding processing delay information for each processing unit. Based on the preset delay limit, calculate the target processing time corresponding to each processing unit and determine the target data throughput capacity corresponding to the target processing time; Based on the target processing time and target data throughput, the number of logic units occupied, the number of parallel computing channels, and the working clock frequency of each processing unit are adjusted. For each adjusted processing unit, latency detection is performed. When the processing latency of any processing unit exceeds the preset latency limit, logic resources are reallocated or parallelism is increased. Latency detection and resource adjustment are repeated until the overall processing latency of the parallel pipeline does not exceed the preset latency limit.

[0008] Furthermore, when adjusting the number of logic units, the number of parallel computing channels, and the operating clock frequency of each processing unit based on the target processing time and target data throughput, the adjustments include: Establish a mapping relationship between the number of logical units occupied and the target data throughput capacity; Collect the data processing complexity corresponding to each level of processing unit, and preliminarily determine the number of logical units occupied by each level of processing unit based on the mapping relationship and data processing complexity. Based on the target data throughput capacity and the number of logical units occupied, the number of parallel computing channels of each processing unit is increased according to the preset parallelism enhancement rules. Based on the available clock resources inside the FPGA and the configuration of the logic units and parallel channels, the operating clock frequency corresponding to each processing unit is set, and the verification steps confirm that each processing unit can operate stably at the corresponding operating clock frequency. When the combination of logic units, parallel channels, and clock frequencies cannot meet the target processing time or target data throughput, the number of logic units, the number of parallel computing channels, or the clock frequency are iteratively adjusted until the processing unit can achieve the target data throughput within the target processing time.

[0009] Furthermore, when initially determining the number of logical units required for each processing level based on the mapping relationship and data processing complexity, the following steps are included: Based on the mapping relationship, determine the initial number of logic units occupied by each processing unit. Obtain the standard complexity corresponding to the data processing complexity, and calculate the ratio of the data processing complexity to the standard complexity, denoted as the complexity ratio. The optimization coefficient for the initial number of logic units is determined based on the complexity ratio, and the product of the optimization coefficient and the initial number of logic units is used as the number of logic units occupied.

[0010] Furthermore, when setting the operating clock frequency for each processing unit, taking into account the available clock resources within the FPGA and the configuration of the logic units and parallel channels, the following steps are included: The operating clock frequency for each processing unit is determined based on the number of logic units occupied and the number of parallel computing channels corresponding to each processing unit. The number of logic units occupied is compared with the threshold number of logic units occupied, and the number of parallel computing channels is compared with the threshold number of parallel computing channels. The working clock frequency is determined based on the comparison results. When the number of logic units occupied is greater than or equal to the threshold number of logic units occupied, and the number of parallel computing channels is greater than or equal to the threshold number of parallel computing channels, the working clock frequency is determined to be the first working clock frequency. When the number of logic units occupied is less than the threshold for the number of logic units occupied, and the number of parallel computing channels is greater than or equal to the threshold for the number of parallel computing channels, the working clock frequency is determined to be the second working clock frequency. When the number of logic units occupied is greater than or equal to the threshold for the number of logic units occupied, and the number of parallel computing channels is less than the threshold for the number of parallel computing channels, the working clock frequency is determined to be the third working clock frequency. When the number of logic units occupied is less than the threshold for the number of logic units occupied, and the number of parallel computing channels is less than the threshold for the number of parallel computing channels, the working clock frequency is determined to be the fourth working clock frequency.

[0011] Furthermore, when the processing latency of any processing unit exceeds the preset latency limit, the reallocation of logical resources or increase in parallelism includes: Calculate the latency exceedance for each processing unit; Based on the latency exceeding the limit, determine the corresponding exceeding level of the processing unit, and determine the corresponding resource adjustment scheme or parallelism adjustment scheme based on the exceeding level; When the level of exceeding the limit is slightly exceeding the limit, select the first type of resource adjustment scheme or the first type of parallelism adjustment scheme; When the over-limit level is moderate over-limit, select the second type of resource adjustment scheme or the second type of parallelism adjustment scheme; When the level of exceeding the limit is severe, select the third type of resource adjustment scheme or the third type of parallelism adjustment scheme.

[0012] Further, adjusting the encoding parameters of the compressed video to be transmitted based on image visual features to ensure that the encoded compressed video meets a preset visual lossless threshold, in order to generate a compressed video bitstream, includes: The image feature information of the first N frames of the compressed video to be transmitted is stored, including average brightness, contrast, texture complexity and motion vector information; The four feature indicators of the current frame image—brightness, contrast, texture complexity, and motion speed—are compared with the feature information of the previous N frames to calculate the feature difference value between the current frame and the historical frames. The trend of visual information change in the current frame is determined based on the feature difference values; Based on the feature difference values ​​and the trend of visual information changes, the quantization step size, DCT transform coefficient scaling ratio and inter-frame coding mode are adjusted so that the adjusted compressed video to be transmitted meets the preset visual lossless threshold.

[0013] Furthermore, when determining the trend of visual information change in the current frame based on the feature difference values, the following steps are included: The average brightness of the current frame is compared frame by frame with the average brightness of the previous N frames to calculate the brightness change rate, and the brightness change amplitude is determined based on the brightness change rate. The texture complexity of the current frame is compared frame by frame with the texture complexity of the previous N frames to calculate the texture complexity change rate, and the texture change magnitude is determined based on the texture complexity change rate. The motion vector information of the current frame is compared with the motion vector information of the previous N frames frame by frame to calculate the motion vector difference, and the motion change amplitude is determined based on the motion vector difference. Based on the brightness change amplitude, texture change amplitude, and motion change amplitude, determine the visual information change trend of the current frame; The visual information change trends include brightness increase / decrease, texture enhancement / reduction, and motion acceleration / deceleration.

[0014] Further, based on the feature difference values ​​and the trend of visual information changes, the quantization step size, DCT transform coefficient scaling ratio, and inter-frame coding mode are adjusted so that the adjusted compressed video to be transmitted meets the preset visual lossless threshold, including: The adjustment range of the quantization step size is determined based on the brightness change range. When the brightness increase is greater than the preset brightness threshold, the quantization step size is reduced by a preset step size ratio; when the brightness decrease is greater than the preset brightness threshold, the quantization step size is increased by a preset step size ratio. The scaling ratio of DCT transform coefficients is adjusted according to the texture change amplitude. When the texture enhancement amplitude exceeds the preset texture threshold, the scaling ratio of high-frequency DCT coefficients is increased to retain more details; when the texture reduction amplitude exceeds the preset texture threshold, the scaling ratio of high-frequency DCT coefficients is decreased to reduce bitstream fluctuations. The inter-frame coding mode is adjusted according to the magnitude of motion change. When the acceleration of motion exceeds a preset motion threshold, the proportion of P-frames or B-frames used is increased; when the deceleration of motion exceeds a preset motion threshold, the proportion of P-frames or B-frames used is decreased. The quantization step size, DCT transform coefficient scaling ratio, and inter-frame coding mode are jointly verified to ensure that the adjusted video compression result is within the preset visual lossless threshold range.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention realizes parallel pipeline processing through FPGA hardware architecture, which effectively improves the real-time performance of video compression and transmission and solves the speed bottleneck of traditional software processing solutions; at the same time, based on the image visual features, the encoding parameters are dynamically adjusted to ensure the visual lossless effect of the video while ensuring transmission efficiency, and successfully balances the contradiction between transmission bandwidth and video quality, which is especially suitable for high-requirement fields such as medical imaging and industrial inspection. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 The diagram shows the structural block diagram of the FPGA-based lossless video transmission system provided in this embodiment of the invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] See Figure 1 As shown in some embodiments of this application, this embodiment provides a visual lossless video transmission system based on FPGA, including: The video acquisition and processing module is used to acquire the video to be processed, and to perform format conversion and frame structure processing on the video to be processed in order to obtain the video to be transmitted. The visual lossless processing module is used to compress the video to be transmitted under a preset delay constraint based on an FPGA hardware architecture and a parallel pipeline structure to obtain a compressed video to be transmitted; it is also configured to adjust the encoding parameters of the compressed video to be transmitted based on the image visual features so that the encoded compressed video to be transmitted meets a preset visual lossless threshold to generate a compressed video bitstream. The data transmission module is used for cache management and link transmission of the compressed video stream; The decoding and display module is used to decode and reconstruct the received compressed video stream and output the video signal.

[0021] It is understood that the FPGA-based visual lossless video transmission system provided in this embodiment implements parallel pipelined processing through FPGA (Field Programmable Gate Array) hardware architecture, which effectively improves the real-time performance of video compression and transmission and solves the speed bottleneck of traditional software processing solutions. At the same time, it dynamically adjusts the encoding parameters based on image visual features, ensuring the visual lossless effect of the video while guaranteeing transmission efficiency, and successfully balancing the contradiction between transmission bandwidth and video quality. It is especially suitable for high-requirement fields such as medical imaging and industrial inspection.

[0022] Specifically, when compressing the video to be transmitted under a preset delay constraint to obtain the compressed video to be transmitted, the process includes: In the FPGA hardware architecture, a parallel pipeline structure with multi-level processing units is constructed, wherein each level of the multi-level processing unit corresponds to a different processing stage in the video compression process. A preset delay limit is determined, and based on the preset delay limit, the operation clock frequency and data throughput bandwidth of each level processing unit are allocated and adjusted by configuring the internal logic resources of the FPGA so that the overall pipeline processing delay does not exceed the preset delay limit. The video to be transmitted is divided into multiple data blocks by frame and input into the parallel pipeline through multiple parallel channels. The multi-level processing unit sequentially completes the transformation processing, quantization processing and encoding processing, and outputs compressed encoded data. The compressed encoded data is integrated to generate the compressed video to be transmitted.

[0023] Understandably, the processing steps include transformation, quantization, and encoding. The preset delay limit refers to the maximum allowable processing delay in a video transmission system; this time needs to be set according to the real-time requirements of the specific application scenario.

[0024] It is understandable that the step of generating the compressed video to be transmitted in this embodiment utilizes the parallel pipeline design of the FPGA hardware architecture to decompose the traditional serial video compression process into multiple independent processing stages, which are executed in parallel by dedicated processing units, significantly shortening the processing time of a single frame of video. For example, data blocks of the video to be transmitted, divided into frames, can simultaneously enter different processing units of the pipeline through different parallel channels. When the data block of the previous frame is quantized, the data block of the next frame can be transformed synchronously, realizing the pipelineization and parallelization of data processing, significantly improving the video compression speed, and ensuring that the system meets the preset delay constraints. At the same time, the operation clock frequency and data throughput bandwidth of each processing unit are precisely allocated and adjusted to ensure the efficient operation of the pipeline without data congestion or excessive processing delay, laying a stable processing foundation for subsequent adjustment of encoding parameters based on image visual features.

[0025] Specifically, when allocating and adjusting the clock frequency and data throughput bandwidth of each processing unit by configuring the internal logic resources of the FPGA, the following is included: Collect the real-time data processing volume and corresponding processing delay information for each processing unit. Based on the preset delay limit, calculate the target processing time corresponding to each processing unit and determine the target data throughput capacity corresponding to the target processing time; Based on the target processing time and target data throughput, the number of logic units occupied, the number of parallel computing channels, and the working clock frequency of each processing unit are adjusted. For each adjusted processing unit, latency detection is performed. When the processing latency of any processing unit exceeds the preset latency limit, logic resources are reallocated or parallelism is increased. Latency detection and resource adjustment are repeated until the overall processing latency of the parallel pipeline does not exceed the preset latency limit.

[0026] In this embodiment, the target processing time refers to the maximum time allowed for each processing unit to complete its corresponding video compression processing step (such as transform processing, quantization processing, or encoding processing) under the constraint of a preset delay upper limit, in order to ensure that the overall processing delay of the parallel pipeline does not exceed the upper limit. The target data throughput capacity refers to the data processing rate that each processing unit must have to complete its processing task within the corresponding target processing time, that is, the amount of data that can be processed per unit time.

[0027] Understandably, the system dynamically monitors the pipeline's operational status by collecting real-time data processing volume and latency information from each processing unit. Based on a preset latency limit, the system precisely calculates the target processing time for each processing unit. For example, with an overall latency limit of 50 milliseconds, and the system containing three processing units (transform, quantization, and encoding), 15 milliseconds can be allocated for transformation, 10 milliseconds for quantization, and 20 milliseconds for encoding, with the remaining 5 milliseconds used as a buffer. Then, based on the target processing time and the amount of video data to be processed, the system determines the target data throughput capacity for each unit. For example, for a video with a resolution of 1920×1080 and a frame rate of 30fps, if the encoding processing unit completes one frame processing within 20 milliseconds, the target data throughput capacity is no less than approximately 311 MB / s. Next, the system optimizes the configuration of the number of internal logic units within the FPGA based on the target parameters. For example, it allocates more DSP and BRAM to the transformation processing unit; adjusts the number of parallel computing channels, such as setting the transformation processing unit to 8 channels; and increases the operating clock frequency, such as raising the encoding processing unit to 200MHz. After initial configuration, the system performs latency detection on each processing unit. If the actual processing latency of a unit (such as the quantization processing unit) exceeds the target time (e.g., 12 milliseconds exceeding 10 milliseconds), the system will take measures such as increasing the number of parallel computing channels and allocating logic resources, and then re-detect. This process is repeated iteratively until the overall processing latency of the parallel pipeline is controlled within the 50-millisecond latency limit, providing hardware performance assurance for real-time video compression and transmission.

[0028] Specifically, when adjusting the number of logic units, the number of parallel computing channels, and the operating clock frequency of each processing unit based on the target processing time and target data throughput, the following steps are taken: Establish a mapping relationship between the number of logical units occupied and the target data throughput capacity; Collect the data processing complexity corresponding to each level of processing unit, and preliminarily determine the number of logical units occupied by each level of processing unit based on the mapping relationship and data processing complexity. Based on the target data throughput capacity and the number of logical units occupied, the number of parallel computing channels of each processing unit is increased according to the preset parallelism enhancement rules. Based on the available clock resources inside the FPGA and the configuration of the logic units and parallel channels, the operating clock frequency corresponding to each processing unit is set, and the verification steps confirm that each processing unit can operate stably at the corresponding operating clock frequency. When the combination of logic units, parallel channels, and clock frequencies cannot meet the target processing time or target data throughput, the number of logic units, the number of parallel computing channels, or the clock frequency are iteratively adjusted until the processing unit can achieve the target data throughput within the target processing time.

[0029] It is understandable that establishing a mapping relationship between the number of logic units occupied and the target data throughput capacity is the foundation for achieving precise resource allocation. This mapping relationship is usually based on the hardware characteristics of the FPGA chip, such as the operational performance and storage capacity of different types of logic units (e.g., lookup tables (LUTs), flip-flops (FFs), digital signal processors (DSPs), block random access memory (BRAM), etc.), as well as the logic resource requirements of specific video processing algorithms (e.g., DCT (Discrete Cosine Transform) / IDCT (Inverse Discrete Cosine Transform) in transform processing, scalar / vector quantization in quantization processing, entropy coding in encoding processing, etc.). Through extensive simulation experiments and hardware testing, the theoretical minimum number of various logic units occupied under different data throughput capacity requirements has been statistically determined. For example, for a transform processing unit, if the target data throughput requires processing 1,000 16x16 image blocks per second, depending on the hardware implementation complexity of the DCT algorithm, it may need to be mapped to 200 LUTs, 150 FFs, and 8 DSP units.

[0030] Specifically, when initially determining the number of logical units required for each processing level based on the mapping relationship and data processing complexity, the following steps are included: Based on the mapping relationship, determine the initial number of logic units occupied by each processing unit. Obtain the standard complexity corresponding to the data processing complexity, and calculate the ratio of the data processing complexity to the standard complexity, denoted as the complexity ratio. The optimization coefficient for the initial number of logic units is determined based on the complexity ratio, and the product of the optimization coefficient and the initial number of logic units is used as the number of logic units occupied.

[0031] It's understandable that standard complexity refers to a representative benchmark data processing complexity value determined through extensive experimental verification under specific video processing algorithms and FPGA chip environments. The optimization coefficient for determining the initial number of logic units is based on the complexity ratio; that is, the coefficient for configuring logic units is dynamically adjusted according to the ratio of data processing complexity to standard complexity. For example, if the data processing complexity is 1.5 times the standard complexity, and the complexity ratio is 1.5, the optimization coefficient can be set to 1.5, multiplying the initial number of logic units by 1.5 to accommodate higher processing complexity. If the data processing complexity is 0.8 times the standard complexity, and the complexity ratio is 0.8, the optimization coefficient can be set to 0.8, reducing the number of logic units and avoiding resource waste. In this way, the system can flexibly adjust the configuration scale of logic units according to the actual data processing complexity of each processing unit, efficiently utilizing FPGA logic resources while meeting the target data throughput requirements.

[0032] Specifically, when setting the operating clock frequency for each processing unit, taking into account the available clock resources within the FPGA and the configuration of the logic units and parallel channels, the following steps are taken: The operating clock frequency for each processing unit is determined based on the number of logic units occupied and the number of parallel computing channels corresponding to each processing unit. The number of logic units occupied is compared with the threshold number of logic units occupied, and the number of parallel computing channels is compared with the threshold number of parallel computing channels. The working clock frequency is determined based on the comparison results. When the number of logic units occupied is greater than or equal to the threshold number of logic units occupied, and the number of parallel computing channels is greater than or equal to the threshold number of parallel computing channels, the working clock frequency is determined to be the first working clock frequency. When the number of logic units occupied is less than the threshold for the number of logic units occupied, and the number of parallel computing channels is greater than or equal to the threshold for the number of parallel computing channels, the working clock frequency is determined to be the second working clock frequency. When the number of logic units occupied is greater than or equal to the threshold for the number of logic units occupied, and the number of parallel computing channels is less than the threshold for the number of parallel computing channels, the working clock frequency is determined to be the third working clock frequency. When the number of logic units occupied is less than the threshold for the number of logic units occupied, and the number of parallel computing channels is less than the threshold for the number of parallel computing channels, the working clock frequency is determined to be the fourth working clock frequency.

[0033] Understandably, the thresholds for the number of logic units and the number of parallel computing channels are reference values ​​set in the field of FPGA hardware architecture design based on the total amount of logic resources of a specific FPGA chip, the resource requirements of typical video processing algorithms, and long-term accumulated engineering experience.

[0034] Understandably, the highest stable operating frequency of a certain FPGA model is 200MHz, the threshold for the number of logic units occupied is set to 60% of the total logic units, and the threshold for the number of parallel computing channels is set to 32 channels. When the logic unit occupancy rate of a processing unit reaches 70% (greater than the threshold) and the number of parallel channels is 40 (greater than the threshold), the operating clock frequency is set to 200MHz (first operating clock frequency); if the logic unit occupancy rate is 50% (less than the threshold) and the number of parallel channels is 40 (greater than the threshold), the operating clock frequency can be set to 160MHz (second operating clock frequency); if the logic unit occupancy rate is 70% (greater than the threshold) and the number of parallel channels is 20 (less than the threshold), the operating clock frequency can be increased to 180MHz (third operating clock frequency); if the logic unit occupancy rate is 50% and the number of parallel channels is 20 (both less than the threshold), the operating clock frequency is set to 120MHz (fourth operating clock frequency). Through this multi-dimensional clock frequency setting strategy, a balance between performance and power consumption can be achieved under different resource configurations.

[0035] Specifically, when the processing latency of any processing unit exceeds the preset latency limit, the process of reallocating logical resources or increasing parallelism includes: Calculate the latency exceedance for each processing unit; Based on the latency exceeding the limit, determine the corresponding exceeding level of the processing unit, and determine the corresponding resource adjustment scheme or parallelism adjustment scheme based on the exceeding level; When the level of exceeding the limit is slightly exceeding the limit, select the first type of resource adjustment scheme or the first type of parallelism adjustment scheme; When the over-limit level is moderate over-limit, select the second type of resource adjustment scheme or the second type of parallelism adjustment scheme; When the level of exceeding the limit is severe, select the third type of resource adjustment scheme or the third type of parallelism adjustment scheme.

[0036] It is understandable that the delay exceeding the limit refers to the specific value by which the actual processing delay of any processing unit exceeds the preset delay limit. The calculation formula is: delay exceeding the limit = actual processing delay - preset delay limit.

[0037] Understandably, the first type of resource adjustment scheme optimizes the combinational logic paths and timing constraints within the logic units, shortens critical path latency, and increases the processing unit's clock frequency, without changing the number of logic units or parallel computing channels. The first type of parallelism adjustment scheme adds 1-2 channels to the existing parallel computing channels, fine-tuning the parallel processing scale, improving data processing speed, and reducing processing latency. The second type of resource adjustment scheme increases the number of logic units, such as from 200 LUTs to 250 LUTs, enhancing logic processing capabilities and increasing unit operation speed. The second type of parallelism adjustment scheme adds 3-5 parallel computing channels, significantly expanding the parallel scale and improving data throughput, suitable for situations with moderate latency exceeding limits. The third type of resource adjustment scheme: significantly increase the number of logic units occupied, or even redesign some logic module architectures to meet higher processing performance requirements; the third type of parallelism adjustment scheme: double or expand the number of parallel computing channels, re-plan the FPGA internal clock allocation and data interaction logic, deal with severe latency exceedance issues, and ensure that the processing unit completes the data processing task within the preset latency limit.

[0038] Specifically, adjusting the encoding parameters of the compressed video to be transmitted based on image visual features to ensure that the encoded compressed video meets a preset visual lossless threshold to generate a compressed video bitstream includes: The image feature information of the first N frames of the compressed video to be transmitted is stored, including average brightness, contrast, texture complexity and motion vector information; The four feature indicators of the current frame image—brightness, contrast, texture complexity, and motion speed—are compared with the feature information of the previous N frames to calculate the feature difference value between the current frame and the historical frames. The trend of visual information change in the current frame is determined based on the feature difference values; Based on the feature difference values ​​and the trend of visual information changes, the quantization step size, DCT transform coefficient scaling ratio and inter-frame coding mode are adjusted so that the adjusted compressed video to be transmitted meets the preset visual lossless threshold.

[0039] Understandably, N is preferably 5-10, which is a reasonable value determined after comprehensively considering the temporal correlation of video sequences and computational complexity.

[0040] Understandably, the feature difference values ​​include brightness difference, contrast difference, texture complexity difference, and motion speed difference. Specifically, the brightness difference is calculated as the absolute difference between the average brightness of the current frame and the average brightness of the previous N frames; the contrast difference is represented by the deviation of the ratio of the current frame's contrast to the average contrast of the previous N frames; the texture complexity difference is measured by the mean square error between the texture feature entropy of the current frame and the texture feature entropy of the previous N frames; and the motion speed difference is determined by the difference between the average motion vector magnitude of the current frame and the average motion vector magnitude of the previous N frames.

[0041] Specifically, when determining the trend of visual information change in the current frame based on the feature difference values, the following are included: The average brightness of the current frame is compared frame by frame with the average brightness of the previous N frames to calculate the brightness change rate, and the brightness change amplitude is determined based on the brightness change rate. The texture complexity of the current frame is compared frame by frame with the texture complexity of the previous N frames to calculate the texture complexity change rate, and the texture change magnitude is determined based on the texture complexity change rate. The motion vector information of the current frame is compared with the motion vector information of the previous N frames frame by frame to calculate the motion vector difference, and the motion change amplitude is determined based on the motion vector difference. Based on the brightness change amplitude, texture change amplitude, and motion change amplitude, determine the visual information change trend of the current frame; The visual information change trends include brightness increase / decrease, texture enhancement / reduction, and motion acceleration / deceleration.

[0042] Understandably, the formula for calculating the brightness change rate is (current frame average brightness - previous frame average brightness) / previous frame average brightness × 100%. When the brightness change rate is greater than 5% for three consecutive frames, it is considered an increasing brightness trend; if the brightness change rate is less than -5% for three consecutive frames, it is considered a decreasing brightness trend. The texture complexity change rate is calculated as (current frame texture feature entropy value - previous frame texture feature entropy value) / previous frame texture feature entropy value × 100%. When this value is greater than 8% for three consecutive frames, it is considered a texture enhancement trend; if it is less than -8% for three consecutive frames, it is considered a texture weakening trend. The motion vector difference is calculated as the difference between the average motion vector magnitude of the current frame and the average motion vector magnitude of the previous frame. When the difference is positive for three consecutive frames and the cumulative difference exceeds 20% of the average motion vector magnitude of the previous N frames, it is considered an accelerating motion trend; if the difference is negative for three consecutive frames and the cumulative absolute value exceeds 20% of the average motion vector magnitude of the previous N frames, it is considered a decelerating motion trend. By comparing frames one by one in this multi-dimensional way and determining trends, we can accurately capture the dynamic changes of visual information in video sequences, providing a reliable basis for the adaptive adjustment of subsequent encoding parameters.

[0043] Specifically, based on the feature difference values ​​and the trend of visual information changes, the quantization step size, DCT transform coefficient scaling ratio, and inter-frame coding mode are adjusted so that the adjusted compressed video to be transmitted meets the preset visual lossless threshold, including: The adjustment range of the quantization step size is determined based on the brightness change range. When the brightness increase is greater than the preset brightness threshold, the quantization step size is reduced by a preset step size ratio; when the brightness decrease is greater than the preset brightness threshold, the quantization step size is increased by a preset step size ratio. The scaling ratio of DCT transform coefficients is adjusted according to the texture change amplitude. When the texture enhancement amplitude exceeds the preset texture threshold, the scaling ratio of high-frequency DCT coefficients is increased to retain more details; when the texture reduction amplitude exceeds the preset texture threshold, the scaling ratio of high-frequency DCT coefficients is decreased to reduce bitstream fluctuations. The inter-frame coding mode is adjusted according to the magnitude of motion change. When the acceleration of motion exceeds a preset motion threshold, the proportion of P-frames or B-frames used is increased; when the deceleration of motion exceeds a preset motion threshold, the proportion of P-frames or B-frames used is decreased. The quantization step size, DCT transform coefficient scaling ratio, and inter-frame coding mode are jointly verified to ensure that the adjusted video compression result is within the preset visual lossless threshold range.

[0044] Understandably, the preset brightness threshold is typically set to ±15% of the average brightness of the previous N frames. When the brightness increase exceeds this threshold, the quantization step size is reduced by 5%-10% to minimize quantization loss and prevent the loss of detail in bright areas. If the brightness decrease exceeds 15%, the quantization step size is increased by 3%-8% to appropriately improve compression efficiency while maintaining visual quality in dark areas. The preset texture threshold is generally set to ±20% of the texture feature entropy of the previous N frames. When the texture enhancement exceeds this threshold, the scaling ratio of high-frequency DCT coefficients is adjusted from the default 0.8 to 0.9-0.95 to ensure complete preservation of texture details. If the texture reduction exceeds 20%, the scaling ratio of high-frequency DCT coefficients is reduced to 0.6-0.7 to control the bitstream size by appropriately discarding redundant high-frequency information. The preset motion threshold is typically set to 30% of the average magnitude of the motion vectors in the first N frames. When the acceleration of motion exceeds this threshold, the proportion of P-frames is increased by 10%-15% or the proportion of B-frames is increased by 5%-10%, utilizing inter-frame prediction to enhance motion compensation. When the deceleration of motion exceeds 30%, the proportion of P-frames or B-frames is reduced by 5%-10%, and the I-frame insertion frequency is increased to improve the coding accuracy of static areas. During joint verification, the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) of the adjusted video frames and the original frames are calculated to ensure that the SSIM is not lower than 0.98 and the PSNR is not lower than 40dB, thereby strictly meeting the preset visual lossless threshold requirements.

[0045] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this invention, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0046] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A lossless visual video transmission system based on FPGA, characterized in that, include: The video acquisition and processing module is used to acquire the video to be processed, and to perform format conversion and frame structure processing on the video to be processed in order to obtain the video to be transmitted. The visual lossless processing module is used to compress the video to be transmitted under a preset delay constraint based on an FPGA hardware architecture and a parallel pipeline structure to obtain a compressed video to be transmitted; it is also configured to adjust the encoding parameters of the compressed video to be transmitted based on the image visual features so that the encoded compressed video to be transmitted meets a preset visual lossless threshold to generate a compressed video bitstream. The data transmission module is used for cache management and link transmission of the compressed video stream; The decoding and display module is used to decode and reconstruct the received compressed video stream and output the video signal. When compressing the video to be transmitted under a preset delay constraint to obtain the compressed video to be transmitted, the process includes: In the FPGA hardware architecture, a parallel pipeline structure with multi-level processing units is constructed, wherein each level of the multi-level processing unit corresponds to a different processing stage in the video compression process. A preset delay limit is determined, and based on the preset delay limit, the operation clock frequency and data throughput bandwidth of each level processing unit are allocated and adjusted by configuring the internal logic resources of the FPGA so that the overall pipeline processing delay does not exceed the preset delay limit. The video to be transmitted is divided into multiple data blocks by frame and input into the parallel pipeline through multiple parallel channels. The multi-level processing unit sequentially completes the transformation processing, quantization processing and encoding processing, and outputs compressed encoded data. The compressed encoded data is integrated to generate the compressed video to be transmitted; When allocating and adjusting the clock frequency and data throughput bandwidth of each processing unit by configuring the internal logic resources of the FPGA, the following is included: Collect the real-time data processing volume and corresponding processing delay information for each processing unit. Based on the preset delay limit, calculate the target processing time corresponding to each processing unit and determine the target data throughput capacity corresponding to the target processing time; Based on the target processing time and target data throughput, the number of logic units occupied, the number of parallel computing channels, and the working clock frequency of each processing unit are adjusted. For each adjusted processing unit, latency detection is performed. When the processing latency of any processing unit exceeds the preset latency limit, logic resources are reallocated or parallelism is increased. Latency detection and resource adjustment are repeated until the overall processing latency of the parallel pipeline does not exceed the preset latency limit.

2. The FPGA-based lossless video transmission system according to claim 1, characterized in that, When adjusting the number of logic units, the number of parallel computing channels, and the operating clock frequency of each processing unit based on the target processing time and target data throughput, the following adjustments are made: Establish a mapping relationship between the number of logical units occupied and the target data throughput capacity; Collect the data processing complexity corresponding to each level of processing unit, and preliminarily determine the number of logical units occupied by each level of processing unit based on the mapping relationship and data processing complexity. Based on the target data throughput capacity and the number of logical units occupied, the number of parallel computing channels of each processing unit is increased according to the preset parallelism enhancement rules. Based on the available clock resources inside the FPGA and the configuration of the logic units and parallel channels, the operating clock frequency corresponding to each processing unit is set, and the verification steps confirm that each processing unit can operate stably at the corresponding operating clock frequency. When the combination of logic units, parallel channels, and clock frequencies cannot meet the target processing time or target data throughput, the number of logic units, the number of parallel computing channels, or the clock frequency are iteratively adjusted until the processing unit can achieve the target data throughput within the target processing time.

3. The FPGA-based lossless video transmission system according to claim 2, characterized in that, When initially determining the number of logical units required for each processing level based on the mapping relationship and data processing complexity, the following steps are included: Based on the mapping relationship, determine the initial number of logic units occupied by each processing unit. Obtain the standard complexity corresponding to the data processing complexity, and calculate the ratio of the data processing complexity to the standard complexity, denoted as the complexity ratio. The optimization coefficient for the initial number of logic units is determined based on the complexity ratio, and the product of the optimization coefficient and the initial number of logic units is used as the number of logic units occupied.

4. The FPGA-based visual lossless video transmission system according to claim 3, characterized in that, When setting the operating clock frequency for each processing unit, taking into account the available clock resources within the FPGA and the configuration of the logic units and parallel channels, the following steps are taken: The operating clock frequency for each processing unit is determined based on the number of logic units occupied and the number of parallel computing channels corresponding to each processing unit. The number of logic units occupied is compared with the threshold number of logic units occupied, and the number of parallel computing channels is compared with the threshold number of parallel computing channels. The working clock frequency is determined based on the comparison results. When the number of logic units occupied is greater than or equal to the threshold number of logic units occupied, and the number of parallel computing channels is greater than or equal to the threshold number of parallel computing channels, the working clock frequency is determined to be the first working clock frequency. When the number of logic units occupied is less than the threshold for the number of logic units occupied, and the number of parallel computing channels is greater than or equal to the threshold for the number of parallel computing channels, the working clock frequency is determined to be the second working clock frequency. When the number of logic units occupied is greater than or equal to the threshold for the number of logic units occupied, and the number of parallel computing channels is less than the threshold for the number of parallel computing channels, the working clock frequency is determined to be the third working clock frequency. When the number of logic units occupied is less than the threshold for the number of logic units occupied, and the number of parallel computing channels is less than the threshold for the number of parallel computing channels, the working clock frequency is determined to be the fourth working clock frequency.

5. The FPGA-based lossless video transmission system according to claim 4, characterized in that, When the processing latency of any processing unit exceeds the preset latency limit, the process of reallocating logical resources or increasing parallelism includes: Calculate the latency exceedance for each processing unit; Based on the latency exceeding the limit, determine the corresponding exceeding level of the processing unit, and determine the corresponding resource adjustment scheme or parallelism adjustment scheme based on the exceeding level; When the level of exceeding the limit is slightly exceeding the limit, select the first type of resource adjustment scheme or the first type of parallelism adjustment scheme; When the over-limit level is moderate over-limit, select the second type of resource adjustment scheme or the second type of parallelism adjustment scheme; When the level of exceeding the limit is severe, select the third type of resource adjustment scheme or the third type of parallelism adjustment scheme.

6. The FPGA-based lossless video transmission system according to claim 5, characterized in that, When adjusting the encoding parameters of the compressed video to be transmitted based on image visual features to ensure that the encoded compressed video meets a preset visual lossless threshold in order to generate a compressed video bitstream, the process includes: The image feature information of the first N frames of the compressed video to be transmitted is stored, including average brightness, contrast, texture complexity and motion vector information; The four feature indicators of the current frame—mean brightness, contrast, texture complexity, and motion speed—are compared with the feature information of the previous N frames to calculate the feature difference value between the current frame and the historical frames. The trend of visual information change in the current frame is determined based on the feature difference values; Based on the feature difference values ​​and the trend of visual information changes, the quantization step size, DCT transform coefficient scaling ratio and inter-frame coding mode are adjusted so that the adjusted compressed video to be transmitted meets the preset visual lossless threshold.

Images