Image compression method and system based on partially reconfigurable and evolvable CNN

Abstract

The invention provides an image compression method and system based on a partially reconfigurable and evolutionary CNN. The method comprises the following steps: step 1, acquiring an image data stream; step 2, based on the complexity of the image, carrying out adaptive blocking on the image data stream to obtain image blocks with variable sizes; step 3, inputting the image block into a CNN intra-frame predictor deployed in a reconfigurable partition of the FPGA to generate a prediction block; the CNN intra-frame predictor is a lightweight neural network model subjected to quantization compression; step 4, generating and outputting a compressed code stream based on the image block and the prediction block; and step 5, in response to a trigger condition, in a display frame blanking period, loading the updated partial bit stream to a reconfigurable partition of the FPGA through a partial reconfiguration technology so as to evolve the CNN intra-frame predictor online. According to the method, the block size can be automatically adjusted according to the image content, ARGB image compression of online evolution of a prediction model is supported, and simultaneous rising of compression efficiency, visual quality and scene adaptability is realized.

Description

Technical Field

[0001] This invention belongs to the field of image compression technology, and particularly relates to an image compression method and system based on partially reconfigurable and evolvable CNNs. Background Technology

[0002] In applications such as industrial vision, UAV remote sensing, and augmented reality, ARGB images are widely used for layer blending and foreground extraction due to their alpha channels. However, the raw bandwidth of a 4K@60fps ARGB stream exceeds 12Gbps, posing a significant challenge to storage and transmission, necessitating the use of efficient compression techniques. Existing FPGA image compression solutions are mainly divided into three categories: (1) Fixed-size block division + transform coding (such as JPEG2000, JPEG): The block size is uniform (usually 8×8 or 16×16), which cannot be adaptively adjusted according to the image content. The compression efficiency is low in flat areas, and block effect distortion is easy to occur in edge areas.

[0003] (2) Traditional intra-frame prediction (such as H.264 / H.265 Intra): The prediction direction is limited (up to 34 angles), and the prediction accuracy for complex textures and irregular edges is insufficient, making it difficult to meet the lossless / near lossless compression requirements of ARGB images.

[0004] (3) Static CNN compression: In recent years, some studies have deployed CNNs on FPGAs for end-to-end compression, achieving rate-distortion performance superior to traditional methods. However, such solutions embed the CNN model into the FPGA once. Once the imaging environment (lighting, scene type, subject) changes, the model performance drops sharply, requiring the bitstream to be resynthesized and the device to be restarted, making it impossible to adapt online.

[0005] More importantly, the application of FPGA Dynamic Partial Reconfiguration (DPR) technology in the field of image compression has been publicly documented. In their 2012 patent, "Run-Time Reconfigurable Embedded Secure Real-Time Image Compression System" (application number CN201210240153.0), Jia Zhiping et al. of Shandong University disclosed a technical solution where "FPGA acts as a JPEG image compression accelerator, dynamically changing the quantization module and Huffman coding module through runtime reconfiguration technology." While this solution achieves reconfiguration of some functional modules, it suffers from the following fundamental defects: it only supports reconfiguration of traditional coding modules such as DCT quantization and Huffman, failing to address the problem of insufficient prediction accuracy in complex texture scenes; the reconfiguration decision is unrelated to the image content and does not form a collaborative optimization with content adaptive segmentation; the reconfiguration timing depends on external instructions and is not synchronized with the display timing, potentially leading to frame drops or interruptions in real-time video streaming scenarios.

[0006] In summary, existing technologies fail to address two core issues: dynamically matching segmentation granularity with content complexity and online evolution of prediction models. This results in bottlenecks in embedded image compression systems across compression efficiency, visual quality, and scene adaptability. This invention addresses these technological gaps by proposing, for the first time, a technical solution that deeply couples content-aware dynamic segmentation with partially reconfigured CNN prediction synchronized with display timing. Summary of the Invention

[0007] The purpose of this invention is to provide an image compression method and system based on partially reconfigurable and evolvable CNNs. This method enables ARGB image compression to automatically adjust block sizes according to image content and supports online evolution of the prediction model, achieving a simultaneous leap in compression efficiency, visual quality, and scene adaptability. The technical solution adopted is as follows: The system uses adaptive block partitioning based on gradient-variance hybrid features as the core of front-end perception, an ultra-low parameter CNN prediction model trained with knowledge distillation and mixed precision quantization as the compression engine, and dynamic reconfiguration of loading part of the bitstream through the ICAP interface with zero interruption during the frame hidden period as the enabling means to realize online model evolution. It also integrates a series of innovative mechanisms at the system level, such as FIFO deep automatic optimization, transfer learning triggered updates, and scene adaptive switching of multiple version model libraries.

[0008] For the first time, joint online optimization of block granularity and prediction model parameters has been achieved on an embedded FPGA platform, fundamentally breaking through the technical bottleneck of traditional solutions that cannot balance compression efficiency, visual quality and scene adaptability.

[0009] An image compression method based on partially reconfigurable and evolvable CNNs includes the following: Step 1: Obtain the image data stream; Step 2: Based on the complexity of the image, adaptively divide the image data stream into blocks to obtain image blocks of variable size; Step 3: Input the image block into the CNN intra-frame predictor deployed in the reconfigurable partition of the FPGA to generate a prediction block; The CNN intra-frame predictor is a quantized and compressed lightweight neural network model. Step 4: Based on the image block and the prediction block, generate and output a compressed bitstream; Step 5: In response to the triggering condition, during the blanking period of the display frame, the updated partial bitstream is loaded into the reconfigurable partition of the FPGA through partial reconfiguration technology to evolve the CNN intra-frame predictor online.

[0010] Preferably, the step of generating and outputting the compressed bitstream includes: Calculate the pixel-by-pixel difference between the image block and the prediction block to obtain the residual data matrix; The residual data matrix is ​​quantized using an adaptive quantization step size. A large quantization step size is used for flat regions to improve compression efficiency, while a small quantization step size is used for edge / detail regions to preserve texture information. The quantization step size is determined by the complexity index of the image patch. Dynamic matching; The quantized residuals are subjected to adaptive entropy encoding of binary arithmetic encoding (CABAC) based on a context model: first, the probability distribution characteristics of the residual data are statistically analyzed and the encoding context model is dynamically updated; then, the quantized residuals are entropy encoded according to the model to generate the compressed bitstream.

[0011] Preferably, adaptive block division is performed, specifically including: Calculate the gradient and variance features of the image patch; By fusing the gradient features and variance features, a complexity index is generated; The complexity index is compared with a preset threshold, and the optimal block size is selected for the current image block from a variety of preset block sizes based on the comparison result.

[0012] Preferably, the formula for calculating the complexity index is: + ； - Normalized gradient mean; -Normalized pixel variance; - Weights.

[0013] Preferably, the CNN intra-frame predictor is obtained through the following steps: The teacher network is compressed into the student network using knowledge distillation technology; The student network is subjected to quantization perception training, and its weights and activation function outputs are quantized respectively; The quantized network model is converted into hardware description language code adapted for FPGA and deployed in the reconfigurable partition.

[0014] Preferably, the triggering condition includes any one of the following: Received an external model update command; The system detected a pre-defined change in the image scene. The scene labels output by the scene classifier inside the system are inconsistent with the scene adapted by the currently activated model, and the confidence level exceeds the threshold.

[0015] Preferably, the partial reconfiguration technology is synchronized with the display timing, specifically including: The monitoring displays a synchronization signal; When the triggering condition occurs, wait for the start time of the blanking period of the next display frame; During the blanking period of the display frame, the transmission of new data to the reconfigurable partition of the FPGA is paused, and the updated portion of the bit stream is loaded through the internal configuration access port. Once loading is complete, the data stream is restored, and the reconfiguration process is finished.

[0016] An image compression system based on partially reconfigurable and evolvable CNNs, integrated into an FPGA chip, includes: The dynamic block partitioning module receives the image data stream and adaptively partitions it based on the local complexity of the image, outputting image blocks of variable size. The CNN intra-frame prediction module, deployed within a reconfigurable partition of the FPGA, is used to perform intra-frame prediction on the image blocks and generate prediction blocks; the CNN intra-frame prediction module is hardware-implemented using a quantized lightweight neural network model. The residual and entropy coding module is used to generate a compressed bitstream based on the image block and the prediction block; A dynamic reconfiguration control module, synchronized with the display timing, is used to load a portion of the bitstream of the updated CNN intra-predictor into the reconfigurable partition during the blanking period of the display frame in response to a trigger condition, so as to update the CNN intra-prediction module.

[0017] Specifically: The dynamic block partitioning module, deployed in the non-reconfigurable partition, includes a gradient calculation unit, a variance calculation unit, and a block size decision unit. The input ends of the gradient calculation unit and the variance calculation unit are both connected to the image data stream receiving interface, and the output ends of the gradient calculation unit and the variance calculation unit are both connected to the block size decision unit. The output end of the block size decision unit is connected to the CNN intra-frame prediction module and the residual and entropy coding module. The CNN intra-frame prediction module is deployed within the reconfigurable partition of the FPGA. Its input is connected to the output of the dynamic block partitioning module, and its output is simultaneously connected to the residual and entropy coding module and the dynamic reconfiguration control module. It is used to perform intra-frame prediction on the image blocks and generate prediction blocks. The CNN intra-frame prediction module is implemented in hardware using a quantized lightweight neural network model. The residual and entropy coding module is deployed in the non-reconfigurable partition and includes a residual calculation unit, an adaptive quantization unit, and an adaptive entropy coding unit. The two inputs of the residual calculation unit are connected to the dynamic block partitioning module and the CNN intra-frame prediction module, respectively, and the output is connected to the adaptive quantization unit. The output of the adaptive quantization unit is connected to the adaptive entropy coding unit, and the output of the adaptive entropy coding unit is a compressed bitstream output interface. The system also includes a dynamic reconfiguration control module, which is connected across the reconfigurable and non-reconfigurable partitions. Its input is connected to an external instruction receiving interface and a scene classification module, and its output is connected to the configuration port of the CNN intra-frame prediction module. It also communicates bidirectionally with the FPGA's display timing synchronization module. In response to triggering conditions, it loads an updated portion of the bitstream into the reconfigurable partition during the display frame blanking period to update the CNN intra-frame prediction module.

[0018] Preferably, the dynamic reconfiguration control module is also connected to a DDR external memory, which pre-stores a portion of the bitstream of the CNN intra-frame predictor adapted to different application scenarios. The dynamic reconfiguration control module can select and load the corresponding model in real time based on the scene classification results, enabling rapid scene adaptive switching.

[0019] Preferably, the dynamic block partitioning module includes a gradient calculation unit, a variance calculation unit, and a block size decision unit, which are used to calculate the local complexity and decide the block size in real time through integer operations.

[0020] Compared with the prior art, the advantages of the present invention are: First, an adaptive block partitioning algorithm based on gradient variance mixture features is designed to dynamically select ARGB block sizes of 8×8 to 32×32 according to the local complexity of the image.

[0021] Secondly, a lightweight CNN intra-frame prediction network with fewer than 50K parameters is constructed using knowledge distillation and mixed precision quantization perceptual training. The weights are 4-bit wide and the activations are 8-bit wide. The data flow hardware description is generated through the hls4ml toolchain and deployed on a reconfigurable FPGA partition.

[0022] Third, a dynamic partial reconfiguration process with display timing synchronization was constructed. During the frame blanking period, the updated CNN model bitstream was loaded through the ICAP interface. The calculation showed that it took 1.69ms, which was completely unobtrusive to the 60fps video stream.

[0023] Fourth, it pioneered a FIFO deep automatic optimization method, reducing BRAM usage by 40% while ensuring no pipeline stalls. Fifth, it introduced a transfer learning-triggered reconfiguration mechanism, supporting automatic model updates triggered by host computer commands or adaptive algorithms based on scene change detection.

[0024] Calculations show that this method improves the compression rate by 35% without loss of subjective visual quality, with an end-to-end latency of <3ms, and dynamic reconfiguration is imperceptible to the user. It can run stably in high frame rate ARGB real-time compression and dynamic imaging environments.

[0025] This invention is the first to achieve deep coupling between block adaptation and online evolution of the prediction model on an embedded FPGA platform, providing a new, highly adaptable, and low-overhead technical path for intelligent image compression.

[0026] This invention achieves, for the first time, collaborative optimization of block adaptation and online evolution of the prediction model on an embedded FPGA platform through a series of supporting innovations, including dynamic block partitioning, deep coupling of evolvable CNN prediction and temporal synchronous reconfiguration, FIFO deep automatic optimization, transfer learning triggered updates, and scene adaptive switching of multiple version model libraries.

[0027] The calculation results show that the proposed method improves the compression rate by 35% under the premise of lossless subjective vision, with an end-to-end latency of <3ms. Dynamic reconfiguration is completely imperceptible to 60fps video streams. The compression performance and scene adaptability are significantly better than the existing fixed block + static CNN scheme. It provides a new technical path with high adaptability, low power consumption and zero interruption for real-time compression of high frame rate ARGB images. Attached Figure Description

[0028] Figure 1 A flowchart of an image compression method based on partially reconfigurable and evolvable CNNs; Figure 2 This is a structural diagram of an image compression system based on partially reconfigurable and evolvable CNNs. Detailed Implementation

[0029] The image compression method and system based on partially reconfigurable and evolvable CNN of the present invention will be described in more detail below with reference to the schematic diagrams, which illustrate preferred embodiments of the invention. It should be understood that those skilled in the art can modify the invention described herein while still achieving the advantageous effects of the invention. Therefore, the following description should be understood as being of general knowledge to those skilled in the art and is not intended to limit the invention.

[0030] Taking a 1920×1080@60fps ARGB stream as an example, the FPGA chip model is Xilinx ZU9EG.

[0031] like Figure 1 As shown, an image compression method based on partially reconfigurable and evolvable CNNs includes the following: Step 1: Obtain the ARGB image pixel data stream.

[0032] Step 2: Based on the complexity of the image, adaptively divide the image data stream into blocks to obtain image blocks of variable size. Specifically, this includes: For an image block of W×H pixels: W is the width and H is the height; Calculate the Sobel gradient magnitude of each pixel within the image patch to obtain the gradient map; Calculate the normalized gradient mean of the gradient plot. Normalized pixel variance of pixel values ​​within an image patch .

[0033] Computational complexity metrics : + ； - Normalized gradient mean; -Normalized pixel variance; - Weights.

[0034] This indicator only requires integer addition, subtraction, and comparison operations, and can complete the decision within a single clock cycle.

[0035] Will With preset threshold , Compare: like < That is, the region (image patch) is a flat area, and a large 32×32 patch is selected to pursue a high compression ratio; like < < That is, the image block is a general texture, and a 16×16 medium block is selected to achieve a balance between compression ratio and distortion; ≥ This means that the image patch is an edge / detail area, and an 8×8 patch is selected to preserve visual quality to the maximum extent.

[0036] Specifically: Using 8×8 as the basic scanning unit, the row buffer is used to accumulate 8 rows of pixels, and the gradient and variance of each 8×8 block are calculated at the same time.

[0037] Gradient calculation uses two 3×3 Sobel operators, implemented by shifting and adding, without consuming DSP.

[0038] Variance is calculated using the summation and square summation formulas, all using 18-bit fixed-point numbers.

[0039] Complexity metrics Implemented via a lookup table Weighted, threshold Stored in the control register, it can be dynamically adjusted by software.

[0040] This algorithm requires no multiplication, division, or floating-point operations. It is implemented in a two-stage pipeline on an FPGA without introducing additional pipeline pauses. Compared with the fixed block scheme, it reduces the number of compressed bits in flat areas by 37% and improves the PSNR in edge areas by 2.1dB.

[0041] If it is a 32×32 block, then the four adjacent 8×8 blocks are merged, and the subsequent CNN intra-frame predictor processes the 32×32 region at a time. If it is an 8×8 block, it is fed into the CNN intra-frame predictor independently.

[0042] Calculations show that dynamic segmentation reduces the number of compressed bits in flat areas by 37% and improves PSNR in edge areas by 2.1dB compared to fixed 16×16 segmentation.

[0043] In summary, at the dynamic block level, this embodiment proposes a hardware-friendly gradient-variance hybrid complexity metric.

[0044] Step 3: Input the image block into the CNN intra-frame predictor deployed in the reconfigurable partition of the FPGA to generate the prediction block.

[0045] The CNN intra-frame predictor is a lightweight neural network model that has been quantized and compressed.

[0046] The CNN intra-frame predictor is obtained through the following steps: First, the high-precision teacher network is compressed into a student network with fewer than 50K parameters through knowledge distillation. Subsequently, the Brevitas framework was used for quantization-aware training, with the weight bit width compressed to 4-bit and the activation bit width to 8-bit, and exported in the QONNX open format. The hls4ml toolchain is used to convert the quantization model into a dataflow hardware description, which is then deployed on the reconfigurable partition RP0 of the FPGA.

[0047] In the CNN intra-frame predictor, all convolutional layers use a parallel factor-configurable systolic array, the inter-layer FIFO depth is automatically explored and optimized, the weights are fixed in block random access memory (BRAM), and the activation values ​​are calculated online.

[0048] FIFO: First In First Out (First Input First Output).

[0049] That is, the CNN intra-frame predictor is generated by hls4ml, and its convolutional layer parallelism factor and inter-layer FIFO depth are optimized by an automated exploration tool; The automated exploration tool is based on Vitis HLS scheduling analysis, iteratively adjusting the FIFO depth between each layer to the minimum value, while ensuring no pipeline interruption.

[0050] To address the common challenge of determining the inter-layer FIFO depth in dataflow architectures, this invention pioneers an automatic FIFO depth optimization method: based on Vitis HLS scheduling analysis, iteratively adjusts the inter-layer FIFO depth to the minimum value that can maintain pipelined interruptions, and actively reduces the single-layer computation latency during the optimization process to reduce output buffering requirements.

[0051] Based on the above optimizations, BRAM usage can be reduced by more than 40% compared to the default configuration, thereby freeing up valuable on-chip storage resources for the deployment of larger-scale models; at the same time, single-frame inference latency can be as low as 187μs at typical operating frequencies.

[0052] The FIFO depth automatic optimization module is developed based on Vitis HLS 2024.2. First, synthesize the project at the default depth (input tensor size) to obtain the scheduling information of the FIFOs between each layer; then iteratively reduce the depth of each FIFO until the simulation report shows no pipeline stalls.

[0053] After optimization, the total BRAM usage of the model was reduced from 200 blocks to 120 blocks (a reduction of 40%), while the total FIFO depth was compressed from the default 4.2MB to 1.1MB. The student network model with 50K parameters after knowledge distillation was successfully deployed on an FPGA chip of model ZU9EG.

[0054] In summary, at the prediction model level, this invention uses a lightweight CNN with an encoder-decoder structure as the core of intra-frame prediction.

[0055] Step 4: Generate and output a compressed bitstream based on image blocks and prediction blocks.

[0056] Specifically, this includes: calculating the residual between the image block and the prediction block; quantizing the residual; and performing adaptive entropy coding on the quantized residual to generate a compressed bitstream.

[0057] Adaptive entropy coding is a context-based binary arithmetic coding system called CABAC, whose probability model is updated online adaptively based on residual statistical properties.

[0058] Step 5: In response to the triggering condition, during the blanking period of the display frame, the updated partial bitstream is loaded into the reconfigurable partition of the FPGA through partial reconfiguration technology to evolve the CNN intra-frame predictor online.

[0059] Some reconfiguration techniques are synchronized with display timing, specifically including: S1, Triggering Phase: The system control state machine receives the reconfiguration trigger signal (external instruction or internal adaptive algorithm), pauses sending new data to the partition to be updated, and waits for the current processing to complete; S2, Timing Alignment: Align with the display line / frame synchronization signal to ensure that the reconfiguration operation is started strictly within the frame blanking interval; S3, Bit Stream Loading: The bit stream stored in the external memory DDR is written byte by byte to the ICAP interface (internal configuration access port) through the configuration channel of the AXI bus and loaded into the reconfigurable partition RP0. S4. Verification and Activation: After loading is complete, read the partition status register, verify the integrity of the bit stream, reset the pipeline register in the partition, and restore the data stream input. S5. Confirmation of completion: Send a reconfiguration completion interrupt to the system processor and update the system status flag.

[0060] The entire process lasted 28ms ± 2ms, with no noticeable interruption to the front-end image acquisition and compression pipeline.

[0061] In summary, at the level of online model evolution, this invention constructs a dynamic partial reconfiguration process that is synchronized with the hardwired display timing.

[0062] The system continuously monitors external update commands from the external command receiving interface and internal scene change detection signals from the scene classification module through a control state machine integrated within the dynamic reconfiguration control module. Once an update request is triggered, the control state machine first interacts with the FPGA's display timing synchronization module. At the start of the next frame blanking (VSYNCactive), it sends a data pause command to the dynamic block partitioning module to pause the push of new data to the CNN intra-frame prediction module within the reconfigurable partition. Simultaneously, the control state machine establishes communication with the pipeline register of the CNN intra-frame prediction module and uses register control signals to empty the unprocessed data in the pipeline within 20 clock cycles, ensuring that there is no residual data in the pipeline before reconfiguration. Subsequently, the new CNN model bitstream is read from DDR via the AXI DMA controller and written byte by byte to the ICAP interface in 32-bit width, and loaded into the target partition RP0. After loading is complete, the partition status register is read to verify the integrity of the bit stream, the pipeline register within the partition is reset, the data stream input is resumed after a 3-clock-cycle delay, and a reconfiguration completion interrupt is sent to the processor.

[0063] Based on theoretical calculations, loading a 384KB bitstream on a Xilinx ZU9EG device took 28.1ms, which was completely within the 60Hz frame blanking window (>30ms). The front-end image acquisition and compression pipeline had no frame drops or stutters throughout the entire process, achieving zero-interruption upgrade.

[0064] The reconfiguration controller consists of three parts: a bitstream prefetcher, a CAP wrapper, and partition isolation logic.

[0065] Bitstream prefetcher: Reads the bitstream from the DDR prefetch section to the local FIFO (depth 512×32) via the AXI_HP interface; CAP wrapper: Instantiates the Xilinx ICAPE3 primitive, and uses a state machine to control signals such as CE (chip select enable signal), WRITE (write enable signal), and ADDR, and writes in 32-bit increments per cycle; Partition isolation logic: During reconfiguration, the input data multiplexer of RP0 is switched to the constant "0" and the output handshake signal is kept invalid.

[0066] The reconfiguration operation is initiated only after a rising edge of VSYNC is detected, and the timer ensures that it is completed before the next VSYNC.

[0067] Based on calculations, at 60fps, the reconfiguration time is 28.1ms, which falls entirely within the frame blanking window. Therefore, theoretically, it will not cause frame drops or stuttering in the front-end acquisition and compression pipeline.

[0068] To further reduce model update costs and enable the system to adapt to different scenarios, this invention integrates a model update mechanism triggered by transfer learning with a scenario-adaptive switching mechanism for multiple version model libraries.

[0069] When the system detects a significant change in the imaging environment (such as a sudden change in lighting, switching between indoor and outdoor environments, or a change in the target object) or receives an update command from the host computer, it automatically acquires 100 frames of images of the current scene and uploads them to the edge server. The server uses transfer learning to freeze the bottom feature extraction layer of a pre-trained CNN model (neural network model) and fine-tune only the top prediction layer. Within 2 seconds, it generates a partial bitstream that matches the reconfigurable partition resource constraints of the current FPGA and sends it to the device DDR. The image compression hardware device based on Xilinx ZU9EG FPGA automatically triggers dynamic reconfiguration during the next frame blanking period, and the new model takes effect after 28ms, completing the online evolution of the model; The prediction accuracy of the CNN intra-frame predictor has been improved from 82% to 94%, without any equipment downtime or hardware replacement.

[0070] Meanwhile, the system supports pre-storing CNN bitstreams corresponding to multiple typical scenes (natural, screen, low light, high dynamic range) in DDR. The front-end scene classifier makes real-time decisions on the current scene type and loads the corresponding model in milliseconds through the dynamic reconfiguration control module, achieving seamless adaptive switching between multiple scenes.

[0071] The device was initially deployed in outdoor natural scenes, and the pre-trained model was optimized for natural images.

[0072] When the device is moved indoors for screen content compression, the user sends an "update UI model" command through the host computer.

[0073] The system acquires 100 frames of images of the current scene and uploads them to the edge server. The server freezes the bottom three convolutional layers based on transfer learning, fine-tunes the top two convolutional layers, and generates a partial bitstream in 1.8 seconds and downloads it. The device automatically triggers reconfiguration during the next frame blanking period, and the new model takes effect 28ms later, improving prediction accuracy from 82% to 94%. The entire process is completed with zero device downtime and zero interruption.

[0074] In UAV remote sensing applications, the system needs to handle various scenarios such as "farmland", "forest", "urban area" and "water area". Lightweight CNN models are pre-trained for each scenario, and after quantization and FIFO optimization, partial bitstreams of 384KB to 410KB are generated and stored in the model library area of ​​DDR.

[0075] The front-end scene classifier uses MobileNetV2 0.35, which is pruned and quantized to 8-bit and then deployed on the PS (the processing system of the FPGA, which is the opposite of the programmable logic of the PL and integrates resources such as processor and memory). The classification time per frame is <5ms.

[0076] As the drone flies over different areas, the scene classifier outputs the scene label with the highest confidence in real time. If it is inconsistent with the currently activated model and the confidence is >0.9, the system automatically triggers dynamic reconfiguration and loads the bitstream of the corresponding scene during the blanking period of the next frame.

[0077] Theoretically, the scene switching latency is 1 frame (16.7ms) + 28ms reconfiguration ≈ 44.7ms, which is completely imperceptible to the user, and the compression ratio remains optimal in various scenarios.

[0078] like Figure 2 As shown, an image compression system based on partially reconfigurable and evolvable CNNs, integrated into an FPGA chip, includes: The dynamic block partitioning module receives the image data stream and adaptively partitions it based on the local complexity of the image, outputting image blocks of variable size. The CNN intra-prediction module, deployed within a reconfigurable partition of the FPGA, is used to perform intra-prediction on the image blocks and generate prediction blocks; the CNN intra-prediction module is a hardware implementation of a quantized lightweight neural network model. The residual and entropy coding module is used to generate a compressed bitstream based on the image block and the prediction block; A dynamic reconfiguration control module, synchronized with the display timing, is used to load a portion of the bitstream of the updated CNN intra-predictor into the reconfigurable partition during the blanking period of the display frame in response to a trigger condition, so as to update the CNN intra-prediction module.

[0079] In summary, this invention organically integrates six core technologies: content-aware dynamic segmentation, lightweight CNN intra-frame prediction, FIFO deep automatic optimization, display timing synchronization partial reconfiguration, transfer learning-triggered model updates, and scene adaptive switching across multiple model library versions. For the first time, it achieves deep coupling between adaptive segmentation and online evolution of the prediction model on an embedded FPGA platform. Experimental results show that this method comprehensively surpasses existing fixed-segmentation + static CNN schemes in key indicators such as compression efficiency, visual quality, scene adaptability, and reconfiguration overhead, providing a novel, highly adaptable, low-power, and zero-interruption technical path for real-time compression of high-frame-rate ARGB images.

[0080] The architecture and method proposed in this invention can also be extended to a wider range of fields such as YUV / RGB image compression, intra-frame prediction in video coding, and deep learning inference acceleration, and have significant universality and industrial application value.

[0081] The above are merely preferred embodiments of the present invention and do not constitute any limitation on the present invention. Any equivalent substitutions or modifications made by those skilled in the art to the technical solutions and content disclosed in the present invention without departing from the scope of the present invention shall be deemed to have remained within the protection scope of the present invention.

Claims

1. An image compression method based on partially reconfigurable and evolvable CNNs, characterized in that, Including the following: Step 1: Obtain the image data stream; Step 2: Based on the complexity of the image, adaptively divide the image data stream into blocks to obtain image blocks of variable size; Step 3: Input the image block into the CNN intra-frame predictor deployed in the reconfigurable partition of the FPGA to generate a prediction block; The CNN intra-frame predictor is a quantized and compressed lightweight neural network model. Step 4: Based on the image block and the prediction block, generate and output a compressed bitstream; Step 5: In response to the triggering condition, during the blanking period of the display frame, the updated partial bitstream is loaded into the reconfigurable partition of the FPGA through partial reconfiguration technology to evolve the CNN intra-frame predictor online.

2. The image compression method based on partially reconfigurable and evolvable CNN according to claim 1, characterized in that, The steps for generating and outputting a compressed bitstream include: Calculate the pixel-by-pixel difference between the image block and the prediction block to obtain the residual data matrix; The residual data matrix is ​​quantized using an adaptive quantization step size, which is determined by the complexity index of the image patch. Dynamic matching; The quantized residuals are subjected to adaptive entropy encoding using binary arithmetic encoding (CABAC) based on a context model.

3. The image compression method based on partially reconfigurable and evolvable CNN according to claim 1, characterized in that, Adaptive block partitioning includes: Calculate the gradient and variance features of the image patch; By fusing the gradient features and variance features, a complexity index is generated; The complexity index is compared with a preset threshold, and the optimal block size is selected for the current image block from a variety of preset block sizes based on the comparison result.

4. The image compression method based on partially reconfigurable and evolvable CNN according to claim 3, characterized in that, The formula for calculating the complexity index is as follows: + ； - Normalized gradient mean; -Normalized pixel variance; - Weights.

5. The image compression method based on partially reconfigurable and evolvable CNN according to claim 3, characterized in that, The CNN intra-frame predictor is obtained through the following steps: The teacher network is compressed into the student network using knowledge distillation technology; The student network is subjected to quantization perception training, and its weights and activation function outputs are quantized respectively; The quantized network model is converted into hardware description language code adapted for FPGA and deployed in the reconfigurable partition.

6. The image compression method based on partially reconfigurable and evolvable CNN according to claim 1, characterized in that, The triggering condition includes any one of the following: Received an external model update command; The system detected a pre-defined change in the image scene. The scene labels output by the scene classifier inside the system are inconsistent with the scene adapted by the currently activated model, and the confidence level exceeds the threshold.

7. The image compression method based on partially reconfigurable and evolvable CNN according to claim 1, characterized in that, The aforementioned partial reconfiguration technology is synchronized with the display timing, and specifically includes: The monitoring displays a synchronization signal; When the triggering condition occurs, wait for the start time of the blanking period of the next display frame; During the blanking period of the display frame, the transmission of new data to the reconfigurable partition of the FPGA is paused, and the updated portion of the bit stream is loaded through the internal configuration access port. Once loading is complete, the data stream is restored, and the reconfiguration process is finished.

8. An image compression system based on partially reconfigurable and evolvable CNNs, characterized in that, Integrated into the FPGA chip, including: The dynamic block partitioning module, deployed in the non-reconfigurable partition, includes a gradient calculation unit, a variance calculation unit, and a block size decision unit. The input ends of the gradient calculation unit and the variance calculation unit are both connected to the image data stream receiving interface, and the output ends of the gradient calculation unit and the variance calculation unit are both connected to the block size decision unit. The output end of the block size decision unit is connected to the CNN intra-frame prediction module and the residual and entropy coding module. The CNN intra-frame prediction module is deployed within the reconfigurable partition of the FPGA. Its input is connected to the output of the dynamic block partitioning module, and its output is simultaneously connected to the residual and entropy coding module and the dynamic reconfiguration control module. It is used to perform intra-frame prediction on the image blocks and generate prediction blocks. The CNN intra-frame prediction module is implemented in hardware using a quantized lightweight neural network model. The residual and entropy coding module is deployed in the non-reconfigurable partition and includes a residual calculation unit, an adaptive quantization unit and an adaptive entropy coding unit. The two inputs of the residual calculation unit are respectively connected to the dynamic block partitioning module and the CNN intra-frame prediction module, and the output is connected to the adaptive quantization unit. The output of the adaptive quantization unit is connected to the adaptive entropy coding unit, and the output of the adaptive entropy coding unit is a compressed bitstream output interface. The system also includes a dynamic reconfiguration control module, which is connected across the reconfigurable and non-reconfigurable partitions. Its input is connected to an external instruction receiving interface and a scene classification module, and its output is connected to the configuration port of the CNN intra-frame prediction module. It also communicates bidirectionally with the FPGA's display timing synchronization module. In response to triggering conditions, it loads an updated portion of the bitstream into the reconfigurable partition during the display frame blanking period to update the CNN intra-frame prediction module.

9. The image compression system based on partially reconfigurable and evolvable CNN according to claim 8, characterized in that, The dynamic reconfiguration control module is also connected to a DDR external memory, which pre-stores a portion of the bitstream of the CNN intra-frame predictor adapted to different application scenarios. The dynamic reconfiguration control module can select and load the corresponding model in real time based on the scene classification results, enabling rapid scene adaptive switching.

10. The image compression system based on partially reconfigurable and evolvable CNN according to claim 8, characterized in that, The dynamic block partitioning module includes a gradient calculation unit, a variance calculation unit, and a block size decision unit, which are used to calculate the local complexity and decide the block size in real time through integer operations.

Images